This is part of The Pile, a partial archive of some open source mailing lists and newsgroups.
Date: Wed, 3 Jan 2001 14:28:15 -0800
To: svlug@svlug.org
Subject: Re: [svlug] backup solutions?
From: Rick Moen <rick@linuxmafia.com>
begin Mike O'Neill quotation:
> What are people using for backups in small shops?
Try MTX. Fast, GPLed, and supports many tape libraries.
http://mtx.sourceforge.net/
> BRU seems pretty good on most accounts but its GUI seems immature.
GUI for tape backup? How bizarre.
===
From: kmself@ix.netcom.com
Date: Wed, 3 Jan 2001 15:40:08 -0800
To: svlug@svlug.org
Subject: Re: [svlug] backup solutions?
Mike O'Neill (mikeo@redhillstudios.com) wrote:
> What are people using for backups in small shops? ArcServeIt looks awesome
> but is a bit pricey. Arkeia has a free version, seems to have lots of
> features, but lacks a disaster recovery. BRU seems pretty good on most
> accounts but its GUI seems immature.
tar
http://kmself.home.netcom.com/Linux/FAQs/backups.html
(Feedback welcomed).
===
Date: Wed, 3 Jan 2001 15:58:36 -0800
To: svlug@svlug.org
Subject: Re: [svlug] backup solutions?
From: Rick Moen <rick@linuxmafia.com>
begin kmself@ix.netcom.com quotation:
> tar
> http://kmself.home.netcom.com/Linux/FAQs/backups.html
> (Feedback welcomed).
But, Karsten....
The cpio manpage is _good_ for you. <ducks and runs>
Hmm, that reminds me: I'm going to have to test and see if my
enhanced version of the rpm2cpio script can still unpack the new-format
RPMs.
===
From: kmself@ix.netcom.com
Date: Wed, 3 Jan 2001 17:30:29 -0800
To: svlug@svlug.org
Subject: Re: [svlug] backup solutions?
Alvin Oga (alvin@planet.fef.com) wrote:
> > > What are people using for backups in small shops? ArcServeIt looks awesome
> > > but is a bit pricey. Arkeia has a free version, seems to have lots of
> > > features, but lacks a disaster recovery. BRU seems pretty good on most
> > > accounts but its GUI seems immature.
> > tar
> and find
I prefer just specifying the partition, by mountpoint:
tar cf /mount/point
..though I believe you can use 'find' to avoid recursing down
sub-mounted partitions. I've managed to avoid this. You can also use
exclusion arguments to tar.
> and pgp if ya paranoid... and coda and ???
I specifically discourage encryption of system backups. Security of
archival media is best handled on a physical, not encrypted, basis. If
you need to encrypt the underlying data, than do so in its primary
storage location, not by possibly breaking your entire backup, as any
single bitwise error in the archive may invalidate the entire backup.
See my page below for more info -- though I may want to emphasize this
section of coverage.
What specifically do you have in mind WRT coda?
> > http://kmself.home.netcom.com/Linux/FAQs/backups.html
===
From: Alvin Oga <alvin@planet.fef.com>
Subject: Re: [svlug] backup solutions? - differentials
To: rick@linuxmafia.com (Rick Moen)
Date: Thu, 4 Jan 2001 14:49:39 -0800 (PST)
Cc: alvin@planet.fef.com (Alvin Oga), svlug@svlug.org
> Rick Moen wrote:
>
> begin Alvin Oga quotation:
>
> > if a file on mon has AAAA
> > and on wed it has AAABBB ...
> > and on fri it has AAA and CCC
> >
> > than on sat we cannot restore differentials from wed to go back
> > to wed version ??
>
> For that, you'd use the Wednesday night differential tape.
hi ya rick
yes....but if wed night tape was bad for some reason....
you can't retreive wed version... even if one purposely
made a wed differential file ( foo.wed )
- thur differential wont pick up foo.wed since no differences
using incrementals since "full backups"... even if
tue and wed incrementals are bad... i can still backup or recover
foo.wed from thur or fri incrementals that was since full backup...
- if thur or fri incrementals did not pick up foo.wed...
( change it so that it does...
thats why i like incrementals vs differentials...
( though a week argument.....anything that can be done with incrementals
( can be done with differentials... to some extent...
differentials backup is good if "security" is a little issue
where they cannot get the whole file from backups...and space is an issue
problem w/ differential is.... the original file needs to exists to restore
or patch the diffs into the original to restore back to wed...
if the file was erased....differentials wont help...
however if wed incrementals/differentials was bad.... and the file was changed
on thru and fri... the only way to save wed incrementals or
differentials is to save daily 24hr incrementals on x media...
and save incremental since backup or past 2 full backups on y media
- all this is how important is their data and how
much time or $$ for backup space they willing to spend
( trivial to add one more crontab entry )
-- remember that full backups has a high probability that it will be bad
-- simply because you run out of disk space or tapes or tapes is bad...
-- or the "backup media" is simply not yet available... so you wait...
and there is a services that does do differential backups
over the net... ( not to risky since its differentials
- and some the save only the inode and assocaited data
- dont ask about url cause i forgot
( hopefully "differential backups" should be enough for searching
have fun
alvin
===
Date: Thu, 4 Jan 2001 15:07:04 -0800
From: Rick Moen <rick@linuxmafia.com>
To: svlug@svlug.org
Subject: Re: [svlug] backup solutions? - differentials
begin Alvin Oga quotation:
> yes....but if wed night tape was bad for some reason.... you can't
> retreive wed version... even if one purposely made a wed differential
> file ( foo.wed )
That is true. But that is also true if, instead of a differential tape,
you use an incremental tape, or a full backup tape, or a CDR, or a full
dump of the hard drive to a second hard drive.
_If_ the file existed only for a day, and thus the copy was made on the
backup medium used later that day -- regardless of the backup method --
then you simply need that backup record to be readable, or you're hosed.
> using incrementals since "full backups"... even if tue and wed
> incrementals are bad... i can still backup or recover foo.wed from
> thur or fri incrementals that was since full backup...
Nope. Per the facts you postulated in your example, the file was
already gone, when those incrementals got made: You said the file
existed on the fileserver for only one day.
> problem w/ differential is.... the original file needs to exists to
> restore or patch the diffs into the original to restore back to wed...
Um, no. I think you're misunderstanding the meaning of the term. I
_didn't say diff_, I said differential tape. Not the same concept at
all.
We seem to have failed to communicate, here.
===
Date: Thu, 4 Jan 2001 15:38:41 -0800
To: Alvin Oga <alvin@planet.fef.com>
Cc: svlug@svlug.org
Subject: Re: [svlug] backup solutions? - stuff
From: Chris Waters <xtifr@dsp.net>
On Wed, Jan 03, 2001 at 11:44:51PM -0800, Alvin Oga wrote:
> and if you move from one distro to another version of same or
> different distro... you dont even need /etc ???
Precisely, although it's still a good idea to back up /etc so you have
a reference copy around.
> just carefully cut and paste old /etc files as compared to the
> new silly/updated/changed formats...
Exactly, which is why you should still back it up. In fact, I didn't
restore /etc directly when I switched machines, because I wanted to
give the new machine a new hostname (among other things).
One thing that might be important to check if you're using more fancy
backup software than I used is whether the program *can* restore to a
new location. With tar, it's pretty straightforward -- just cd to a
parent directory, i.e. "/", then use relative pathnames, i.e. "tar cf
/dev/backupdevice etc". But I've seen commercial backup software
(especially GUI software) for other systems that insisted on restoring
files to the same absolute location. I don't know if any of the
heavyweight UNIX backup solutions suffer from this problem, but it's
something to check.
cheers
p.s. in case anyone hadn't noticed, I'm back from my Hawaiian
vacation, so re-hi to all my svlug friends. :)
===
Date: Thu, 4 Jan 2001 16:32:03 -0800
From: Rick Moen <rick@linuxmafia.com>
To: svlug@svlug.org
Subject: Re: [svlug] backup solutions? - differentials
begin Alvin Oga quotation:
> problem w/ differential is.... the original file needs to exists to restore
> or patch the diffs into the original to restore back to wed...
> if the file was erased....differentials wont help...
FYI, definition of the term "differential" is here:
http://www.estinc.com/strategies.php
===
Date: Wed, 3 Jan 2001 23:58:54 -0800
To: svlug@svlug.org
Subject: Re: [svlug] backup solutions?
From: Rick Moen <rick@linuxmafia.com>
begin Mike O'Neill quotation:
> Let's hear some comments about dump/restore, which as I understand it
> is essentially an easier way to use tar.
1. Works on entire filesystems. 2. No, you're wrong about it being an
easier way to use tar (or easier _than_ tar, for that matter).
> Another thread: if using tar or dump then how do you back up another
> linux box (client) that is not using NFS?
Obvious way is over an SSH tunnel.
===
Date: Thu, 4 Jan 2001 00:08:59 -0800 (PST)
From: Rafael <raffi@linwin.com>
To: Mike O'Neill <mikeo@redhillstudios.com>
Subject: Re: [svlug] backup solutions?
On Wed, 3 Jan 2001, Mike O'Neill wrote:
>
> > on Wed, Jan 03, 2001 at 02:13:37PM -0800, Mike O'Neill
> > (mikeo@redhillstudios.com) wrote:
> >> What are people using for backups in small shops? ArcServeIt looks awesome
> >> but is a bit pricey. Arkeia has a free version, seems to have lots of
> >> features, but lacks a disaster recovery. BRU seems pretty good on most
> >> accounts but its GUI seems immature.
> >
> > tar
> >
> > http://kmself.home.netcom.com/Linux/FAQs/backups.html
>
> Okay so tar seems fairly popular. My guess is that it's because it's as
> basic and barebones as a toothbrush. Let's hear some comments about
> dump/restore, which as I understand it is essentially an easier way to use
> tar.
Huh? dump is dump, tar is tar. Can't mix.
I use ufsdump/ufsrestore on Solaris with success, but tar for Linux boxes.
>
> Another thread: if using tar or dump then how do you back up another linux
> box (client) that is not using NFS?
rsync -avz www:/home/httpd/* /backup/httpd
then tar or whatever for /backup/httpd. Good for slower connections or
huge file systems.
My favorite:
(ssh www.mydomain.com 'cd /home/httpd;tar cfp - html')|(cd /backup;tar xvfp -)
This one asks for the password if you have it setup the right way, i.e.
more secure. For scripts you need to go around that.
rsh instead of ssh is OK (?) for LAN if the rest is not much more secure
(NIS, rlogin, etc.)
There are variations on this theme of course. I backup www servers over
distance that way.
What would life be without pipes?
===
Date: Thu, 4 Jan 2001 00:52:19 -0800 (PST)
From: Rafael <raffi@linwin.com>
To: World Domination <svlug@svlug.org>
Subject: Re: [svlug] backup solutions? - tell me more ???
On Thu, 4 Jan 2001, Alvin Oga wrote:
>
> hi ya karsten..
>
> ( just my silly comments follows...
>
.........................
>
> ... what dont i understand ??? - using the same "line of thought"
>
> ... remember when the somebody walked off with 30,000 visa numbers ???
> from a supposedly high security area ?
That's the most important point of this discussion. It used to be, huge
computers protected by armed guards, tapes stored in vaults. Now bad guys
from Russia, Romania and Middle East can "walk into store" at any time and
do more damage to your business than earthquake or flood combined.
> ... remember our wrongly(?) accused incidents at sandia/lbl/??
>
> ...
> ... remember when the 10 yr old kid sat in front of your pc ???
> ... that made you nervous...knowing that they know about unix commands
> ...
>
>
> physical security is the weakest link....of all security precautions
As mentioned some time back, the safest place to keep drives is behind
the copy machine. Nobody expects them there.
>
>
> > An insulated
> > safe for on-sight backups. A sealed lockbox transported to a secure,
> > inert-atmosphere, armored storage site (e.g.: DataStor) for off-site.
That's reasonable precausionary measure. Some companies that have offices
in different parts of the country backup across the net which is another
way to do it. And of course, there are dot.com services for online
backups. Not sure what happens when they "go under".
>
> if ya think thats secure..... telnet is secure tooo....if you wanna
> try using telnet.... up to some limits ... offsite tape backups is
> highly insecure in my opinion...
>
> dataStore is good for things like ..."in case there is a fire"
> not backups....cause it takes um a day or two ot get the flimsy "lockbox"
It usualy takes about a week to get the files back, much longer than some
managers are willing to wait for you to fix the mistake.
> back to ya....and so allthe affected employee might as well go home
> till the tape arrives ....and gets it loaded....and clean the tape head...et al...
>
>
> and if protecting against fire or stuff like that is important
> to have offsite backups...
>
> ...store office backups at home .... and store home backups at the office...
> why trust it to 3rd parties ???
OK for small business but not for companies over 10 people I think.
Storing your private files at work might cause legal trouble. Store your
backups at friends or relatives place or just put it up on the net for
others to back it up (ftp) ...
>
===
Date: Thu, 4 Jan 2001 01:04:38 -0800
To: World Domination <svlug@svlug.org>
Subject: Re: [svlug] backup solutions? - tell me more ???
From: Rick Moen <rick@linuxmafia.com>
begin Rafael quotation:
>> dataStore is good for things like ..."in case there is a fire" not
>> backups....cause it takes um a day or two ot get the flimsy "lockbox"
>
> It usualy takes about a week to get the files back, much longer than
> some managers are willing to wait for you to fix the mistake.
The original assertion about DataStor is simply wrong: It's a tradeoff
of time vs. convenience vs. expense, when you're using them -- your
choice. If you suddenly need a box that they have in cold storage, you
can call and have it delivered the next business day -- or the same day
if you call really early. Or you can pay them a whopping fee and have
it driven over right away. Or you can take time away from work to drive
to the storage building, sign for the lockbox, and get your tapes
immediately thereafter.
===
Date: Thu, 4 Jan 2001 01:46:47 -0800
From: Rick Moen <rick@linuxmafia.com>
To: svlug@svlug.org
Subject: Re: [svlug] backup solutions? - tell me more ???
begin Alvin Oga quotation:
> yeah...but than again...we are talking about backups for users vs
> corps... most users dont have tape libraries ??? or mammath drives
> etc..etc.... mammoths are nice drives and nice tapes...but....
Well, that 500GB hard disk size figure was from your post.
But if you're talking about an individual with a 36.7 GB SCSI drive
($590, wow!) he'd probably be best served by a DDS3 DAT drive. Cheap,
effective.
> and i dont trust tapes...especially not a set of ummm to make a
> backup of 500Gb system.... lose a tape and *ooopss*...
Very rare with decent drive maintenance, and enforcement of a
tape-retirement scheme. Besides, that's why you keep multiple
_generational_ full backups plus daily differentials. Reduces single
points of failure and gives you fallbacks.
> and yeah...big difference between archives for the lawyers to sift
> thru....vs "backups" and restor the disk or the file cause somebody
> did something...
To elaborate on what I was saying:
Archival storage: Moving a body of data off of primary storage
to less-available media where it still might be randomly accessed
and searched at high speed, but probably with lower convenience. E.g.,
burn a CDR of all files relating to a corporate project, after it's
been closed out and billed to customers, then delete the project tree
from the company's live fileserver. Great Murphy almost guarantees that
your staff will soon thereafter plead to recover some of those files,
but won't be able to remember the exact pathnames to that data, as it
was stored on the fileserver. With tapes, the sysadmins would probably
have an excruciating series of iterative wild-goose chases, as the
clueless staffers requested restores of numerous wrong files, before
hitting on the right one. By contrast, if using archival-type media
such as CDRs, the staff can sign out the CDR from the archive
collection, search it on his own time, and return it when he's done,
saving sysadmin time and frustration.
Backup storage: Both full and incremental (or, better, differential)
snapshots of system state at intervals, generated at high speed with
maximal automation, with the aim of ensuring recovery from mass data
loss or file corruption. Emphasis on high data-insurance value, at
the expense of convenience / labour-intensiveness for casual restore
jobs. (Classically, this means DAT, DLT, AIT tape drives, and larger
drives or libraries where needed.)
===
From: kmself@ix.netcom.com
Date: Thu, 4 Jan 2001 01:58:16 -0800
To: svlug@svlug.org
Subject: Re: [svlug] backup solutions? - tell me more ???
Alvin Oga (alvin@planet.fef.com) wrote:
> we disagree...so lets just agree to disagree and drop
> subject about compressed or encrypted....makes no difference
> to me.... a dropped bit will simply make life more difficult
> to recover...... and i prefer to compressed backups to conserve space....
Much snippage, much of which is in agreement.
We'll disagree on crypt and compr.
Incrementals I actually *do* buy into. *When* they're suggested by data
loads, tape capacities, and infrastructure to manage the several
restoresets required to recover an archive. For a simple system, such
as mine, and possibly many small business server needs, I don't think
they're appropriate given the complexity they introduce. For a remote
webserver with a weekly or monthly tape rotation, incrementals (and
network backups to a remote site) start to look much more attractive.
Rick Moen's notes on archival vs. recovery are also well taken.
Tools, fitted to needs, and a very healthy dose of keep it simple,
stupid.
===
Date: Thu, 4 Jan 2001 09:05:46 -0800
To: kmself@ix.netcom.com
Cc: svlug@svlug.org
Subject: Re: [svlug] backup solutions? - tell me more ???
From: Rick Moen <rick@linuxmafia.com>
begin kmself@ix.netcom.com quotation:
> Incrementals I actually *do* buy into. *When* they're suggested by data
> loads, tape capacities, and infrastructure to manage the several
> restoresets required to recover an archive.
Use differentials instead of incrementals.
Let's say you do a full backup starting every Friday night. Some people
then supplement that with incremental tapes for Monday, Tuesday,
Wednesday, and Thursday. The problem with that comes, as you suggest,
if you end up having to revert to a Thursday backup (worst case):
You'd then have to restore the prior Friday tape, then restore each of
the four incrementals. Lots of restore time, and with lots of single
points of failure. Restoring just a file might be equally painful and
failure-prone.
So, instead of incrementals on Monday - Thursday, make each tape a
differential. That is, each includes all files modified not since the
day before, but rather all changes since the last Friday full backup.
Thus, the Monday - Thursday tapes no longer rely on one another. And,
the system is simpler and more foolproof.
===
Date: Thu, 4 Jan 2001 23:58:37 -0800 (PST)
From: Rafael <raffi@linwin.com>
To: World Domination <svlug@svlug.org>
Subject: Re: [svlug] backup solutions? - tell me more ???
On Thu, 4 Jan 2001, Rick Moen wrote:
> begin Rafael quotation:
>
> >> dataStore is good for things like ..."in case there is a fire" not
> >> backups....cause it takes um a day or two ot get the flimsy "lockbox"
> >
> > It usualy takes about a week to get the files back, much longer than
> > some managers are willing to wait for you to fix the mistake.
>
> The original assertion about DataStor is simply wrong: It's a tradeoff
> of time vs. convenience vs. expense, when you're using them -- your
> choice. If you suddenly need a box that they have in cold storage, you
> can call and have it delivered the next business day -- or the same day
> if you call really early. Or you can pay them a whopping fee and have
> it driven over right away. Or you can take time away from work to drive
> to the storage building, sign for the lockbox, and get your tapes
> immediately thereafter.
That was not the case where I used to work. Tapes were transported out of
the area and it would take at least 3 work days to get them back. We did
not use DataStor though and did not keep tapes in the car trunks. The
amount of tapes was huge and there was so much data that not all was
backed up even though we used some of the largest libraries (cabinets)
available.
No library can save the amount of data programers in semiconductor
industry can produce in a week! There is simply no time to backup data on
a large number of cluster systems. So there are cases where complete
backup is impossible. Fortunately RAID systems and reliable disk drives
that are replaced/upgraded on time save the day.
===
Date: Fri, 5 Jan 2001 11:18:27 -0800
To: World Domination <svlug@svlug.org>
Subject: Re: [svlug] backup solutions? - tell me more ???
From: Rick Moen <rick@linuxmafia.com>
begin Rafael quotation:
> http://www.networkappliance.com is my final answer. [when somebody else
> comes up with $$$$]
If a NetApp is the answer, then the _question_ must be coming from
someone truly demented. (Aside from the deficiencies of the product and
the company[1], which we could explore for some days, I hope I don't
have to detail the differences between network storage and backup?)
[1] Seldom have I had a more-satisfying moment than the one at
$PRIOR_FIRM when I notified the smarmy NetApp sales rep that I'd
reversed my predecessor's decision, and NetApp wasn't going to be able
to raid our company bank account, after all. The fireworks were most
gratifying.
===
Date: Fri, 5 Jan 2001 11:44:08 -0800
From: "Jeremy D. Zawodny" <jzawodn@yahoo-inc.com>
To: hvrietsc@yahoo.com
Cc: svlug@svlug.org
Subject: Re: [svlug] rsync and backup anyone?
On Fri, Jan 05, 2001 at 11:17:43AM -0800, hvrietsc@yahoo.com wrote:
> I must admit i have not read all the many backup emails flying
> around but did anyone mention the use of rsync for backups yet?
>
> (if yes then delete this message, if no keep reading if you want)
>
> rsync will keep two (or more computers) in sync and will only ship
> differences across the wire so its real quick.
>
> I know this is not _real_ backup, but having the same data on two or
> more machines does give some safety. I regularly run rsync between
> my desktop and laptop so that i have a full copy of my laptop (/etc
> /root /home /etc) just in case. after the initial rsync all
> incrementals are quick (due to rsyncs incremental algorithms).
Yep. I do something very similar at home. I have a couple of spare
filesystems on large hard disks. One is mounted as /backup/home and
another as /backup/root. I have a cron job in place to periodically
rsync so that if the real thing dies, I don't lose much data.
Hey hard drives are cheap... why not? :-)
===