This is part of The Pile, a partial archive of some open source mailing lists and newsgroups.
To: "Eric Pretorious" <epretorious@hotmail.com>
Subject: Re: [svlug] Duplicating an entire HDD
Date: Thu, 13 Sep 2001 02:24:46 -0700
From: J C Lawrence <claw@kanga.nu>
On Wed, 12 Sep 2001 23:59:37 -0700
Eric Pretorious <epretorious@hotmail.com> wrote:
> I need to duplicate an entire 40GB ATA66 drive complete with its 5
> partitions and MBR [onto an identical 40GB ATA66 drive for use in
> an identical machine] so I tried using `dd if=/dev/hde
> of=/dev/hdg`. The machine boots from one of the partitions
> /dev/hde. The drive is mostly empty but has 5 ext2 partitions.
> The dd process consumed 30-35% of the CPU for 45 minutes before I
> canceled the process.
You can safely figure its going to take much of a day. Without
bothering to do the arithmetic start out with the optimal case of
35MB/sec and then realise that you're more likely the be seeing
30MB/sec.
> The output reported that ~66 million descriptors had been copied
> but I don't believe that I achieved what I set out to.
> How can I duplicate this drive?
dd over the MBR and everything up to and encluding the root
partition. Use {c]fdisk to fix the partition table as needed and
create the rest of the partitions. Put filesystems on the rest of
the partitions. Mount them, in their normal arrangement under
/mnt. Use
tar c /dir | ( cd /mnt/dir ; tar xf - )
to copy over the partition contents.
chroot to the root of your new tree. Use LILO with the chroot
option to install a new MBR if needed.
===
Date: Thu, 13 Sep 2001 08:13:14 -0500
From: Jeffrey Siegal <jbs@quiotix.com>
To: J C Lawrence <claw@kanga.nu>
Subject: Re: [svlug] Duplicating an entire HDD
J C Lawrence wrote:
> > I need to duplicate an entire 40GB ATA66 drive complete with its 5
> > partitions and MBR [onto an identical 40GB ATA66 drive for use in
> > an identical machine] so I tried using `dd if=/dev/hde
> > of=/dev/hdg`. The machine boots from one of the partitions
> > /dev/hde. The drive is mostly empty but has 5 ext2 partitions.
>
> > The dd process consumed 30-35% of the CPU for 45 minutes before I
> > canceled the process.
>
> You can safely figure its going to take much of a day. Without
> bothering to do the arithmetic start out with the optimal case of
> 35MB/sec and then realise that you're more likely the be seeing
> 30MB/sec.
Perhaps you should have done the arithmetic. A 40GB drive at 30MB/sec
will take about 1300 seconds, or between 1/4 and 1/2 hour.
I suspect that increasing the buffer size as kmself suggested would make
the process proceed much faster (i.e. closer to the above theoretical
rate) and use less CPU. Also, one would be well advised to make sure
the IDE driver is using one of the fast DMA modes, and also put the two
drives on different IDE channels.
===
To: Jeffrey Siegal <jbs@quiotix.com>
Cc: Eric Pretorious <epretorious@hotmail.com>, svlug@svlug.org
Subject: Re: [svlug] Duplicating an entire HDD
Date: Thu, 13 Sep 2001 09:39:06 -0700
From: J C Lawrence <claw@2wire.com>
On Thu, 13 Sep 2001 08:13:14 -0500
Jeffrey Siegal <jbs@quiotix.com> wrote:
> J C Lawrence wrote:
>> You can safely figure its going to take much of a day. Without
>> bothering to do the arithmetic start out with the optimal case of
>> 35MB/sec and then realise that you're more likely the be seeing
>> 30MB/sec.
> Perhaps you should have done the arithmetic. A 40GB drive at
> 30MB/sec will take about 1300 seconds, or between 1/4 and 1/2
> hour.
Uuurk. Reasons not to post after the cranium has shut down.
Experience tells me that copying a 10Gig drive takes in excess of 3
hours (translation: I waited 3 hours the first time, then went home,
after that I just do it over night).
> I suspect that increasing the buffer size as kmself suggested
> would make the process proceed much faster (i.e. closer to the
> above theoretical rate) and use less CPU.
Yup. I typically use 1Meg buffers.
===
From: Alvin Oga <alvin@planet.fef.com>
Subject: Re: [svlug] Duplicating an entire HDD -- cpu speed
To: claw@2wire.com (J C Lawrence)
Date: Thu, 13 Sep 2001 10:14:59 -0700 (PDT)
Cc: svlug@svlug.org
hi ya
> >> You can safely figure its going to take much of a day. Without
> >> bothering to do the arithmetic start out with the optimal case of
> >> 35MB/sec and then realise that you're more likely the be seeing
> >> 30MB/sec.
>
> > Perhaps you should have done the arithmetic. A 40GB drive at
> > 30MB/sec will take about 1300 seconds, or between 1/4 and 1/2
> > hour.
>
> Uuurk. Reasons not to post after the cranium has shut down.
>
> Experience tells me that copying a 10Gig drive takes in excess of 3
> hours (translation: I waited 3 hours the first time, then went home,
> after that I just do it over night).
the cpu speed makes a major difference if it can sustain its 20Mb/sec
transfer speed or not... most systems cannot sustain it...
celeron-500 vs p3-800 is like 3x-4x faster on p3 for the same disk
- ie 30 minute or so for stuff i copy from disk to disk ( 20-40gb )...
c ya
alvin
===
Date: Thu, 13 Sep 2001 14:16:04 -0500
From: Jeffrey Siegal <jbs@quiotix.com>
To: J C Lawrence <claw@2wire.com>, Eric Pretorious <epretorious@hotmail.com>,
Subject: Re: [svlug] Duplicating an entire HDD
Jeffrey Siegal wrote:
> J C Lawrence wrote:
> > Experience tells me that copying a 10Gig drive takes in excess of 3
> > hours (translation: I waited 3 hours the first time, then went home,
> > after that I just do it over night).
>
> 10 Gig drives are probably about 4-times slower than 40 Gig drives,
> assuming the same number of platters. (Essentially all of the recent
> increases in drive capacity have come from higher on-media bit density,
> which translates directly into higher media transfer rates.)
Duh. Now it is my turn to space on simple arithmetic. Such drives
probably differ in transfer rate by about a factor of four, but they
also (of course) differ by a factor of four in size. So the amount of
time taken to do a low level copy should be about the same.
I suspect your copy would have gone a lot faster if you had sufficient
CPU, had the IDE interface tweaked, and had used a sufficiently large
buffer size (sounds like you did this), as has been indicated by various
people on this thread.
===