This is part of The Pile, a partial archive of some open source mailing lists and newsgroups.
Date: Fri, 25 Jan 2002 11:28:40 -0500
From: "Justin F. Knotzke" <shampoo@cam.org>
To: Svlug <svlug@svlug.org>
Subject: [svlug] ext3 and dd disk cloning
Hi,
It appears you can't "clone" a disk in ext3 because the journal has
a unique identifier and the kernel will not allow you to mount a dd'd
disk. I might have to create the partitions, create the filesystem and
then tar or cpio the files over.
This is a bit of a pain as compared to just doing a dd from one hd
to another (same disk, same geometry etc).
Does anyone know a way to dd a ext3 drive? I'm going to try
recreating the journal after the dd but I'm not sure if that will work.
Thanks
Justin.
===
Date: Fri, 25 Jan 2002 08:50:28 -0800
From: Marc MERLIN <marc_news@vasoftware.com>
To: Svlug <svlug@svlug.org>
Subject: Re: [svlug] ext3 and dd disk cloning
On Fri, Jan 25, 2002 at 11:28:40AM -0500, Justin F. Knotzke wrote:
> It appears you can't "clone" a disk in ext3 because the journal has
> a unique identifier and the kernel will not allow you to mount a dd'd
> disk. I might have to create the partitions, create the filesystem and
> then tar or cpio the files over.
Mmmh, I'm not convinced.
I don't think the kernel knows that you have duplicated the disk and are
trying to mount it on another machine.
(it'd have to create a hash per machine, just like XP)
If this were really the case however, you could remove the journal with:
tune2fs -O ^has_journal /dev/part
which effectively turns your ext3 partition into an ext2 one.
Marc
===
Date: Fri, 25 Jan 2002 11:57:49 -0500
From: "Justin F. Knotzke" <shampoo@cam.org>
To: Svlug <svlug@svlug.org>
Subject: Re: [svlug] ext3 and dd disk cloning
On Fri, Jan 25, 2002 at 08:50:28AM -0800, Marc MERLIN wrote:
> Mmmh, I'm not convinced.
> I don't think the kernel knows that you have duplicated the disk and are
> trying to mount it on another machine.
> (it'd have to create a hash per machine, just like XP)
http://linux.lexilog.org.uk/fs.html
Not sure if that's the case but I do know that when I do clone a
drive using dd and I put the cloned drive into another machine the
kernel boots and just as it attempts to mount the drive, I get a login
prompt and it will not allow me to login. There are no errors. This has
happened twice. I can boot the machine using the original drive and
mount the cloned drive and all the data appears to be there but I cannot
boot with that cloned drive.
> If this were really the case however, you could remove the journal with:
> tune2fs -O ^has_journal /dev/part
> which effectively turns your ext3 partition into an ext2 one.
Ok, I will try that after the clone.
Thanks Marc.
Justin.
--
Justin F. Knotzke
shampoo@cam.org
pgp public key http://www.shampoo.ca/pubkey.html
===
Date: Fri, 25 Jan 2002 09:07:03 -0800
From: Marc MERLIN <marc_news@vasoftware.com>
To: Svlug <svlug@svlug.org>
Subject: Re: [svlug] ext3 and dd disk cloning
On Fri, Jan 25, 2002 at 11:57:49AM -0500, Justin F. Knotzke wrote:
> On Fri, Jan 25, 2002 at 08:50:28AM -0800, Marc MERLIN wrote:
>
> > Mmmh, I'm not convinced.
> > I don't think the kernel knows that you have duplicated the disk and are
> > trying to mount it on another machine.
> > (it'd have to create a hash per machine, just like XP)
>
> http://linux.lexilog.org.uk/fs.html
It says that if you clone the disk, you should expect problems if you mount
both disks on the same machine. I can believe that.
> Not sure if that's the case but I do know that when I do clone a
> drive using dd and I put the cloned drive into another machine the
> kernel boots and just as it attempts to mount the drive, I get a login
> prompt and it will not allow me to login. There are no errors. This has
Ok, so this confirms that it's very unlikely that ext3 is ihvolved, from
what you say, everything mounts fine.
Try to boot with
init single
or
init=/bin/bash ( + "mount -wno remount /; mount -a" when you get a prompt)
and work from there, and look at your logs to see if they say why you
couldn't log in.
> happened twice. I can boot the machine using the original drive and
> mount the cloned drive and all the data appears to be there but I cannot
> boot with that cloned drive.
Not to be picky, but it does "boot". You are just having some problem that
prevents you from login in.
Make sure that when you do
dd if=/dev/sda of=/dev/sdb bs=1M
both /dev/sda and /dev/sdb are fully unmounted (use a boot CD or floppy)
> > If this were really the case however, you could remove the journal with:
> > tune2fs -O ^has_journal /dev/part
> > which effectively turns your ext3 partition into an ext2 one.
>
> Ok, I will try that after the clone.
Note that I very much doubt it will make a difference, but you're welcome to
try.
===
Date: Fri, 25 Jan 2002 12:26:59 -0500
From: "Justin F. Knotzke" <shampoo@cam.org>
To: Svlug <svlug@svlug.org>
Subject: Re: [svlug] ext3 and dd disk cloning
On Fri, Jan 25, 2002 at 09:07:03AM -0800, Marc MERLIN wrote:
> Ok, so this confirms that it's very unlikely that ext3 is ihvolved, from
> what you say, everything mounts fine.
> Try to boot with
> init single
> or
> init=/bin/bash ( + "mount -wno remount /; mount -a" when you get a prompt)
>
> and work from there, and look at your logs to see if they say why you
> couldn't log in.
I've been doing dd's with these very same machine for a few months
without a problem. Since converting to ext3 however, the problem I
mentioned above has surfaced.
I am doing a brand new clone and then I'll try what you mentioned.
> Note that I very much doubt it will make a difference, but you're welcome to
> try.
It didn't. Kernel panic..
Let's see what happens after this dd and after I
boot with init single.
===
Date: Fri, 25 Jan 2002 10:32:39 -0800
From: Marc MERLIN <marc_news@vasoftware.com>
To: Svlug <svlug@svlug.org>
Subject: Re: [svlug] ext3 and dd disk cloning
On Fri, Jan 25, 2002 at 12:26:59PM -0500, Justin F. Knotzke wrote:
> On Fri, Jan 25, 2002 at 09:07:03AM -0800, Marc MERLIN wrote:
>
> > Ok, so this confirms that it's very unlikely that ext3 is ihvolved, from
> > what you say, everything mounts fine.
> > Try to boot with
> > init single
> > or
> > init=/bin/bash ( + "mount -wno remount /; mount -a" when you get a prompt)
> >
> > and work from there, and look at your logs to see if they say why you
> > couldn't log in.
>
> I've been doing dd's with these very same machine for a few months
> without a problem. Since converting to ext3 however, the problem I
> mentioned above has surfaced.
I can see dding an live ext2 filesystem working, by sheer luck, and dding an
live ext3 filesystem possibly causing problems like what you describe, hence
me making sure that the dd wasn't done while the source filesystems were
mounted (i.e. you shouldn't do the dd with the system running from the drive
you are trying to dd)
===
Date: Fri, 25 Jan 2002 11:44:54 -0800
From: Rafael <raffi@ark.linwin.com>
To: Svlug <svlug@svlug.org>
Subject: Re: [svlug] ext3 and dd disk cloning
On Fri, Jan 25, 2002 at 11:28:40AM -0500, Justin F. Knotzke wrote:
>
> Hi,
>
> It appears you can't "clone" a disk in ext3 because the journal has
> a unique identifier and the kernel will not allow you to mount a dd'd
> disk. I might have to create the partitions, create the filesystem and
> then tar or cpio the files over.
>
> This is a bit of a pain as compared to just doing a dd from one hd
> to another (same disk, same geometry etc).
>
> Does anyone know a way to dd a ext3 drive? I'm going to try
> recreating the journal after the dd but I'm not sure if that will work.
Using dd to clone large disk drives is a waste of time and not appropriate
IMO. In my experience it's faster to clone a partition table, mount
partitions and tar or cpio files from one drive to another. This method
would let you copy from a working disk even though that might not be
desirable in some cases.
Any other way is a waste of time on copying otherwise unused space. dd
preserves fragmentation and other undesirable effects (open files
issues,etc.) so other methods are preffered. dd is good to make exact copy
of the drive for forensic purposes.
===
Date: Fri, 25 Jan 2002 11:48:53 -0800
From: Marc MERLIN <marc_news@vasoftware.com>
To: Rafael <raffi@ark.linwin.com>
Cc: Svlug <svlug@svlug.org>
Subject: Re: [svlug] ext3 and dd disk cloning
On Fri, Jan 25, 2002 at 11:44:54AM -0800, Rafael wrote:
> > Does anyone know a way to dd a ext3 drive? I'm going to try
> > recreating the journal after the dd but I'm not sure if that will work.
>
> Using dd to clone large disk drives is a waste of time and not appropriate
> IMO. In my experience it's faster to clone a partition table, mount
> partitions and tar or cpio files from one drive to another. This method
> would let you copy from a working disk even though that might not be
> desirable in some cases.
It's a all a matter of perspective.
I value my time a lot more than care about how long it takes to dd the
drive.
If you need the copy right now, sure copying partitions _can_ go faster (not
necessarily, because copying files one per one is slower than copying the
disk blocks).
If you don't care how long it takes as long as it get done, and you don't
have to babysit it, dding the whole drive is better.
===
Date: Fri, 25 Jan 2002 16:01:56 -0500
From: "Justin F. Knotzke" <shampoo@cam.org>
To: Svlug <svlug@svlug.org>
Subject: Re: [svlug] ext3 and dd disk cloning
On Fri, Jan 25, 2002 at 10:32:39AM -0800, Marc MERLIN wrote:
> I can see dding an live ext2 filesystem working, by sheer luck, and dding an
> live ext3 filesystem possibly causing problems like what you describe, hence
> me making sure that the dd wasn't done while the source filesystems were
> mounted (i.e. you shouldn't do the dd with the system running from the drive
> you are trying to dd)
Guilty.
I was doing my dd's by booting into single user mode.
I just finished a dd after booting using a CD and the dd worked
perfectly with a ext3 filesystem.
Thanks again.
===
Date: Fri, 25 Jan 2002 13:28:45 -0800
From: Marc MERLIN <marc_news@vasoftware.com>
To: Svlug <svlug@svlug.org>
Subject: Re: [svlug] ext3 and dd disk cloning
On Fri, Jan 25, 2002 at 04:01:56PM -0500, Justin F. Knotzke wrote:
> On Fri, Jan 25, 2002 at 10:32:39AM -0800, Marc MERLIN wrote:
>
> > I can see dding an live ext2 filesystem working, by sheer luck, and dding an
> > live ext3 filesystem possibly causing problems like what you describe, hence
> > me making sure that the dd wasn't done while the source filesystems were
> > mounted (i.e. you shouldn't do the dd with the system running from the drive
> > you are trying to dd)
>
> Guilty.
>
> I was doing my dd's by booting into single user mode.
You think can do this if you boot with init=/bin/bash, which leaves the root
filesystem mounted read only, and nothing mounted read/write
But with a boot in single user mode, you have several partitions that are
mounted read/write, and that's bad if you do a dd :-)
===
Date: Sat, 26 Jan 2002 14:42:09 -0700 (MST)
From: "Karl F. Larsen" <k5di@zianet.com>
To: "Justin F. Knotzke" <shampoo@cam.org>
Subject: Re: [svlug] ext3 and dd disk cloning
On Fri, 25 Jan 2002, Justin F. Knotzke wrote:
>
> Hi,
>
> It appears you can't "clone" a disk in ext3 because the journal has
> a unique identifier and the kernel will not allow you to mount a dd'd
> disk. I might have to create the partitions, create the filesystem and
> then tar or cpio the files over.
You are correct. The dd disk cloning is not going to work with the
new XFS file system. You need to make the new file system which will have
a new address given it by the kernel, and then you can mount it and I have
used cp -a with entire huge directories like /usr/ with sucess.
Read http://linux.lexilog.org.uk/fs.html which is a very clear explanation
of this subject.
--
Yours Truly,
- Karl F. Larsen, k5di@arrl.net (505) 524-3303 -
http://www.zianet.com/k5di/
===
Date: Sat, 26 Jan 2002 15:02:33 -0800
From: Jeffrey Siegal <jbs@quiotix.com>
To: "Karl F. Larsen" <k5di@zianet.com>
Subject: Re: [svlug] ext3 and dd disk cloning
Karl F. Larsen wrote:
> Read http://linux.lexilog.org.uk/fs.html which is a very clear explanation
> of this subject.
That article has at least one serious error:
"A journaling filesystem avoids this problem by keeping track of all
changes to the metadata (and to some extent to the data itself) as they
happen, in such a way that the system can always be returned to a
consistent state if a disk update is abnormally interrupted. A small
amount of data may well be lost in such an event, but the consequences
are not catastrophic: the last second or so's activity before the power
suddenly went off may have gone, but the state of the disks when the
machine is brought up again will be what it would have been if the
system had been shut down in an orderly way that second or so prior to
the actual abrupt switch-off."
This is not correct. A journaling filesystem ensures that the
filesystem itself can be returned to a consistent state, but not the
*system*. Data within files (and also an expected relationship between
different files) may still be rendered inconsistent and unusable, unless
the applications themselves are also written in a transactional manner.
===
Date: Sat, 26 Jan 2002 16:33:13 -0700 (MST)
From: "Karl F. Larsen" <k5di@zianet.com>
To: Jeffrey Siegal <jbs@quiotix.com>
Subject: Re: [svlug] ext3 and dd disk cloning
Yes your right and the author covers the serious *system* problem by
saying "a few bytes of data may be lost".
The main thing different is that with a power down you come right back up
without the old ext2 10 minute dance with fixing the file system. That
something might not work anymore is sure true.
On Sat, 26 Jan 2002, Jeffrey Siegal wrote:
> Karl F. Larsen wrote:
>
> > Read http://linux.lexilog.org.uk/fs.html which is a very clear explanation
> > of this subject.
>
> That article has at least one serious error:
>
> "A journaling filesystem avoids this problem by keeping track of all
> changes to the metadata (and to some extent to the data itself) as they
> happen, in such a way that the system can always be returned to a
> consistent state if a disk update is abnormally interrupted. A small
> amount of data may well be lost in such an event, but the consequences
> are not catastrophic: the last second or so's activity before the power
> suddenly went off may have gone, but the state of the disks when the
> machine is brought up again will be what it would have been if the
> system had been shut down in an orderly way that second or so prior to
> the actual abrupt switch-off."
>
> This is not correct. A journaling filesystem ensures that the
> filesystem itself can be returned to a consistent state, but not the
> *system*. Data within files (and also an expected relationship between
> different files) may still be rendered inconsistent and unusable, unless
> the applications themselves are also written in a transactional manner.
>
>
>
--
Yours Truly,
- Karl F. Larsen, k5di@arrl.net (505) 524-3303 -
http://www.zianet.com/k5di/
===