This is part of The Pile, a partial archive of some open source mailing lists and newsgroups.
Date: Mon, 5 Feb 2001 21:56:08 -0800
To: svlug@svlug.org
Subject: Re: [svlug] Security question: read-only drive
From: Rick Moen <rick@linuxmafia.com>
begin Drew Bertola quotation:
> I want to lock down a system so that no one can install a root kit in
> any useful way. I know it's possible to boot of of read-only media,
> and I have a SCSI HDD which can be set read-only.
>
> Does anyone have experience with this? What would my directory layout
> be? I know I have to put /etc on the root partition, and /etc has to
> be writable (to accomodate mtab, ssh_random_seed, etc.), so what would
> I want to put on the read-only disk?
Relocate those files _off_ /etc. (Try using symlinks.) /etc is one of
the filesystems you'll want to mount read-only for a high-security
system (along with /usr other than /usr/local). To the extent your
distribution needs (by default) to write to files there, its
directory-layout design is broken.
Suggestion: Connect the SCSI read-only jumper to the (unused) "turbo"
switch on your system's front-panel. That way, you can hardware-protect
the static directory tree in a way no remote intruder can overcome,
while still retaining the ability to toggle writable access from the
console. (You can even mount some of those filesystems read-write at
that point, if you wish, such as /etc.)
The directory trees that are supposed to have basically static contents
(and thus be mountable read-only) are documented in the Filesystem
Hierarchy Standard (FHS) document, http://www.pathname.com/fhs/ . It's
worthwhile reading.
===
From: Drew Bertola <drew@drewb.com>
Date: Tue, 6 Feb 2001 08:19:28 +0000 ()
To: Wayne Earl <wayne@qconcepts.net>
Cc: Drew Bertola <drew@drewb.com>, svlug@svlug.org
Subject: Re: [svlug] Security question: read-only drive
Not if the C code to control the switch is binary only (locally) and
protected by hardcoded password (argv[1]) with an algorithm like MD5
to keep it safe within the binary. (Not to mention the obscurity
involved).
Wayne Earl writes:
> On Tue, 6 Feb 2001, Drew Bertola wrote:
> > One thing that would be quite useful is to have a remotely controlled
> > switch since the box will be colocated. I'll need to cook up
> > something. Otherwise, I'll put a switch on the enclosure.
> Drew:
>
> Wouldn't a remotely controlled switch defeat the purpose? If you can
> switch the HDD to a writable state remotely, than someone else could as
> well.
>
> It is an interesting idea, though.
At least a random intruder would have a tough time enabling it.
Imagine /usr is read-only. I issue:
# /usr/bin/unlock-drive seCret94
unlock-drive then performs an MD5 hash on "seCret94" and if that
matches the hardcoded MD5 hash of "seCret94" stored within
unlock-drive, the program proceeds to open the switch for rw access to
the drive. If argv[1] isn't exact, the hash fails and the program
fails w/ a "Get Stuffed!" message.
A seperate "lock-drive" program, not requiring a password, can then be
run to secure the drive. It would also be run during bootup.
What's great is that nobody can replace "unlock-drive" without knowing
the password. Even if they do, I could have a NOC flip a manual
overide switch.
What do you think?
===
Date: Tue, 6 Feb 2001 04:51:18 -0800 (PST)
From: "Dagmar d'Surreal" <dagmar@dsurreal.org>
To: Drew Bertola <drew@drewb.com>
Subject: Re: [svlug] Security question: read-only drive
On Tue, 6 Feb 2001, Drew Bertola wrote:
> I want to lock down a system so that no one can install a root kit in
> any useful way. I know it's possible to boot of of read-only media,
> and I have a SCSI HDD which can be set read-only.
>
> Does anyone have experience with this? What would my directory layout
> be? I know I have to put /etc on the root partition, and /etc has to
> be writable (to accomodate mtab, ssh_random_seed, etc.), so what would
> I want to put on the read-only disk?
Whoohoo! I never could get anyone in Nashville interested in trying
this. I've done it a few times, and recommended it as a fast way to raise
the bar in a harsh fashion on script kiddies. There are plenty of people
out there who can do live binary patching in memory, but that's plenty in
the global sense.
There are also small (but useable) flash disks that can accommodate a
modest filesystem for holding the OS as well made by Sandisk that a few
vendors are using in their turnkey products. These are a little pricey
for expermenting with compared to $40 for an old full-height Seagate
drive, but nice for a finished product.
Filesystems you will need to keep writeable are /tmp, /var, and /etc (it
can be eliminated as writeable, but it's a major pain, better to rebuild
the important stuff so that it looks for it's configuration files in
*non*writable places other than /etc, like /usr/etc). Everything under
/usr can be quite non-writeable (samba will need a little special
attention) and everything in /lib *should* be on a non-writeable
partition. The biggest problem you'll run into will be dealing with /etc
and /lib, and I never came up with what could be construed as a "clean"
solution (ugly ugly source file hacking and slashing). You will likely
wind up recompiling most of what passes for your start up binaries (some
people don't normally make /bin/sh wholly static. Hello /lib issues.) to
suit your own needs, since most distros aren't quite ready to be dropped
on partially non-writeable hardware.
About the only thing specific I can recommend is patience. I managed to
get one up and working with the majority of everything in non-writeable
space, but didn't have a journaling filesystem at the time that could
handle having the power killed on the machine on a regular basis because
the only way to be 100% sure the integrity checker gets run when it should
is to cold boot repeatedly (goodbye massive uptimes). I didn't go much
further than the experimental machine because I estimated I could spend
probably half the time straighening out the rest of the writeable parts
and come back to the other bits later. I also had to make the hard
decision of doing some bizarre (for a machine with hard drives, yes) stuff
with ramdisks to get a little workspace and invoking mke2fs to simply wipe
out and rebuild the writeable parts on each boot, meaning no state
information would be preserved between boots. A mostly non-writeable
configuration is still fairly good for making a firewall, and that sort of
thing will fit on even a small non-writeable drive, with configuration
files fitting nicely on a non-writeable (but more easily changeable to a
writeable state) floppy disk.
Last note, LS-120 drives are handy for some things other than making
Maxell's advertising department look bad. ;)
===
Date: Tue, 6 Feb 2001 04:53:19 -0800 (PST)
From: "Dagmar d'Surreal" <dagmar@dsurreal.org>
To: Aaron Lehmann <aaronl@vitelus.com>
Subject: Re: [svlug] Security question: read-only drive
On Mon, 5 Feb 2001, Aaron Lehmann wrote:
> On Tue, Feb 06, 2001 at 03:29:10AM +0000, Drew Bertola wrote:
> > Does anyone have experience with this? What would my directory layout
> > be? I know I have to put /etc on the root partition, and /etc has to
> > be writable (to accomodate mtab, ssh_random_seed, etc.), so what would
> > I want to put on the read-only disk?
>
> Preferably, nothing. I was considering a read only /usr until I
> realized how difficult that would make installing security fixes, or
> other updates. Ironically, you're probably more secure with a
> read-write /usr.
You *should* have /usr read-only, or stick everything important to your
integrity checker *elsewhere*. Prepare to see unhappy things when you
start turning ldd loose on your current core system binaries.
===
Date: Tue, 6 Feb 2001 05:06:32 -0800 (PST)
From: "Dagmar d'Surreal" <dagmar@dsurreal.org>
To: Rick Moen <rick@linuxmafia.com>
Subject: Re: [svlug] Security question: read-only drive
On Mon, 5 Feb 2001, Rick Moen wrote:
> begin Drew Bertola quotation:
> > I want to lock down a system so that no one can install a root kit in
> > any useful way. I know it's possible to boot of of read-only media,
> > and I have a SCSI HDD which can be set read-only.
> >
> > Does anyone have experience with this? What would my directory layout
> > be? I know I have to put /etc on the root partition, and /etc has to
> > be writable (to accomodate mtab, ssh_random_seed, etc.), so what would
> > I want to put on the read-only disk?
>
> Relocate those files _off_ /etc. (Try using symlinks.) /etc is one of
> the filesystems you'll want to mount read-only for a high-security
> system (along with /usr other than /usr/local). To the extent your
> distribution needs (by default) to write to files there, its
> directory-layout design is broken.
I recompiled a *whole* lotta stuff to go looking in /usr/etc for it's
configuration files. init was the highly unamusing one to tamper with,
since it's the part that almost no one works on unless they are 100% sure
of what they're doing, so the documentation is pretty thin to put it
mildly. (Almost as much fun as trying to rebuild ld86 and as86 on a
strange configuration)
> Suggestion: Connect the SCSI read-only jumper to the (unused) "turbo"
> switch on your system's front-panel. That way, you can hardware-protect
> the static directory tree in a way no remote intruder can overcome,
> while still retaining the ability to toggle writable access from the
> console. (You can even mount some of those filesystems read-write at
> that point, if you wish, such as /etc.)
A tapered reamer and a cheap switch from RatShack works, too. :)
> The directory trees that are supposed to have basically static contents
> (and thus be mountable read-only) are documented in the Filesystem
> Hierarchy Standard (FHS) document, http://www.pathname.com/fhs/ . It's
> worthwhile reading.
Most people should be aware that this document is geared towards *very*
generic UNIX systems, and Linux is not one of those. Putting XFree86 into
/usr/X11 as it suggests is asking for trouble the way some people build
their makefiles (although a symlink isn't utterly out of the question,
it's pretty pointless because just as many Makefiles will likely be
broken the *other* way) ...and for some reason just seeing the
three letters o-p-t makes me think of HP-UX and then I want to wash my
brain out afterwards. It's seems to me to be a vendor's way of saying "We
don't want your files tampering with ours" and sometimes even "you aren't
smart enough to understand how our software works so we're not going to
let you screw it up".
===
Date: Tue, 6 Feb 2001 05:44:59 -0800 (PST)
From: "Dagmar d'Surreal" <dagmar@dsurreal.org>
To: Drew Bertola <drew@drewb.com>
Subject: Re: [svlug] Security question: read-only drive
On Tue, 6 Feb 2001, Drew Bertola wrote:
> Not if the C code to control the switch is binary only (locally) and
> protected by hardcoded password (argv[1]) with an algorithm like MD5
> to keep it safe within the binary. (Not to mention the obscurity
> involved).
>
> At least a random intruder would have a tough time enabling it.
Yes, but anyone capable of running a gdb or strace will breeze right by
this type of check. It would be simpler to cut your losses and just plan
on making personal visits to the facility occasionally to replace an
ejectable drive bay with a different hard drive, and focus on being able
to *detect* anything suspicious happening to the hardened host which might
slip past the binaries on it. Counting 100% on prevention in an attempt
to eliminate the need for detection is fallacious at best.
> Imagine /usr is read-only. I issue:
>
> # /usr/bin/unlock-drive seCret94
>
> unlock-drive then performs an MD5 hash on "seCret94" and if that
> matches the hardcoded MD5 hash of "seCret94" stored within
> unlock-drive, the program proceeds to open the switch for rw access to
> the drive. If argv[1] isn't exact, the hash fails and the program
> fails w/ a "Get Stuffed!" message.
>
> A seperate "lock-drive" program, not requiring a password, can then be
> run to secure the drive. It would also be run during bootup.
>
> What's great is that nobody can replace "unlock-drive" without knowing
> the password. Even if they do, I could have a NOC flip a manual
> overide switch.
>
> What do you think?
Imagine /lib *isn't* read-only. Imagine some happy soul slaps something
into /lib that replaces something nice and important like open(). You
issue the command /usr/bin/unlock-drive seCret94 and a copy of ARGV[0]
goes flying out across the internet embedded in an ICMP packet.
"seCret94" seems to be hard-coded into /usr/bin/unlock-drive so you're
rather SOL for changing it on a regular basis, but now your attacker
should have no problem trojaning a few things and cleaning up the
integrity checker database. All forms of static passwords are sitting
ducks, and allowing the switch to be controlled from that machine from a
remote location will ultimately allow someone else to do it as well, and
unless you're a kinky sort who is into PIC modules that can manage RSA
encryption, it seems like a dead-end to me.
I must admit the argument in the previous paragraph is pretty weak, tho.
Most people capable of breaking into a hardened system in the first place
would likely just debug the binary and go make whatever function calls are
necessary to do whatever it did *without* requiring the password.
I should probably quit repeating this same theme and just point out that a
mostly-non-writable filesystem is mainly only good for running your
integrity checker in a known and untampered-with configuration. Beyond
that it's a LOT of work for very little return unless you've hardened
everything else on the system and gotten every happy little subsystem
running chrooted and under restricted uids. (I spent almost two weeks'
worth of evenings just going from an empty / to having basic system
functionality and an integrity checker.) I think I probably got more
productive results out of configuring a machine to spontaneously crash if
any of a particular set of shady events occurred (ugly kludge), not to
mention the securelevel medding. (Note: Telling the system the system it
may no longer start new processes or let old ones die has rather
horrifying results the first few times you may try it. Heheh)
===
Date: Tue, 6 Feb 2001 07:31:24 -0800
From: Rick Moen <rick@linuxmafia.com>
To: svlug@svlug.org
Subject: Re: [svlug] Security question: read-only drive
begin Dagmar d'Surreal quotation:
> I recompiled a *whole* lotta stuff to go looking in /usr/etc for it's
> configuration files.
That is (slightly) interesting, but I'm not sure what, if anything, is
the relevance to Drew's concerns.
[FHS:]
> Most people should be aware that this document is geared towards *very*
> generic UNIX systems, and Linux is not one of those.
Beg pardon? I think you need to re-read it. For starters, FHS was
created as an abstraction of filesystem logic _from_ Linux
distributions.
> Putting XFree86 into /usr/X11 as it suggests....
FHS does not suggest "putting XFree86 into /usr/X11". Again, you really
need to re-read.
> ...and for some reason just seeing the three letters o-p-t makes me
> think of HP-UX and then I want to wash my brain out afterwards.
Yes. I don't like /opt either, and either eliminate it entirely, or
symlink it to somewhere in /usr/local. But you're missing the point:
FHS describes the basic logic of which filesystems are shareable vs.
non-shareable, and static vs. dynamic. The latter is an important
consideration if you're trying to decide which directory trees can be
usably mounted from read-only-jumpered media.
===
Date: Tue, 6 Feb 2001 07:45:13 -0800
To: svlug@svlug.org
Subject: Re: [svlug] Security question: read-only drive
From: Rick Moen <rick@linuxmafia.com>
begin Dagmar d'Surreal quotation:
> Yes, but anyone capable of running a gdb or strace will breeze right by
> this type of check.
One solution is to have out-of-band access. Dial-up, say.
Out of band or not, any such setup has risks. The trick is to
understand the threat model.
> Beyond that it's a LOT of work for very little return unless you've
> hardened everything else on the system and gotten every happy little
> subsystem running chrooted and under restricted uids.
Bah. Chroot is a thin reed on which to pin one's hopes. Not root safe.
====
Date: Tue, 6 Feb 2001 07:51:36 -0800
To: svlug@svlug.org
Subject: Re: [svlug] Security question: read-only drive
From: Rick Moen <rick@linuxmafia.com>
begin Dagmar d'Surreal quotation:
> Filesystems you will need to keep writeable are /tmp, /var, and /etc (it
> can be eliminated as writeable, but it's a major pain, better to rebuild
> the important stuff so that it looks for it's configuration files in
> *non*writable places other than /etc, like /usr/etc).
Wrong. No.
Look, you apparently don't buy into the basic concept of hardware-level
write-protection combined with a mechanism by which the sysadmin can
enable hardware write access on the fly. (I would have this be possible
only locally at the physical box location, whereas Drew thinks it's
possible to provide remote hardware-level write-enabling. He may be
correct. It's a question of details.)
That's fine, but that means you're carrying on an overlapping but
distinctly different conversation from the one everyone else is having.
===
Date: Tue, 6 Feb 2001 11:20:25 -0800
To: "Dagmar d'Surreal" <dagmar@dsurreal.org>
Cc: svlug@svlug.org
Subject: Re: [svlug] Security question: read-only drive
From: Chris Waters <xtifr@dsp.net>
On Tue, Feb 06, 2001 at 05:06:32AM -0800, Dagmar d'Surreal wrote:
> I recompiled a *whole* lotta stuff to go looking in /usr/etc for it's
> configuration files.
You duplicated a lot of work that Debian has already done, then:
starless ~ $ ls /usr/etc
ls: /usr/etc: No such file or directory
starless ~ $ ls /usr/local/etc
starless ~ $ ls /opt
ls: /opt: No such file or directory
I guess /usr/local/etc is there in case you want to use it yourself,
but nothing in a Debian system actually uses it.
> init was the highly unamusing one to tamper with,
No doubt! :-)
[re: the FHS]
> Most people should be aware that this document is geared towards *very*
> generic UNIX systems, and Linux is not one of those.
>From section 1.4:
The process of developing a standard filesystem hierarchy began in
August 1993 with an effort to restructure the file and directory
structure of Linux. The FSSTND, a filesystem hierarchy standard
specific to the Linux operating system, was released on February 14,
1994. Subsequent revisions were released on October 9, 1994 and March
28, 1995.
In early 1995, the goal of developing a more comprehensive version of
FSSTND to address not only Linux, but other UNIX-like systems was
adopted with the help of members of the BSD development community.
So it started as a Linux standard, and has been modified to cover BSD
as well. Neither Linux nor BSD really qualifies as a true UNIX(tm)
system.
> Putting XFree86 into /usr/X11 as it suggests is asking for trouble
> the way some people build their makefiles [...]
That's not what it suggests. Unless you mean /usr/X11R6. And really,
only X11 itself should go there, not X-based apps.
In any case, most software either uses GNU autoconf (in which case, it
should default to /usr/local, and can easily be overridden) or imake,
in which case you can edit the configuration files.
So that just leaves a handful of programs that need to be audited, and
those should probably be viewed with deep suspicion in any case.
> ...and for some reason just seeing the three letters o-p-t makes me
> think of HP-UX
I think it was actually a Sun "innovation", but I first saw it on
HPUCHHS too, so my reaction is the same. YUX! :)
One of the reasons I don't have StarOffice installed is that I'm not
willing to create /opt on my system!
Still, the FHS merely allows /opt, it doesn't require it.
===
From: Alvin Oga <alvin@planet.fef.com>
Subject: Re: [svlug] Security question: read-only drive - humm
To: xtifr@dsp.net (Chris Waters)
Date: Tue, 6 Feb 2001 15:12:36 -0800 (PST)
Cc: alvin@planet.fef.com (Alvin Oga), svlug@svlug.org
hi ya...
to simplify things...
for "read-only"... why not enforce it to be read only instead
of trying to treat it like a regular rw system ???
==
== the issue is... what files/directories needs to be written ??
==
if the beer has not killed too many brain celll...
the only part of the distro that needs to be writable is...
readonly boot media or booting method
- 1.44Mb floppy
- linuxcare bbc
- real cdrom
/etc/mtab
- make a link to /dev/ram0 mounted as /tmp
- remove silly things refering to /etc/issue and /etc/issue.net
/tmp run it in memory ( /dev/ram0 )
- link /usr/tmp to /tmp
- link /var/tmp to /tmp
/var run it in memory ( /dev/ram1 )
- /var/run /var/log ... stuff you do wanna keep ...
/proc - turn it off in the kernel...
- think might be a major problem its rw to /dev/ram??
- donno what needs to be done in lieu of /proc/mounts
( /proc is not required ?? )
<core> - core files from crashes....
- if it crashed...you have other problems anyway...
if you need /usr/local to be read/write....
- rebuild your distribution media that creates your
readonly system
- allowing rw on /usr/local creates a mess
if this thing is meant to be a secure linux server due to its
read-only drive.... follow the normal "hardening rules" ...
- turn off telnet, ftp, pop3, blah..blah..blah...
if you try to vi any file... it usually creates <path>/file.swp file
so you can't edit any file... ( which is good -- try other editors
but te file itself is still read-only so wont matter about swap file)
if 4Mb or 8Mb /dev/ramxx partitions is a problem...use /dev/loop
c ya
alvin
===
Date: Tue, 6 Feb 2001 16:11:18 -0800
To: svlug@svlug.org
Subject: Re: [svlug] Security question: read-only drive - humm
From: Rick Moen <rick@linuxmafia.com>
begin Alvin Oga quotation:
> for "read-only"... why not enforce it to be read only instead of
> trying to treat it like a regular rw system ???
Because then you have to burn / create / edit media every time you
want to make even trivial system changes. OK for a router; painful for
anything more complex.
Drew Bertola's original question addressed _only_ the question of which
directory trees are static enough in content that one might locate them
on read-only media. FHS provides (at least) a theoretical model for
which directory trees _should_ be essentially static.
I then suggested the refinement of the sysadmin (only) being able to
toggle the disc's writeable status. Doing so should alleviate much of
the awkwardness and inconvenience that approach otherwise entails.
===