svlug_recovering_from_a_crack

This is part of The Pile, a partial archive of some open source mailing lists and newsgroups.



Date: Tue, 16 Jan 2001 23:57:10 -0500
From: Bill Jonas <bill@billjonas.com>
To: World Domination <svlug@svlug.org>
Subject: Re: [svlug] Look what I found

On Tue, Jan 16, 2001 at 07:03:06PM -0800, Todd Lyons wrote:
> Comments anybody?

Well, I wouldn't have mentioned it but no one else has.  In a word,
reinstall.

Audit all your configuration files, and back them up.  If you're using
Debian, save the output of 'dpkg --get-selections' (so you can easily get
back to where you were later with 'dpkg --set-selections <filename.txt'
followed by 'apt-get dselect-upgrade'), audit it, and save it.  Save
anything else that might be of use after auditing it.  (Note that this
specifically excludes any binary executables; text executables are a
different matter, if you audit them.)

Then reinstall.  Because with root kits being so easily available, you can
no longer trust what your system is telling you about itself.  You might
have a trojaned version of ps that won't show you processes with certain
names, or an ls that won't list certain files.  No matter how well you
scrub the system, there's no good way (in my mind, anyway) that you will
be able to say for sure that your system isn't still compromised.

If you haven't already done so, comment out the "/bin/sh" lines from
inetd.  Execute the command "grep '^[^:]*:[0]*:' /etc/passwd |grep -v
'^root:'" and ensure no other accounts show up.  (If they do, get rid of
them, perhaps saving a copy of the passwd file first; these are extra root
accounts.)  Then reboot, to kill off any processes that aren't supposed to
be running and log out anyone who's not supposed to be on.  *Then* do the
damage control, followed by a reinstall as quickly as possible.  Better
yet, unplug the network cable.

Good luck.

===

Date: Tue, 16 Jan 2001 21:53:29 -0800
To: World Domination <svlug@svlug.org>
Subject: Re: [svlug] Look what I found
From: Rick Moen <rick@linuxmafia.com>

begin  Todd Lyons quotation:

> I guess I didn't word my thoughts properly. 

If I understand you correctly, it sounds like you're pretty sure neither
you nor some process initiated by you (such as Webmin) added those 
services for any legitimate reason.  (FYI, nothing widely known has any 
claim to ports 2679 or 4464.  Nor is it BackOrifice or any of that lot.)

And, if it wasn't you, then I guarantee it was someone who meant you
ill.  In which case, you need to secure your data, blow away all current
binaries, and rebuild from trusted code.

After securing your data, you might want to take some time to study the
current machine state (out of curiosity):  Use copies of system
utilities such as ps, ls, netstat, du, df, find, etc. that you can trust
-- from elsewhere -- and see what's really installed and running on your
system.  (The installed binaries may well be fakes -- trojaned versions
-- from a rootkit.)  You might, say, find a whole bunch of intruders'
files in a directory tree named "..." or ".. " inside /dev -- but maybe 
not using the installed system utilities.  You may find concealed irc
and eggdrop daemons running.  You might find all sorts of interesting
things.

Of course, if _you_ or something acting on your behalf are using those
services for something legitimate, then never mind.  And it's your job
as sysadmin to know if that is the case.

===

Date: Tue, 16 Jan 2001 22:12:18 -0800
To: World Domination <svlug@svlug.org>
Cc: Bill Jonas <bill@billjonas.com>
Subject: Re: [svlug] Look what I found
From: Rick Moen <rick@linuxmafia.com>

begin  Bill Jonas quotation:

> Better yet, unplug the network cable.

Well, yeah.  

Todd, part of the point Bill's driving at is that, once you arrive at a 
reasonable probability that your machine has been compromised, you
cannot trust any executable, or any configuration file, on the system --
with the sole exception of any that are on hardware-level-write-protected
media such as read-only-jumpered SCSI drives.  You don't control your
system, and you can't predict what it's about to do.  It may be about to
perform the equivalent of "rm -rf /" or zero out all the superblocks at
any moment.

A truly paranoid sysadmin, in that situation, would therefore
immediately power off the system without even doing an orderly shutdown.
Then, boot from some other media, such as a Linux maintenance floppy, a
Linuxcare bootable business card, or another hard drive.  Mount your
filesystems from your presumed-compromised system.  _Don't_ run any
executables on it.  Copy off your package database.  Copy off a tarball
of /etc .  Back up /home, /usr/local (discarding all executables),
/root (discarding all executables), /var/spool/mail, and any other data
files you want to save.  Wipe the suspect filesystems.  Reinstall from
trusted sources.  Suspiciously recreate local state by recreating the
function of (not just copying back) your prior configuration files.
(Remember that users' home-directory dotfiles are suspect, too.  Any
~/.rhosts files sitting around?)

Sit back.  Have a fresh cup of coffee.  Recreate user accounts.  Issue
new passwords to all users.  Consider locking down /usr/bin/passwd, to 
drive home the message to users that you _mean it_ when you tell them
they may _not_ change back to their old passwords.  Explain matters to 
them.  (You do have telephone numbers or other out-of-band methods for 
reaching them, right?)  Apply system updates.  Do a new-system security
check and general lockdown.  Consider long and hard how the bad guys may
have gotten in, recently.  Rethink your system security policy in light
of that.  Make any changes required.

Then and only then, plug the network cable back in.

===

the rest of The Pile (a partial mailing list archive)

doom@kzsu.stanford.edu