This is part of The Pile, a partial archive of some open source mailing lists and newsgroups.
Subject: Bugzilla Security Concerns (was: External Bugzilla)
From: "Jon Robertson" <JRobertson@MedEvolve.com>
Date: Wed, 19 Jul 2000 14:28:54 -0500
IMHO, it's a BAD IDEA to host the MySQL install on the
> same machine as your Apache server. That's basically asking
> to be cracked, particularly if you run MySQL as root.
But what if you have followed the security advisories?
Our Bugzilla machine has two users, 'root' and 'bugs'. 'bugs' is just an
ordinary user. MySQL runs as 'bugs'. Apache still runs as 'nobody'.
'bugs' is the only MySQL account. 'root' doesn't have a MySQL account. If
the root account needs to use bugs, you have to use -u bugs to get in. This
doesn't prevent 'root' from doing anything in MySQL. But it may slow down
someone who expects 'root' to have access to MySQL.
The 'bugs' MySQL account does use a password. The account is wide open as
far as MySQL security is concerned. Since Bugzilla is the only thing on
this box and certainly the only thing using MySQL, I didn't see a reason to
optimize MySQL security. The 'bugs' user has no special privs, aside from
access to the Bugzilla and MySQL directories.
The mysql init script runs safe_mysqld *and* passes in --skip-networking. I
didn't realize one provided the other and haven't changed the script since I
learned this. :) Regardless, there is definitely no external access to
MySQL.
All writable areas of the hard drive are under /u01 (which is mounted from a
separate partition from /). This includes /u01/bugswww for the Bugzilla
code/files and /u01/mysql for the MySQL data directory. Nothing in /
(excluding /u01 and /home/bugs) is open to anyone besides root.
The only area accessible from Apache is /u01/bugswww. All static html files
and scripts reside in this directory.
I'm continually amazed at what Linux crackers are able to do. But could
this machine really be that much safer?
As for other services, I know the ftp server is running and I believe
anonymous ftp is enabled. I want the ftp server running because I use it
internally. This service (and any other service) are blocked thanks to the
firewall (see below).
And for that extra layer of security, the machine is behind a Cisco
router-provided firewall which only allows access to port 80 on this
machine. But I'm curious how secure the machine is even without the
firewall.
Jon
===
Subject: Re: External Bugzilla
From: Jonathan Li <jli@london.kivera.com>
Date: Wed, 19 Jul 2000 12:12:09 -0700
Matthew Barnson wrote:
> Jonathan,
> IMHO, it's a BAD IDEA to host the MySQL install on the same machine as your
> Apache server. That's basically asking to be cracked, particularly if you
> run MySQL as root.
> Some time ago, we had planned on running Bugzilla externally. Since we
> generate such a huge volume of support calls and emails from visitors, we
> decided logging them in Bugzilla directly by customer emails was a very bad
> idea, and instead have them filtered through our technical support dep't,
> which searches for duplicates before submitting. If you don't have a
> dedicated tech support dep't to filter incoming emails and calls, here's what
> I would suggest:
>
> 1. Use two machines. Put your Apache and MySQL servers behind a firewall,
> if possible.
> 2. Set up a dedicated link (100-base-T) between your two machines. This
> means each machine will have to be dual-homed... kinda' like this:
>
> (Internet) <----> Apache Server <----> MySQL server <---> LAN (maintenance
> interface for MySQL server)
>
> Even better, put a firewall in front of your Apache server that restricts
> all traffic to port 80 so you don't get hacked.
>
> 3. Your reasoning is a bit flawed -- you would put Bugzilla on an
> external-facing dual-homed Apache web server, and the MySQL installation
> behind it.
>
> Are you doing some kind of bizarre integration with another bug database? If
> that's the case, these guidelines don't apply.
>
Matthew.
I'm not quite sure if I totally understand what you're
saying. We already have our mysql server on a different
machine and we plan to have bugzilla on the external facing
machine use a mysql client from that machine to connect to
the mysql server on the other machine. Does that sound
right?
Jonathan
===
Subject: Re: External Bugzilla
From: Matthew Barnson <mbarnson@excitehome.net>
Date: Wed, 19 Jul 2000 13:10:39 -0600
On Wed, 19 Jul 2000, Jonathan Li scratched this in the dirt:
> we plan to have bugzilla on the external facing machine use a mysql client
>from that machine to connect to the mysql server on the other machine.
Right on the button. You original posting sounded kinda' strange, like you
were running the MySQL database and Bugzilla on the same, publicly-accessible
machine. The SysAdmin inside of me was screaming, "No! No! Don't do it!" :)
Considering the Bugzilla exploits that have been used in the past, I strongly
recommend firewalling or port-filtering the Bugzilla machine. And then,
setting up IPCHAINS so that any request to any port on the box ends up being
redirected to port 80, just in case :)
====
Subject: Re: Bugzilla Security Concerns (was: External Bugzilla)
From: Matthew Barnson <mbarnson@excitehome.net>
Date: Wed, 19 Jul 2000 13:31:17 -0600
Jon,
If your Bugzilla machine is not well-firewalled, I think it's a bad idea to
run MySQL on the same machine as Bugzilla.
I love Apache web server, and have worked with it nearly exclusively for
years. (I've had a couple affairs with IIS, but she kept pressuring me for
more control over our relationship and wanted me to swing with her friends ASP
and C#. Anybody who puts "pound" {yes, I know it's C-sharp} into their name
should be avoided at all costs, and I try to avoid snakes as well.)
However, Apache's had a number of exploits over the years, principally due
to poorly-written CGI programs (not it's fault, really, but nevertheless
it's there). Bugzilla, AFAIK, has never had a competent line-by-line security
audit, and has, historically, been used for intrusion. It's not a question
of if there's another Bugzilla exploit -- there almost surely are.
Unfortunately, most systems administrators are too (lazy/ incompetent/
overworked/ insert another excuse here) to really audit the security of their
systems. I thought of going into details how, in general, one could exploit a
Bugzilla system running MySQL on it, but came to the conclusion that any
argument I could present could be prevented by a competent sysadmin. That's
the key right there: a competent sysadmin willing to keep the system
up-to-date with all the latest bugfixes. Barring that, firewalling the
machine, setting up IPCHAINS (or IPFilter, for Solaris-heads) to redirect all
traffic on the external interface to port 80, eliminating all non-critical
SUID programs on the system, etc etc ad nauseum would also do the trick.
I operate under the assumption that most people running Bugzilla are not,
by trade, UNIX Systems Administrators, and should not, as a matter of
principle, run their database on a publicly-accessible network. It should be
on a private, non-addressable network. The information in your bug database,
one can assume, is vital to your business. If an intruder gains access to
Bugzilla, they can corrupt that data (even if it is on another machine,
although that lowers the chances). If they corrupt your database
information, at the very least, if you have been careful, you'll need to
restore from a backup. If gaining access to the "bugs" database is as easy
as running "mysql -u bugs" as the "nobody" user on the system, you're
screwed. At least by removing the database from that machine, you force them
to have a knowledge of Perl::DBD SQL calls in order to hose your DB.
Sorry for the lengthy treastise; I'm positive I've not explained myself
well. Simply put: in the absence of professional, well-trained UNIX admins
on-staff who have adequately secured your Bugzilla box from intrusion, you
should run MySQL on a separate machine. It's cheaper to spend $5,000 on a
second machine than hire a UNIX security guru at $70,000 per year :)
I've intentionally left the whole question of defaced web sites to other
authors.
===
Subject: Re: External Bugzilla
From: Matthew Barnson <mbarnson@excitehome.net>
Date: Wed, 19 Jul 2000 14:14:57 -0600
FYI: I am the security Nazi in my organization; I err on the side of "I don't
care if it breaks other things" side of security -- security first, fix the
problems excessive security causes later.
I see no inherent problems with mysql binaries existing on your
external-facing machine. However, you shouldn't need any of them after
installing Bugzilla; the Perl::MySQL modules take care of communication for
you. Once you've installed Bugzilla and confirmed it works as expected, you
should be able to safely delete these binaries on your Bugzilla external
machine:
bin/isamchk
bin/isamlog
bin/perror
bin/replace
bin/resolveip
bin/safe_mysqld
bin/msql2mysql
bin/mysql*
As a matter of face, you should be able to safely just compile MySQL, and
manually copy the lib/ and include/ directories to your desired location,
then build your Perl::MySQL and Perl::DBD stuff.
This isn't the way we run it, though, so take it with a grain of salt. We
have our system firewalled from the outside completely, and firewalled off,
in turn, from all the "inexperienced" and "untrusted" users. There's a short
list of people who have physical or SSH access to the machine. We run MySQL
on the same machine, but plan on moving it to a separate machine shortly
(we're at over 6700 bugs right now).
===
Subject: Re: Bugzilla Security Concerns (was: External Bugzilla)
From: "Jon Robertson" <JRobertson@MedEvolve.com>
Date: Wed, 19 Jul 2000 15:51:15 -0500
I've had a couple affairs with IIS, but she kept pressuring me for more
> control over our relationship and wanted me to swing with her friends ASP
> and C#. Anybody who puts "pound" {yes, I know it's C-sharp} into their
> name should be avoided at all costs, and I try to avoid snakes as well.)
Unfortunately, our web server is IIS and our pages use FrontPage extensions.
Every time our web page is updated, the ASP scripts quit working and I'm
recruited to fix them. Oddly enough, all I do to fix them is republish then
until they work again (sometimes once, sometimes four or five times).
Sounds like an awfully reliable system to me (I can't stand it)...
> That's the key right there: a competent sysadmin willing
> to keep the system up-to-date with all the latest bugfixes.
Oh, we have a fairly competent network admin who knows a lot about network
security and Linux. He's just had better things to do than to take over
admin of the Bugzilla machine. My job was to get it working and to
admin/support Bugzilla itself and he insisted on admining the machine
itself - he just hasn't done it yet. :(
> I operate under the assumption that most people running Bugzilla are not,
> by trade, UNIX Systems Administrators, and should not, as a matter of
> principle, run their database on a publicly-accessible network.
I may go on a search for a/several decent *nix security document/s that can
be read by newbie Bugzilla admins who don't know *nix security issues.
Having a link to those documents in the FAQ would be a good thing (tm)...
:)
> If gaining access to the "bugs" database is as easy as running
> "mysql -u bugs" as the "nobody" user on the system, you're
> screwed.
Well, it isn't because the 'bugs' mysql user requires a password. So it
would require "mysql -u bugs -p validpassword". Someone with knowledge of
Perl::DBD and Bugzilla would have a better chance since they could probably
retrieve the password from ::db_pass.
But if I didn't use MySQL passwords, how could someone from the outside run
"mysql -u bugs" from the "nobody" user?
Jon
===
Subject: Re: Bugzilla Security Concerns (was: External Bugzilla)
From: Matthew Barnson <mbarnson@excitehome.net>
Date: Wed, 19 Jul 2000 14:37:02 -0600
On Wed, 19 Jul 2000, Jon Robertson scratched this in the dirt:
> But if I didn't use MySQL passwords, how could someone from the outside run
> "mysql -u bugs" from the "nobody" user?
She'd have to get a shell on the machine. The 'mysql' executable defaults
to world-executable (another example of a problem that could be averted by
good systems administration). Simply run "mysql -u bugs" and you're in. If
your localconfig is world-readable (which it must be in order to use it, if
Apache runs as "nobody") the user can just "cat localconfig", as you
mentioned, looking for $db_pass.
I'm not sure how you would get around it.
The basic problem is a user getting a shell on your machine. If they can
get a shell, they can root you. Period. No matter what you do, you must
assume if the user can obtain shell access, they can get root.
I'm not saying running MySQL on a separate machine is a panacea -- far from
it. But it provides an additional level of complexity. Most script kiddies
are pretty lazy, and want to do the least for the most result. Attempting to
ruin your Bugzilla data is probably harder than simply defacing the main
Bugzilla page.
===
Subject: Re: External Bugzilla
From: Matthew Barnson <mbarnson@excitehome.net>
Date: Wed, 19 Jul 2000 14:44:08 -0600
On Wed, 19 Jul 2000, Jon Robertson scratched this in the dirt:
> Oh lord. So is our admin. And it is *so* frustrating. Sometimes I just
> want to tell him to get a life. :-)
Yup, I don't have a life. My team kinda' has to err on the side of
paranoia, though. If you had any idea how much traffic Excite@Home gets
daily, and how many network probes, attacks, attempted mail relays, and so
forth, you'd be paranoid too. Paranoia is not optional when you're running
millions of hits and dozens of attempted hacks per day.
The sad thing is, most of the script kiddies have excite@home network
addresses :) That's how they use their always-on connection, I guess.
===
Subject: Re: Bugzilla Security Concerns (was: External Bugzilla)
From: "Jon Robertson" <JRobertson@MedEvolve.com>
Date: Wed, 19 Jul 2000 16:11:10 -0500
She'd have to get a shell on the machine.
That makes sense. As initially mentioned, there are only two accounts on
this machine, 'root' and 'bugs'. No one touches it except via Apache, which
runs as nobody.
I had the impression from your post that someone could run mysql -u bugs via
the nobody account. My understanding is that "nobody" is an account that
someone could log into.
===