This is part of The Pile, a partial archive of some open source mailing lists and newsgroups.
From: "Bryan Henry" <bryan.henry@mail.utexas.edu> To: <modperl@perl.apache.org> Subject: performance testing - emulating real world use Date: Tue, 12 Mar 2002 15:45:04 -0600 Anyone know of good guides or general info on performance testing and emulating real use of an application. I would like to understand how to identify potential bottlenecks before I deploy web apps. === Date: Tue, 12 Mar 2002 13:52:36 -0800 From: clayton cottingham <drfrog@telus.net> To: Bryan Henry <bryan.henry@mail.utexas.edu> Subject: Re: performance testing - emulating real world use Bryan Henry wrote: > > Anyone know of good guides or general info on > performance testing and emulating real use of > an application. > > I would like to understand how to identify > potential bottlenecks before I deploy web apps. try httpd.apache.org/test/ and perl framework there in as well look on freshmeat for siege it does testing too === Date: Tue, 12 Mar 2002 13:54:13 -0800 From: Paul Lindner <lindner@inuus.com> To: clayton cottingham <drfrog@telus.net> Cc: Bryan Henry <bryan.henry@mail.utexas.edu>, modperl@perl.apache.org Subject: Re: performance testing - emulating real world use On Tue, Mar 12, 2002 at 01:52:36PM -0800, clayton cottingham wrote: > Bryan Henry wrote: > > > > Anyone know of good guides or general info on > > performance testing and emulating real use of > > an application. > > > > I would like to understand how to identify > > potential bottlenecks before I deploy web apps. > > > > thank you, > > ~ b r y a n I've used HTTPD::Bench::ApacheBench (available on CPAN) to do load testing. It seems to do a good job. The hardest part is writing the testing script (especially for form transactions..). However, if you can do your requests with LWP it's fairly straightforward to convert over to the ApacheBench data structures. I'm considering writing a little mod_perl proxy server that records the different transactions. Then I could just munge the separate Authorization: headers to do some serious load testing... === Date: Tue, 12 Mar 2002 15:30:38 -0800 (PST) From: Andrew Ho <andrew@tellme.com> To: Bryan Henry <bryan.henry@mail.utexas.edu> Cc: modperl@perl.apache.org Subject: Re: performance testing - emulating real world use Heyas, BH>Anyone know of good guides or general info on BH>performance testing and emulating real use of BH>an application. As a general rule, it's easiest if you have a production system already running. Record all information that you need to reproduce the requests (typically, HTTP request headers and POST data if applicable), from a production server and you can replay any amount of data on a sandboxed QA environment. You can either eliminate or proportionally shorten the time period between requests to space out load arbitrarily. This is extremely effective if you have enough real user data because you're not inventing user load. You're using real user load. I don't know of any product that does this all at once, but it's not hard to hack together. If your site is entirely GET based, you can probably just make do with parsing access logs and turning those into requests. I believe Apache::DumpHeaders might get you most of the way on the capturing side if you need special headers, cookies, or POST information. Feeding scripts into commercial products like SilkPerformer will give you the best client side testing and reports. However, a homebrew Perl user agent will do okay, too. Unfortunately, ab doesn't support taking in a custom URL workload. For a simple record/replay load test tool that works remarkably well, check out the resource CD that ships with Windows 2000 and you will find the Microsoft Web Stress Tester. It's free and GUI based and can record IE sessions and replay them with an arbitrary number of threads. It uses Access databases to hold the tests and results so you can probably use Perl on Windows to populate it with your custom tests. === Date: Tue, 12 Mar 2002 16:19:47 -0800 (PST) From: Ask Bjoern Hansen <ask@valueclick.com> To: Andrew Ho <andrew@tellme.com> Subject: Re: performance testing - emulating real world use On Tue, 12 Mar 2002, Andrew Ho wrote: [...] > This is extremely effective if you have enough real user data because > you're not inventing user load. You're using real user load. Not really; you also have to emulate the connection speeds of the users. Or does the tools you mentioned do that? === Date: Tue, 12 Mar 2002 18:04:52 -0800 (PST) From: Andrew Ho <andrew@tellme.com> To: Ask Bjoern Hansen <ask@valueclick.com> Cc: Bryan Henry <bryan.henry@mail.utexas.edu>, modperl@perl.apache.org Subject: Re: performance testing - emulating real world use Hello, ABH>Not really; you also have to emulate the connection speeds of the ABH>users. Or does the tools you mentioned do that? Both of the commercially produced tools I mentioned (SilkPerformer and the free Microsoft Web Stress program) can throttle bandwidth. Rolling your own is a bunch harder. So you're correct. My point though is not so much that the load profile of what pages get loaded in what order, and what data calls and dynamic scripts are run in what order are genuine. If you simulate the timing between requests, you'll even get spikes that are similar to the real thing. It's definitely not reality! You also miss anomalies like users closing browsers and (unless you capture full headers) which clients support keep-alives for example. But, it's closer to reality than most scripts that are invented (especially be developers ;)). Humbly, === Date: Tue, 12 Mar 2002 19:19:14 -0800 (PST) From: Andrew Ho <andrew@tellme.com> To: Ask Bjoern Hansen <ask@valueclick.com> Cc: Bryan Henry <bryan.henry@mail.utexas.edu>, Subject: Re: performance testing - emulating real world use Hello, AH>So you're correct. My point though is not so much that the load profile of AH>what pages get loaded in what order, and what data calls and dynamic AH>scripts are run in what order are genuine. If you simulate the timing AH>between requests, you'll even get spikes that are similar to the real AH>thing. It's definitely not reality! You also miss anomalies like users AH>closing browsers and (unless you capture full headers) which clients AH>support keep-alives for example. But, it's closer to reality than most AH>scripts that are invented (especially be developers ;)). Man, I can't type worth anything today. The gist of what I meant to type was this: the exact load of the production server will not be replicated in your simulation; but the load from data calls and dynamically generated content will be similar in nature, patterned after how your server is hit in real life. This will likely be a better exercise of your server than a developer-invented test script. If you have a production environment (or proxy) set up that can capture real user requests, this is also far less work for creating a convincing simulation load than having to sit down and write a new script every time your application changes. === Date: Tue, 12 Mar 2002 22:27:43 -0800 (PST) From: Jauder Ho <jauderho@carumba.com> To: Andrew Ho <andrew@tellme.com> Subject: Re: performance testing - emulating real world use On Tue, 12 Mar 2002, Andrew Ho wrote: > Heyas, > > BH>Anyone know of good guides or general info on > BH>performance testing and emulating real use of > BH>an application. > > As a general rule, it's easiest if you have a production system already > running. Record all information that you need to reproduce the requests > (typically, HTTP request headers and POST data if applicable), from a > production server and you can replay any amount of data on a sandboxed QA > environment. You can either eliminate or proportionally shorten the time > period between requests to space out load arbitrarily. > > This is extremely effective if you have enough real user data because > you're not inventing user load. You're using real user load. > > I don't know of any product that does this all at once, but it's not hard > to hack together. If your site is entirely GET based, you can probably > just make do with parsing access logs and turning those into requests. I > believe Apache::DumpHeaders might get you most of the way on the capturing > side if you need special headers, cookies, or POST information. > > Feeding scripts into commercial products like SilkPerformer will give you > the best client side testing and reports. However, a homebrew Perl user > agent will do okay, too. Unfortunately, ab doesn't support taking in a > custom URL workload. > > For a simple record/replay load test tool that works remarkably well, > check out the resource CD that ships with Windows 2000 and you will find > the Microsoft Web Stress Tester. It's free and GUI based and can record IE > sessions and replay them with an arbitrary number of threads. It uses > Access databases to hold the tests and results so you can probably use > Perl on Windows to populate it with your custom tests. Another application (commercial) is Mercury Interactive's LoadRunner. It actually records events and plays it back on "load generator" machines. It's fairly complex, has LOTs of knobs to turn and can load test quite a bit more than just web apps, I use it to load test/benchmark Oracle 11i for instance. The software is not cheap but definitely worth looking into if you are serious about testing. (www.merc-int.com) They also sell something called ActiveTest which may be more suited to a web applications. In this case, they will test your site for you using their hardware at a colo site. === Date: Wed, 13 Mar 2002 07:37:05 +0000 (GMT) From: Matt Sergeant <matt@sergeant.org> To: Jauder Ho <jauderho@carumba.com> Subject: Re: performance testing - emulating real world use On Tue, 12 Mar 2002, Jauder Ho wrote: > > Another application (commercial) is Mercury Interactive's LoadRunner. It > actually records events and plays it back on "load generator" machines. > It's fairly complex, has LOTs of knobs to turn and can load test quite a > bit more than just web apps, I use it to load test/benchmark Oracle 11i > for instance. The software is not cheap but definitely worth looking into > if you are serious about testing. (www.merc-int.com) > > They also sell something called ActiveTest which may be more suited to a > web applications. In this case, they will test your site for you using > their hardware at a colo site. Before anyone even looks into this, be warned they quoted me