This is part of The Pile, a partial archive of some open source mailing lists and newsgroups.
To: <modperl@apache.org> From: "Ed Park" <epark@athenahealth.com> Subject: [ANNOUNCE] new site: scaling mod_perl (+tool: mod_perl + DBD::Oracle) Date: Fri, 8 Dec 2000 05:47:00 -0500 The enterprise mod_perl architectures idea that I posted earlier has evolved into a slightly modified idea: a 'scaling mod_perl' site: http://www.lifespree.com/modperl. The point of this site will be to talk about & synthesize techniques for scaling, monitoring, and profiling large, complicated mod_perl architectures. So far, I've written up a basic scaling framework, and I've posted a particular development profiling tool that we wrote to capture, time, and explain all SQL select queries that occur on a particular page of a mod_perl + DBD::Oracle application: -http://www.lifespree.com/modperl/explain_dbitracelog.pl -http://www.lifespree.com/modperl/DBD-Oracle-1.06-perfhack.tar.gz Currently, I'm soliciting thoughts and code on the following subjects in particular: 1. Performance benchmarking code. In particular, I'm looking for tools that can read in an apache log, play it back realtime (by looking at the time between requests in the apache log), and simulate slow & simultaneous connections. I've started writing my own, but it would be cool if something else out there existed. 2. Caching techniques. I know that this is a topic that has been somewhat beaten to a pulp on this list, but it keeps coming up, and I don't know of any place where the current best thinking on the subject has been synthesized. I haven't used any caching techniques yet myself, but I intend to begin caching data at the mod_perl tier in the next version of my application, so I have a very good incentive to synthesize and benchmark various techniques. If folks could just send me pointers to various caching modules and code, I'll test them in a uniform environment and let folks know what I come up with. Or, if someone has already done all that work of testing, I'd appreciate if you could point me to the results. I'd still like to run my own tests, though. If folks could point me towards resources/code for these topics (as well as any other topics you think might be relevant to the site), please let me know. I'm offering to do the legwork required to actually test, benchmark, and synthesize all of this stuff, and publish it on the page. I'm also still interested in actually talking with various folks. If anyone who has been through some significant mod_perl scaling exercise would like to chat for 15-30 minutes to swap war stories or tactical plans, I'd love to talk with you; send me a private email. cheers, Ed === To: Ed Park <epark@athenahealth.com> From: Stas Bekman <stas@stason.org> Subject: Re: [ANNOUNCE] new site: scaling mod_perl (+tool: mod_perl + Date: Fri, 8 Dec 2000 12:18:53 +0100 (CET) On Fri, 8 Dec 2000, Ed Park wrote: > The enterprise mod_perl architectures idea that I posted earlier has evolved > into a slightly modified idea: a 'scaling mod_perl' site: > http://www.lifespree.com/modperl. Ed, this is all cool, but is it possible that we put this on one of the public sites? At some point you will probably become too busy to maintain this site... And maintaining the site whose sources are in HTML is a big mess. My suggestion is to pick the guide's code code base (Pod::HtmlPSPdf), write in POD and deliver it as a separate package e.g. 'scale', which then can be placed to perl.apache.org/scale or else and maintained with CVS. It will also allow you to deliver ps/pdf formats. Re-use the existing mirrors structure and the search facilities. === To: "Ed Park" <epark@athenahealth.com>, <modperl@apache.org> From: "Gerald Richter" <richter@ecos.de> Subject: Re: [ANNOUNCE] new site: scaling mod_perl (+tool: mod_perl + DBD::Oracle) Date: Fri, 8 Dec 2000 12:48:52 +0100 > 2. Caching techniques. I know that this is a topic that has been somewhat > beaten to a pulp on this list, but it keeps coming up, and I don't know of > any place where the current best thinking on the subject has been > synthesized. I haven't used any caching techniques yet myself, but I intend > to begin caching data at the mod_perl tier in the next version of my > application, so I have a very good incentive to synthesize and benchmark > various techniques. If folks could just send me pointers to various caching > modules and code, I'll test them in a uniform environment and let folks know > what I come up with. Or, if someone has already done all that work of > testing, I'd appreciate if you could point me to the results. I'd still like > to run my own tests, though. > Embperl 2.0 will have build in caching. I hope to release the first beta during the next week === To: "Ed Park" <epark@athenahealth.com>, <modperl@apache.org> From: "Perrin Harkins" <perrin@primenet.com> Subject: Re: [ANNOUNCE] new site: scaling mod_perl (+tool: mod_perl + DBD::Oracle) Date: Fri, 8 Dec 2000 09:36:13 -0800 > The enterprise mod_perl architectures idea that I posted earlier has evolved > into a slightly modified idea: a 'scaling mod_perl' site: > http://www.lifespree.com/modperl. > > The point of this site will be to talk about & synthesize techniques for > scaling, monitoring, and profiling large, complicated mod_perl > architectures. No offense, but the content you have here looks really well suited to be part of the Guide. It would fit nicely into the performance section. Making it a separate site kind of fragments the documentation. > So far, I've written up a basic scaling framework, and I've posted a > particular development profiling tool that we wrote to capture, time, and > explain all SQL select queries that occur on a particular page of a mod_perl > + DBD::Oracle application: > -http://www.lifespree.com/modperl/explain_dbitracelog.pl > -http://www.lifespree.com/modperl/DBD-Oracle-1.06-perfhack.tar.gz Take a look at DBIx::Profile as well. > 1. Performance benchmarking code. In particular, I'm looking for tools that > can read in an apache log, play it back realtime (by looking at the time > between requests in the apache log), and simulate slow & simultaneous > connections. I've started writing my own, but it would be cool if something > else out there existed. The mod_backhand project was developing a tool like this called Daquiri. > If folks could just send me pointers to various caching > modules and code, I'll test them in a uniform environment and let folks know > what I come up with. There are a bunch of discussions about this in the archives, including one this week. Joshua Chamas did some benchmarking on a dbm-based approach recently. - Perrin === To: Perrin Harkins <perrin@primenet.com> From: Matt Sergeant <matt@sergeant.org> Subject: Re: [ANNOUNCE] new site: scaling mod_perl (+tool: mod_perl + Date: Fri, 8 Dec 2000 17:53:49 +0000 (GMT) On Fri, 8 Dec 2000, Perrin Harkins wrote: > > 1. Performance benchmarking code. In particular, I'm looking for tools > that > > can read in an apache log, play it back realtime (by looking at the time > > between requests in the apache log), and simulate slow & simultaneous > > connections. I've started writing my own, but it would be cool if > something > > else out there existed. > > The mod_backhand project was developing a tool like this called Daquiri. I also have a tool here that you install as a TransHandler, and it reads querystring and POST data [*] and saves it to an XML log file, then it appends to the PerlHandler phase and dumps the output that would go to the browser to the same XML log file (of course this requires the output to be stored somewhere, like Apache::Filter does, rather than printed direct to the browser). Then there's a little playback tool. Its a bit specific to the app I'm working on, because there's code in place to make Apache::Request a singleton and also collect the output, but I could be persuaded to open it up to people. [*] using Apache::Request at the moment - but we need to modify Apache::Request to be a singleton as it can't be instantiated twice in the same application at the moment. === To: <modperl@apache.org> From: "Ed Park" <epark@athenahealth.com> Subject: RE: [ANNOUNCE] new site: scaling mod_perl will be movin to the Guide Date: Fri, 8 Dec 2000 14:14:18 -0500 I've gotten in touch with Stas, and the 'scaling mod_perl' site will eventually be folded into the Guide. woohoo! I'm going to spend several weeks fleshing it out and cleaning it up before it goes in, though. -Ed -----Original Message----- From: Perrin Harkins [mailto:perrin@primenet.com] Sent: Friday, December 08, 2000 12:36 PM To: Ed Park; modperl@apache.org Cc: jtlin@teamplay.net Subject: Re: [ANNOUNCE] new site: scaling mod_perl (+tool: mod_perl + DBD::Oracle) > The enterprise mod_perl architectures idea that I posted earlier has evolved > into a slightly modified idea: a 'scaling mod_perl' site: > http://www.lifespree.com/modperl. > > The point of this site will be to talk about & synthesize techniques for > scaling, monitoring, and profiling large, complicated mod_perl > architectures. No offense, but the content you have here looks really well suited to be part of the Guide. It would fit nicely into the performance section. Making it a separate site kind of fragments the documentation. > So far, I've written up a basic scaling framework, and I've posted a > particular development profiling tool that we wrote to capture, time, and > explain all SQL select queries that occur on a particular page of a mod_perl > + DBD::Oracle application: > -http://www.lifespree.com/modperl/explain_dbitracelog.pl > -http://www.lifespree.com/modperl/DBD-Oracle-1.06-perfhack.tar.gz Take a look at DBIx::Profile as well. > 1. Performance benchmarking code. In particular, I'm looking for tools that > can read in an apache log, play it back realtime (by looking at the time > between requests in the apache log), and simulate slow & simultaneous > connections. I've started writing my own, but it would be cool if something > else out there existed. The mod_backhand project was developing a tool like this called Daquiri. > If folks could just send me pointers to various caching > modules and code, I'll test them in a uniform environment and let folks know > what I come up with. There are a bunch of discussions about this in the archives, including one this week. Joshua Chamas did some benchmarking on a dbm-based approach recently. - Perrin ===