This is part of The Pile, a partial archive of some open source mailing lists and newsgroups.
From: Ian Robertson <ian@eziba.com>
To: mason-users@lists.sourceforge.net
Subject: [Mason] Performance differences between Mason 1.03 and Mason 1.11
Date: Fri, 12 Jul 2002 12:11:31 -0400
In preparing to upgrade our site from 1.03 to 1.11, I just ran a test
comparing the speed of Mason versions 1.03 and 1.11. The test
basically consists of running through a simple template 50 times, and
reporting the average time per run. The upshot is a nearly nine-fold
slow-down in Mason 1.11:
ian1:Mason1.03$ ./runner.pl simple.tmpl
average time: 0.00142055988311768
ian1:Mason1.11$ ./runner.pl simple.tmpl
average time: 0.0127044010162354
Is this a known issue? Am I doing something stupid that is making
1.11 so slow?
The template being run was:
-----simple.tmpl-----
% for( my $i = 0; $i < 100; $i++ )
% {
That's great, it starts with an earthquake, birds and snakes, an
aeroplane and Lenny Bruce is not afraid. Eye of a hurricane, listen to
yourself churn - world serves its own needs, dummy serve your own
needs. Feed it off an aux speak, grunt, no, strength. Ladder start to
clatter with fear fight down height. Wire in a fire, represent the
seven games a government for hire and a combat site. Left of west and
coming in a hurry with the furies breathing down your neck. Team by
team reporters baffled, trumped, tethered cropped. Look at that low
playing! Fine, then. Uh oh, overflow, population, common food, but
it'll do. Save yourself, serve yourself. World serves its own needs,
listen to your heart bleed dummy with the rapture and the revered in
the right - right. You vitriolic, patriotic, slam, fight, bright
light, feeling pretty psyched.
% }
-----simple.tmpl-----
The code to run Mason 1.03 was:
-----runner.pl-----
#!/usr/bin/perl
use strict;
use lib "."; # Use our local copy of Mason-1.03
use HTML::Mason;
use Time::HiRes qw(time);
use Cwd;
my $TRIES = 50;
my $component = $ARGV[0];
my $output;
my $interp = HTML::Mason::Interp->
new( comp_root => cwd, data_dir => cwd,
out_method => \$output, out_mode => "stream" );
$interp->exec( "/$component" ); # compile
my $start = time;
for( my $i = 0; $i< $TRIES; $i++)
{
$output = "";
$interp->exec( "/$component" );
}
print "average time: ", (time - $start)/$TRIES, "\n";
-----runner.pl-----
The code to run Mason 1.11 was:
-----runner.pl-----
#!/usr/bin/perl
use strict;
use HTML::Mason;
use Time::HiRes qw(time);
use Cwd;
my $TRIES = 50;
my $component = $ARGV[0];
my $output;
my $interp = HTML::Mason::Interp->
new( comp_root => cwd, data_dir => cwd, out_method => \$output );
$interp->exec( "/$component" ); # compile
my $start = time;
for( my $i = 0; $i< $TRIES; $i++)
{
$output = "";
$interp->exec( "/$component" );
}
print "average time: ", (time - $start)/$TRIES, "\n";
-----runner.pl-----
P.S. I also noticed that by moving the text body from directly inside
the loop into def subcomponent increased the running time in both
versions by nearly a factor of 10. I would expect an increase, but
should it be that much?
===
From: Dave Rolsky <autarch@urth.org>
To: Ian Robertson <ian@eziba.com>
Subject: Re: [Mason] Performance differences between Mason 1.03 and Mason
Date: Fri, 12 Jul 2002 14:09:49 -0500 (CDT)
On Fri, 12 Jul 2002, Ian Robertson wrote:
> In preparing to upgrade our site from 1.03 to 1.11, I just ran a test
> comparing the speed of Mason versions 1.03 and 1.11. The test
> basically consists of running through a simple template 50 times, and
> reporting the average time per run. The upshot is a nearly nine-fold
> slow-down in Mason 1.11:
It is slower, but your test is one that exposes one of the weaknesses of
Mason's implementation. Basically, very large loops create a very large
number of method calls in the generated object code.
We might be able to introduce an optimization to improve this, but it
would probably require us to remove the autoflush feature entirely.
===
From: Dave Rolsky <autarch@urth.org>
To: Ian Robertson <ian@eziba.com>
Subject: Re: [Mason] Performance differences between Mason 1.03 and Mason
Date: Fri, 12 Jul 2002 14:18:55 -0500 (CDT)
On Fri, 12 Jul 2002, Ian Robertson wrote:
> In preparing to upgrade our site from 1.03 to 1.11, I just ran a test
> comparing the speed of Mason versions 1.03 and 1.11. The test
> basically consists of running through a simple template 50 times, and
> reporting the average time per run. The upshot is a nearly nine-fold
> slow-down in Mason 1.11:
Also, for a more realistic test, I'd suggest you do something in your
component _besides_ simply looping over the same text over and over again,
because this is _not_ how most people use Mason.
Try adding a database query or two into that loop and _then_ time it.
Most likely the database overhead will far outweigh the apparent slowdown
between the two versions. I suspect 1.11 will still be slower, but not 9
times slower.
Artificial benchmarks like this are just that, artificial. Since most
Mason apps are likely to access some sort of outside resource (RDBMS, db
file, filesystem, etc.), it is important to include that in your
benchmarks rather than simply benchmarking Mason itself.
Because if its your database that's the bottleneck (as it often will be)
then Mason's speed isn't nearly as important until _after_ you've
optimized all your queries as much as possible.
FWIW, the CVS version seems to be a bit faster, maybe as a result of
eliminating memory leaks?
I'll probably be release 1.12 soon, as eliminating these leaks is pretty
important.
===
From: "Jonathan Swartz" <swartz@pobox.com>
To: "'Dave Rolsky'" <autarch@urth.org>, "'Ian Robertson'" <ian@eziba.com>
Cc: <mason-users@lists.sourceforge.net>, <jay@eziba.com>
Subject: RE: [Mason] Performance differences between Mason 1.03 and Mason 1.11
Date: Fri, 12 Jul 2002 12:32:57 -0700
> On Fri, 12 Jul 2002, Ian Robertson wrote:
> > In preparing to upgrade our site from 1.03 to 1.11, I
> > just ran a test comparing the speed of Mason versions
> > 1.03 and 1.11. The test basically consists of running
> > through a simple template 50 times, and reporting the
> > average time per run. The upshot is a nearly nine-fold
> > slow-down in Mason 1.11:
> It is slower, but your test is one that exposes one of the
> weaknesses of Mason's implementation. Basically, very
> large loops create a very large number of method calls in
> the generated object code.
> We might be able to introduce an optimization to improve
> this, but it would probably require us to remove the
> autoflush feature entirely.
Thought about this a bit more: what about an
'enable_autoflush' Compiler parameter that determines
whether the object files call $m->print or just append to a
buffer. It would default to 0 for maximum default
performance. Then autoflush remains a Request parameter
that only makes sense when enable_autoflush is turned on.
We could use this to tune other rarely used features that
affect performance. But getting rid of the print method
calls is a big one, I think (still haven't had a chance to
run the numbers).
===
From: Ian Robertson <ian@eziba.com>
To: Dave Rolsky <autarch@urth.org>
Cc: Ian Robertson <ian@eziba.com>, mason-users@lists.sourceforge.net,
Subject: Re: [Mason] Performance differences between Mason 1.03 and Mason
Date: Fri, 12 Jul 2002 15:44:20 -0400
>>>>> "DR" == Dave Rolsky <autarch@urth.org> writes:
DR> On Fri, 12 Jul 2002, Ian Robertson wrote:
>> In preparing to upgrade our site from 1.03 to 1.11, I just ran a test
>> comparing the speed of Mason versions 1.03 and 1.11. The test
>> basically consists of running through a simple template 50 times, and
>> reporting the average time per run. The upshot is a nearly nine-fold
>> slow-down in Mason 1.11:
DR> Also, for a more realistic test, I'd suggest you do something in your
DR> component _besides_ simply looping over the same text over and over again,
DR> because this is _not_ how most people use Mason.
It's actually not so far from how we use Mason - we use a fair amount of
component and def-component calls. For example, every link and image
source tag is generated by component which customizes urls depending
on a number of factors. Our standard font is in a component, so all
text blocks are surrounded by component calls to open and close a font
tag. (We could use </font> to close, but we've found that
<& /font:open &>Text<& /font:close &> has a nice balance to it ).
This was all developed under mason 1.03, and we determined that the
total rendering time was in fact small compared to other factors, such
as database access.
DR> Try adding a database query or two into that loop and _then_ time it.
DR> Most likely the database overhead will far outweigh the apparent slowdown
DR> between the two versions. I suspect 1.11 will still be slower, but not 9
DR> times slower.
Actually, it was observations of what was happening to total request
times after a Mason upgrade that led us to investigate Mason
specifically. We have two machines which, prior to upgrading Mason,
ran identical in time; now the one running Mason 1.11 is taking as
2-3 times as long as the Mason 1.03 box to service http requests, and
the load is substantially higher to boot.
===
From: Dave Rolsky <autarch@urth.org>
To: Ian Robertson <ian@eziba.com>
Subject: Re: [Mason] Performance differences between Mason 1.03 and Mason
Date: Fri, 12 Jul 2002 15:29:24 -0500 (CDT)
On Fri, 12 Jul 2002, Ian Robertson wrote:
> It's actually not so far from how we use Mason - we use a fair amount of
> component and def-component calls. For example, every link and image
> source tag is generated by component which customizes urls depending
> on a number of factors. Our standard font is in a component, so all
> text blocks are surrounded by component calls to open and close a font
> tag. (We could use </font> to close, but we've found that
> <& /font:open &>Text<& /font:close &> has a nice balance to it ).
>
> This was all developed under mason 1.03, and we determined that the
> total rendering time was in fact small compared to other factors, such
> as database access.
So is it individual component rendering or component calls that seem much
slower? The component call code hasn't changed that much, I don't think,
but we'll take a look.
We've really done very little profiling of the new code yet, most of our
work was just concerned with getting it to actually work ;)
> Actually, it was observations of what was happening to total request
> times after a Mason upgrade that led us to investigate Mason
> specifically. We have two machines which, prior to upgrading Mason,
> ran identical in time; now the one running Mason 1.11 is taking as
> 2-3 times as long as the Mason 1.03 box to service http requests, and
> the load is substantially higher to boot.
Ah, that is worth looking into, indeed.
We'll take a look at component call overhead, hopefully before 1.12.
===
From: "Jonathan Swartz" <swartz@pobox.com>
To: "Dave Rolsky \(E-mail\)" <autarch@urth.org>
Cc: <mason-users@lists.sourceforge.net>, <jay@eziba.com>
Subject: RE: [Mason] Performance differences between Mason 1.03 and Mason 1.11
Date: Sat, 13 Jul 2002 05:35:04 -0700
> Also, for a more realistic test, I'd suggest you do something in your
> component _besides_ simply looping over the same text over
> and over again, because this is _not_ how most people use Mason.
> Try adding a database query or two into that loop and _then_ time it.
We should also use an autohandler and, say, five component calls in our
test. We can debate the "average" number of component calls per Mason page,
but it is definitely higher than one, which is what the hello world test
provides.
It's important to get component calls in there because it gives the
once-per-component code more weight in the benchmarks than the
once-per-request code.
===
From: Ian Robertson <ian@eziba.com>
To: <swartz@pobox.com>
Cc: "'Dave Rolsky'" <autarch@urth.org>, "'Ian Robertson'" <ian@eziba.com>,
Subject: RE: [Mason] Performance differences between Mason 1.03 and Mason 1.11
Date: Sat, 13 Jul 2002 14:52:19 -0400
>>>>> "JS" == Jonathan Swartz <swartz@pobox.com> writes:
JS> Thought about this a bit more: what about an 'enable_autoflush' Compiler
JS> parameter that determines whether the object files call $m->print or just
JS> append to a buffer. It would default to 0 for maximum default performance.
JS> Then autoflush remains a Request parameter that only makes sense when
JS> enable_autoflush is turned on.
I'm not sure that this is the source of the problem. I've done some
further testing in which I've subclasses Request and overridden the
print method to simply append to a string, or even to do nothing at
all. Even without producing output Maon 1.11 still runs substantially
slower that an unmodified 1.03.
I've also run both versions against the components that make up our
actual homepage. Mason 1.03 processes it in .23 second, while it
takes Mason 1.11 a full 1.38 seconds; overriding print to simply noop
only brings it down to 1.28 seconds. Admittedly, this is running on
my home pc; on actual servers, it will run probably 2-3 times faster,
but it's still significant :).
===
From: Dave Rolsky <autarch@urth.org>
To: Ian Robertson <ian@eziba.com>
Subject: RE: [Mason] Performance differences between Mason 1.03 and Mason
Date: Sat, 13 Jul 2002 14:02:22 -0500 (CDT)
On Sat, 13 Jul 2002, Ian Robertson wrote:
> I'm not sure that this is the source of the problem. I've done some
> further testing in which I've subclasses Request and overridden the
> print method to simply append to a string, or even to do nothing at
> all. Even without producing output Maon 1.11 still runs substantially
> slower that an unmodified 1.03.
Yep, my testing suggested this wasn't the biggest issue.
> I've also run both versions against the components that make up our
> actual homepage. Mason 1.03 processes it in .23 second, while it
> takes Mason 1.11 a full 1.38 seconds; overriding print to simply noop
> only brings it down to 1.28 seconds. Admittedly, this is running on
> my home pc; on actual servers, it will run probably 2-3 times faster,
> but it's still significant :).
We've been discussing this on the dev list but what it comes down to is
the following:
- Params::Validate is being called a lot. Optimizing it helps (so install
the new 0.20 version once it hits a CPAN mirror near you). Mason 1.12
will require the new, faster version.
- Class::Container is being called a lot too. I've sent Ken some speedup
patches so hopefully there will be a new release of this soon as well that
Mason can require.
- Making some objects not be Class::Container subclasses seems to help.
This change will probably be in Mason 1.12.
- Setting the environment variable PERL_NO_VALIDATION to a true value will
make Params::Validate run _much_ faster, because it won't do any
validation ;) Turning this on in production is probably worthwhile but I
wouldn't recommend it during development, except for benchmarking.
The speedup that this env var provides is _much_ greater with version 0.20
of P::V.
- Setting the static_source parameter for the Interp object may give you a
small but noticeable speed boost. Again, this is good for production but
would make development unweildy.
Anyway, try some of these out (as the relevant modules become available)
and let us know what you see. I think it'll provide a noticeable boost,
though it will probably still be slower than version 1.05.
===
From: Ian Robertson <ian@eziba.com>
To: <mason-users@lists.sourceforge.net>
Cc: <jay@eziba.com>
Subject: RE: [Mason] Performance differences between Mason 1.03 and Mason
Date: Sun, 14 Jul 2002 23:03:03 -0400
>>>>> "DR" == Dave Rolsky <autarch@urth.org> writes:
DR> On Sat, 13 Jul 2002, Ian Robertson wrote:
IBR> I've also run both versions against the components that make up our
IBR> actual homepage. Mason 1.03 processes it in .23 second, while it
IBR> takes Mason 1.11 a full 1.38 seconds; overriding print to simply noop
IBR> only brings it down to 1.28 seconds. Admittedly, this is running on
IBR> my home pc; on actual servers, it will run probably 2-3 times faster,
IBR> but it's still significant :).
DR> We've been discussing this on the dev list but what it comes down to is
DR> the following:
DR> - Setting the environment variable PERL_NO_VALIDATION to a true value will
DR> make Params::Validate run _much_ faster, because it won't do any
DR> validation ;) Turning this on in production is probably worthwhile but I
DR> wouldn't recommend it during development, except for benchmarking.
DR> - Setting the static_source parameter for the Interp object may give you a
DR> small but noticeable speed boost. Again, this is good for production but
DR> would make development unweildy.
Those two in combination make a pretty substantial difference in the
aforementioned test:
no optimizations: 1.40 seconds
PERL_NO_VALIDATION: 0.98 seconds
static_source: 0.74 seconds
static_source + NO_VAL: 0.54 seconds
This is still double of:
mason 1.03: 0.23 seconds
But it's getting into much more managable territory :)
BTW, this advice is greatly appreciated. The new Mason has some very
nice features, and we're itching to get it into production :). All of
the suggestions are both helpful and reassuring.
===
Subject: Re: [Mason] Performance differences between Mason 1.03 and Mason 1.11
Cc: <mason-users@lists.sourceforge.net>, <jay@eziba.com>
To: Ian Robertson <ian@eziba.com>
From: Ken Williams <ken@mathforum.org>
Date: Mon, 15 Jul 2002 13:25:57 +1000
On Monday, July 15, 2002, at 01:03 PM, Ian Robertson wrote:
> Those two in combination make a pretty substantial difference in the
> aforementioned test:
>
> no optimizations: 1.40 seconds
> PERL_NO_VALIDATION: 0.98 seconds
> static_source: 0.74 seconds
> static_source + NO_VAL: 0.54 seconds
>
> This is still double of:
> mason 1.03: 0.23 seconds
>
> But it's getting into much more managable territory :)
Yeah, that's looking much better.
>
> BTW, this advice is greatly appreciated. The new Mason has some very
> nice features, and we're itching to get it into production :). All of
> the suggestions are both helpful and reassuring.
We're working pretty hard to make some real gains in performance
now, and most of the changes are in two other modules
(Class::Container and Params::Validate) that will have lots of
internal optimizations in upcoming releases. Most of the
Class::Container gains are coming from caching, and most of the
Params::Validate gains are coming from avoiding lots of data
structure manipulations. Mason uses both of these modules
really heavily.
===
From: Dave Rolsky <autarch@urth.org>
To: Ian Robertson <ian@eziba.com>
Subject: RE: [Mason] Performance differences between Mason 1.03 and Mason
Date: Sun, 14 Jul 2002 23:13:14 -0500 (CDT)
On Sun, 14 Jul 2002, Ian Robertson wrote:
> Those two in combination make a pretty substantial difference in the
> aforementioned test:
>
> no optimizations: 1.40 seconds
> PERL_NO_VALIDATION: 0.98 seconds
> static_source: 0.74 seconds
Wow, that's bigger than I thought. Cool!
> static_source + NO_VAL: 0.54 seconds
Even better but turning off validation may not be worth it. Wait til 1.12
which will incorporate 3 new avenues of optimization:
1. Mason itself has been optimized for the common cases of code flow,
particularly when it comes to component calls.
2. Mason will require Params::Validate 0.21, which is quite a bit faster,
particularly the validate_with() function, which Class::Container uses
(and therefore Mason uses), is now more than doubled in speed.
And its PERL_NO_VALIDATION mode is also a bit faster, though I really hope
to avoid pushing people into using it.
I suspect this will translate into a noticeable speed increase for Mason
in general, though I'm not prepared to predict exactly what ;)
3. Mason will require Class::Container 0.06 (or something > 0.05), which
incorporates some good optimizations that I think will also have a
noticeable speed impact on Mason.
> This is still double of:
> mason 1.03: 0.23 seconds
Double isn't _that_ bad, especially for the majority of people out there
whose bottlenecks will _still_ be external resources. Although it sounds
like you guys have a somehwat unusual case.
> BTW, this advice is greatly appreciated. The new Mason has some very
> nice features, and we're itching to get it into production :). All of
> the suggestions are both helpful and reassuring.
Honestly, the slowdown didn't surprise me. Mason 1.10 added a lot of new
features, and involved lots of rearchitecting. We spent basically no time
optimizing, and in fact I think I actively worked to not do so (and
discourage others from doing so), because I wanted it to be _right_ first.
Now that we know we have a good design that works, we can start to
optimize appropriately.
I also think we should create some docs on this sort of stuff, since
things like the PERL_NO_VALIDATION bit are not at all obvious (unless
you're me, cause I'm the author of Params::Validate ;). And static_source
is documented, but it'd be good to consolidate this all in an "optimizing
Mason" section. Perhaps in the admin docs.
===
From: Dave Rolsky <autarch@urth.org>
To: Ken Williams <ken@mathforum.org>
Subject: Re: [Mason] Performance differences between Mason 1.03 and Mason
Date: Sun, 14 Jul 2002 23:16:17 -0500 (CDT)
On Mon, 15 Jul 2002, Ken Williams wrote:
> Class::Container gains are coming from caching, and most of the
> Params::Validate gains are coming from avoiding lots of data
> structure manipulations. Mason uses both of these modules
Its not so much the manipulations as the copying. I work _really_ hard to
avoid copying arrays/hashes more than I absolute must.
This is not to say that everyone should run out and pass references around
_all_ the time, as there's various potential problems with this (plus the
code can be less readable). It seemed to be a win in this case simply
because the code in question is called a _lot_.
There's also a lot of other funkiness that was involved. It might make an
interesting case study in Perl code optimization for someone some day.
===
From: "Jonathan Swartz" <swartz@pobox.com>
To: "'Dave Rolsky'" <autarch@urth.org>, "'Ian Robertson'" <ian@eziba.com>
Cc: <mason-users@lists.sourceforge.net>, <jay@eziba.com>
Subject: RE: [Mason] Performance differences between Mason 1.03 and Mason 1.11
Date: Sun, 14 Jul 2002 21:40:10 -0700
> - Setting the environment variable PERL_NO_VALIDATION to a
> true value will
> make Params::Validate run _much_ faster, because it won't do any
> validation ;) Turning this on in production is probably
> worthwhile but I
> wouldn't recommend it during development, except for benchmarking.
>
> The speedup that this env var provides is _much_ greater with
> version 0.20
> of P::V.
Ok. Some questions/concerns:
1. Do we know the breakdown of parameter validation? e.g. are there a few
validations that make up the majority of calls?
2. Are some/many of the parameter validations protecting internal Mason code
that shouldn't need protection once Mason is out in the field? For example,
no regular Mason user will call Interp::write_object_file or Lexer::lex
directly, so those validations shouldn't need to happen for them. Out in the
field Mason should be validating, at most, user-level API calls like
$interp->exec and $m->comp.
3. Having to set PERL_NO_VALIDATION is unfortunate, because (1) it isn't
configured like everything else, and (2) it will turn off validation
everywhere, including in the user's code and any other modules they call.
It's quite reasonable for the user to want to turn off validation in Mason
(because most of that is useful only for Mason's development) but to use
Params::Validate in their own application code.
Unfortunately making this a regular object parameter is difficult, as it
doesn't belong in any one object. Bring back global config? :) At the very
least it could be a different environment variable.
===
Subject: Re: [Mason] Performance differences between Mason 1.03 and Mason 1.11
Cc: Mason List <mason-users@lists.sourceforge.net>
To: Dave Rolsky <autarch@urth.org>
From: Ken Williams <ken@mathforum.org>
Date: Mon, 15 Jul 2002 18:23:10 +1000
On Monday, July 15, 2002, at 02:16 PM, Dave Rolsky wrote:
> On Mon, 15 Jul 2002, Ken Williams wrote:
>
>> Class::Container gains are coming from caching, and most of the
>> Params::Validate gains are coming from avoiding lots of data
>> structure manipulations. Mason uses both of these modules
>
> Its not so much the manipulations as the copying. I work
> _really_ hard to
> avoid copying arrays/hashes more than I absolute must.
Yeah, that's what I meant, actually. I remember last time I
looked at the code there was a lot of copying going on, and I'm
glad you've found ways to excise it.
===
Subject: Re: [Mason] Performance differences between Mason 1.03 and Mason 1.11
Cc: "'Dave Rolsky'" <autarch@urth.org>, "'Ian Robertson'" <ian@eziba.com>,
To: <swartz@pobox.com>
From: Ken Williams <ken@mathforum.org>
Date: Mon, 15 Jul 2002 18:37:23 +1000
On Monday, July 15, 2002, at 02:40 PM, Jonathan Swartz wrote:
> 3. Having to set PERL_NO_VALIDATION is unfortunate, because (1)
> it isn't
> configured like everything else, and (2) it will turn off validation
> everywhere, including in the user's code and any other modules
> they call.
> It's quite reasonable for the user to want to turn off
> validation in Mason
> (because most of that is useful only for Mason's development)
> but to use
> Params::Validate in their own application code.
>
> Unfortunately making this a regular object parameter is
> difficult, as it
> doesn't belong in any one object. Bring back global config? :)
It's okay for several classes to share a parameter with
Class::Container. It'll filter down to all classes that declare
it.
===
From: jmelesky@performics.com
Subject: Re: [Mason] Performance differences between Mason 1.03 and Mason 1.11
To: mason-users@lists.sourceforge.net
Date: Mon, 15 Jul 2002 14:55:42 -0500 (CDT)
On 14 Jul, Dave Rolsky wrote:
> Honestly, the slowdown didn't surprise me. Mason 1.10 added a lot of new
> features, and involved lots of rearchitecting. We spent basically no time
> optimizing, and in fact I think I actively worked to not do so (and
> discourage others from doing so), because I wanted it to be _right_ first.
I think that's valid and admirable (premature optimization being the root of
all evil, and all that), but, especially considering some of the problems
that 1.10 had, is it time to have a separate track for development releases?
Jon, i think, has complained in the past that nobody seems to use the beta
releases, which is probably true. But i think you'd get more feedback from
something that's released under a special "new features" banner. Then you'd
have time to optimize to the point of production-readiness, without people
having already deployed it mistakenly.
It would also serve as an incentive to help test. Early adopters could get
it when they wanted it (early, without needing to go into cvs), and
production-only users would get exposure to new features through the
listserver enough that they might install it just to help get it to the
production-ready version.
Just tossing the idea out there. Thoughts?
===
From: Dave Rolsky <autarch@urth.org>
To: jmelesky@performics.com
Subject: Re: [Mason] Performance differences between Mason 1.03 and Mason
Date: Mon, 15 Jul 2002 15:27:09 -0500 (CDT)
On Mon, 15 Jul 2002 jmelesky@performics.com wrote:
> I think that's valid and admirable (premature optimization being the root of
> all evil, and all that), but, especially considering some of the problems
> that 1.10 had, is it time to have a separate track for development releases?
We did. I released a 1.09_01 and 1.09_02. A _few_ people used them.
Most did not. I could have done a 1.09_03, but frankly, I think that
_most_ of the bugs fixed between 1.10 and 1.11 would _not_ have been
caught with a 1.09_03 anyway.
> Jon, i think, has complained in the past that nobody seems to use the beta
> releases, which is probably true. But i think you'd get more feedback from
> something that's released under a special "new features" banner. Then you'd
> have time to optimize to the point of production-readiness, without people
> having already deployed it mistakenly.
When I released 1.10, I said this:
While we've done a lot of testing, and have fixed all the reported bugs
from our two betas, we encourage you to consider this a ".0" release,
and to test it before putting it into production.
Anyone who rushed out and stuck it in production, and had problems with
it, probably needs to think about revising their development process.
It's worth noting that 1.10 came out on June 25, followed a mere 8 days
later by 1.11. 1.12 should be out this week, meaning it'll have been
about 2 weeks since 1.11. Anyone who grabbed 1.10 and has gotten it into
production _already_ is probably not doing any serious testing or QA work.
I believe that with 1.12 we will have resolved almost all of the major
issues with the 1.10 release. In fact, almost all of the bugs were fixed
with 1.11, and very few new ones have been reported, except for the memory
leaks fixed in 1.12. _And_ 1.12 will have much better performance.
It's worth noting that the 1.0x series, despite being relatively stable,
_still_ had bugs needing fixing even in the last release (1.05).
> It would also serve as an incentive to help test. Early adopters could get
> it when they wanted it (early, without needing to go into cvs), and
> production-only users would get exposure to new features through the
> listserver enough that they might install it just to help get it to the
> production-ready version.
Actually, what might have been better was simply to have made _fewer_
changes for 1.10 (which might as well be 2.0, actually!), in which case it
probably would have been more bug free. Future major releases will almost
certainly be a smaller jump than 1.05 => 1.10 was.
===