This is part of The Pile, a partial archive of some open source mailing lists and newsgroups.
To: modperl@apache.org
From: "Ken Williams" <ken@mathforum.org>
Subject: Apache::Session getting DESTROYed in wrong order
Date: Mon, 31 Dec 2001 17:13:33 -0600
Hey,
I'm having problems with Apache::Session, the symptom is that none of my
data is getting written to the database. It's not the nested-data
problem, since I'm not using any nested data structures.
After some investigation, I've discovered that the
Apache::Session::Store::MySQL::DESTROY routine is getting called before
the Apache::Session::MySQL::DESTROY routine, so when the latter is
called it doesn't have any way to write to the database.
I think Perl is supposed to guarantee that the outer object's DESTROY is
called before the inner object's, but I think maybe this guarantee
doesn't extend to the "global destruction" phase. So I'm wondering why
the session is getting cleaned up in global destruction rather than
refcount destruction. I've declared %session as a locally-scoped
variable, so it should evaporate before global destruction, unless it's
got circular data structures or something. Anyone know what might be
going on?
This is Apache::Session version 1.53.
Note: this problem isn't related to mod_perl, but IIRC this list is the
proper place for discussing Apache::Session.
===
To: "Aaron E. Ross" <ross@coreference.com>
From: "Ken Williams" <ken@mathforum.org>
Subject: Re: Apache::Session getting DESTROYed in wrong
order
Date: Thu, 3 Jan 2002 02:42:50 -0600
Hi Aaron,
I don't have a test case involving Apache::Session yet (I've been out of
town for a couple days), but here's a simple one in Perl that
demonstrates the DESTROY order problem:
--------------------------------------------------------------
#!/usr/bin/perl
{
package Outer;
sub new { bless {inner => 'Inner'->new} }
sub DESTROY { warn "Destroying $_[0]" }
}
{
package Inner;
sub new { bless {} }
sub DESTROY { warn "Destroying $_[0]" }
}
{
warn "refcount destruction:\n";
my $foo = 'Outer'->new;
}
warn "\nglobal destruction:\n";
my $bar = 'Outer'->new;
$bar->{self} = $bar;
--------------------------------------------------------------
So I think I need to find out why the Apache::Session objects aren't
being destroyed until global destruction time, i.e. why their refcounts
aren't going to zero.
This is in the context of testing the new HTML::Mason 1.10, so something
complicated might be happening with that too.
===
To: "Ken Williams" <ken@mathforum.org>, "Aaron E. Ross"
<ross@coreference.com>
From: "Perrin Harkins" <perrin@elem.com>
Subject: Re: Apache::Session getting DESTROYed in wrong
order
Date: Thu, 3 Jan 2002 12:57:29 -0500
> I don't have a test case involving Apache::Session yet (I've been out of
> town for a couple days), but here's a simple one in Perl that
> demonstrates the DESTROY order problem:
That's sort of a weird example, since it has a circular reference. Does it
have problems without the circular ref?
> So I think I need to find out why the Apache::Session objects aren't
> being destroyed until global destruction time, i.e. why their refcounts
> aren't going to zero.
At a guess, I'd say you're making an unintentional closure somewhere.
- Perrin
===
To: "Perrin Harkins" <perrin@elem.com>
From: "Ken Williams" <ken@mathforum.org>
Subject: Re: Apache::Session getting DESTROYed in wrong
order
Date: Thu, 3 Jan 2002 12:26:26 -0600
On Thursday, January 3, 2002, at 11:57 AM, Perrin Harkins wrote:
>> I don't have a test case involving Apache::Session yet (I've been out
>> of
>> town for a couple days), but here's a simple one in Perl that
>> demonstrates the DESTROY order problem:
>
> That's sort of a weird example, since it has a circular reference.
> Does it
> have problems without the circular ref?
The circular reference was the only way I could think of to force an
object to be destroyed during global destruction. I don't know whether
it has a problem without circularity or not.
> At a guess, I'd say you're making an unintentional closure somewhere.
>
Hmm, that may be - Mason does create more closures now than it used to.
It seems like only 'named' closures would create this problem, though,
and not 'anonymous' closures (since the refcount of the anonymous
closure itself should go to zero, freeing its contents). Mason is
supposed to be using all anonymous closures.
I'm finding the destruction behavior highly unpredictable with the
'named' closure actually, so maybe I should bring it up on p5p. Perhaps
the order of destruction is just random during global destruction,
because I've seen it happen from the inside out & from the outside in.
===
To: "Ken Williams" <ken@mathforum.org>
From: "Perrin Harkins" <perrin@elem.com>
Subject: Re: Apache::Session getting DESTROYed in wrong
order
Date: Thu, 3 Jan 2002 13:36:01 -0500
> The circular reference was the only way I could think of to force an
> object to be destroyed during global destruction.
What happens if you use a global?
> Hmm, that may be - Mason does create more closures now than it used to.
> It seems like only 'named' closures would create this problem, though,
> and not 'anonymous' closures (since the refcount of the anonymous
> closure itself should go to zero, freeing its contents).
I was thinking of this situation:
my %session = get_session();
sub transmogrify {
$session{'foo'}++;
}
I could be wrong, but I think that will make %session stick around, because
transmogrify() now has a private copy of it.
===
To: Ken Williams <ken@mathforum.org>
From: "Jeffrey W. Baker" <jwbaker@acm.org>
Subject: Re: Apache::Session getting DESTROYed in wrong
order
Date: Thu, 3 Jan 2002 12:02:08 -0800 (PST)
On Mon, 31 Dec 2001, Ken Williams wrote:
> Hey,
>
> I'm having problems with Apache::Session, the symptom is that none of my
> data is getting written to the database. It's not the nested-data
> problem, since I'm not using any nested data structures.
>
> After some investigation, I've discovered that the
> Apache::Session::Store::MySQL::DESTROY routine is getting called before
> the Apache::Session::MySQL::DESTROY routine, so when the latter is
> called it doesn't have any way to write to the database.
>
> I think Perl is supposed to guarantee that the outer object's DESTROY is
> called before the inner object's, but I think maybe this guarantee
> doesn't extend to the "global destruction" phase. So I'm wondering why
> the session is getting cleaned up in global destruction rather than
> refcount destruction. I've declared %session as a locally-scoped
> variable, so it should evaporate before global destruction, unless it's
> got circular data structures or something. Anyone know what might be
> going on?
>
> This is Apache::Session version 1.53.
>
> Note: this problem isn't related to mod_perl, but IIRC this list is the
> proper place for discussing Apache::Session.
Ken,
Yeah this is the right list for Apache::Session discussion, and Perrin is
the unofficial guy who answers all the questios, since I don't pay that
much attention.
This seems like a really weird problem. The Store module is destroyed
while another module still has a reference to it. Unfortunately for you
and I, the only conclusion I have been able to draw is that Perl's DESTROY
magic is unreliable. We have modules in my company where things are
randomly undefined in DESTROY subroutines, because the DESTROY of the
referenced thing has already been called. So, somewhere in Perl there is
a bug, possibly an off-by-one in the reference counting.
Anyway you can work around it. Explicitly call tied(%session)->save()
when the time is right.
===
To: "Jeffrey W. Baker" <jwbaker@acm.org>
From: "Ken Williams" <ken@mathforum.org>
Subject: Re: Apache::Session getting DESTROYed in wrong
order
Date: Thu, 3 Jan 2002 14:23:14 -0600
On Thursday, January 3, 2002, at 02:02 PM, Jeffrey W. Baker wrote:
> This seems like a really weird problem. The Store module is destroyed
> while another module still has a reference to it. Unfortunately for you
> and I, the only conclusion I have been able to draw is that Perl's
> DESTROY
> magic is unreliable. We have modules in my company where things are
> randomly undefined in DESTROY subroutines, because the DESTROY of the
> referenced thing has already been called. So, somewhere in Perl there
> is
> a bug, possibly an off-by-one in the reference counting.
It's probably not the reference counting, since the global destruction
phase uses a mark-and-sweep system rather than refcounts (and that's
where my knowledge ends).
I think that the order of global object destruction is totally random,
whereas refcount destruction is predictable. After searching p5p, this
seems to be known and accepted behavior.
I suppose there could be a refcount bug that's causing %session not to
be cleaned up until global destruction.
> Anyway you can work around it. Explicitly call tied(%session)->save()
> when the time is right.
True, I guess I'll do that, but I'd like to figure out a little more
about it too.
-Ken
===
To: "Perrin Harkins" <perrin@elem.com>
From: "Ken Williams" <ken@mathforum.org>
Subject: Re: Apache::Session getting DESTROYed in wrong
order
Date: Fri, 4 Jan 2002 02:22:07 -0600
Hey,
For the sake of thread completion, here's a script which demonstrates
the bug. It turns out to be a Perl bug (5.6.1, at least), not an
Apache::Session bug. I'll post to p5p after I post here.
Note that $foo and %bar are cleaned up by refcount, but %foo isn't
cleaned up until global destruction. This means there must be some bad
interaction between tie(), closures, and global variables, I guess.
-------------------------------------------------------------
#!/usr/bin/perl
use strict;
{
package Dummy;
sub new { bless {@_[1,2]} }
sub TIEHASH { bless {@_[1,2]} }
sub DESTROY { warn "Destroying $_[0]->{name}: $_[0]" }
}
use vars qw(%foo $foo);
{
# Will get cleaned up properly
local $foo = new Dummy(name => '$foo');
# Will get cleaned up properly
my %bar;
tie %bar, 'Dummy', name => '%bar';
# Won't get cleaned up properly
local %foo;
tie %foo, 'Dummy', name => '%foo';
}
------------------------------------------------------------
Destroying %bar: Dummy=HASH(0x632c) at destroy.pl line 9.
Destroying $foo: Dummy=HASH(0x641c) at destroy.pl line 9.
Destroying %foo: Dummy=HASH(0x22ccc) at destroy.pl line 9 during global
destruction.
------------------------------------------------------------
Investigating with Devel::Peek suggests that it's a %foo refcount
problem, it's somehow getting set to 2 after tie(%foo).
===
To: "Ken Williams" <ken@mathforum.org>, "Perrin Harkins"
<perrin@elem.com>
From: "Gerald Richter" <richter@ecos.de>
Subject: Re: Apache::Session getting DESTROYed in wrong
order
Date: Fri, 4 Jan 2002 09:48:57 +0100
> # Won't get cleaned up properly
> local %foo;
> tie %foo, 'Dummy', name => '%foo';
local only make a copy of the original value and restores it at the end of
the scope, so %foo will not destroyed, but restored at the end of the scope.
I guess this is the reason my it still stays tied.
In my experiences there are more weired behaviours with tied hashs and
arrays. (e.g. don't access a tied hash inside of a method of a tied hash,
use FETCH instead, tied hash element doesn't always spring into existence,
like normal hash elements does). You have to use them with some care.
>
> Investigating with Devel::Peek suggests that it's a %foo refcount
> problem, it's somehow getting set to 2 after tie(%foo).
>
2 is ok. one for %foo itself and one because it's tied to another object
Gerald
===
To: "Gerald Richter" <richter@ecos.de>
From: "Ken Williams" <ken@mathforum.org>
Subject: Re: Apache::Session getting DESTROYed in wrong
order
Date: Fri, 4 Jan 2002 17:36:30 -0600
On Friday, January 4, 2002, at 02:48 AM, Gerald Richter wrote:
>> # Won't get cleaned up properly
>> local %foo;
>> tie %foo, 'Dummy', name => '%foo';
>
> local only make a copy of the original value and restores it at the end
> of
> the scope, so %foo will not destroyed, but restored at the end of the
> scope.
> I guess this is the reason my it still stays tied.
AMS just posted this small test case to p5p:
sub X::TIEHASH{bless{}}
{ local %x; tie %x, "X" } print tied %x ? "a" : "b";
5.004_03 prints "b", and 5.004_04 (and higher) prints "a". I think "b"
is the proper behavior, at least that's my opinion.
-Ken
===
To: Ken Williams <ken@mathforum.org>
From: "Jeffrey W. Baker" <jwbaker@acm.org>
Subject: Re: Apache::Session getting DESTROYed in wrong
order
Date: Tue, 8 Jan 2002 12:41:08 -0800 (PST)
On Fri, 4 Jan 2002, Ken Williams wrote:
>
> On Friday, January 4, 2002, at 02:48 AM, Gerald Richter wrote:
>
> >> # Won't get cleaned up properly
> >> local %foo;
> >> tie %foo, 'Dummy', name => '%foo';
> >
> > local only make a copy of the original value and restores it at the end
> > of
> > the scope, so %foo will not destroyed, but restored at the end of the
> > scope.
> > I guess this is the reason my it still stays tied.
>
> AMS just posted this small test case to p5p:
>
> sub X::TIEHASH{bless{}}
> { local %x; tie %x, "X" } print tied %x ? "a" : "b";
>
> 5.004_03 prints "b", and 5.004_04 (and higher) prints "a". I think "b"
> is the proper behavior, at least that's my opinion.
Well, you can say I'm cold-hearted, but I think if you use every feature
Perl has in one line, you can expect trouble. Apache::Session has a
history of this. With every Perl < 5.6, it cause a segfault when calling
die() from within TIEHASH.
-jwb
===
To: "Ken Williams" <ken@mathforum.org>
From: "Ken Williams" <ken@mathforum.org>
Subject: Re: Apache::Session getting DESTROYed in wrong
order
Date: Thu, 17 Jan 2002 23:54:13 -0600
On Friday, January 4, 2002, at 02:22 AM, Ken Williams wrote:
> For the sake of thread completion, here's a script which demonstrates
> the bug. It turns out to be a Perl bug (5.6.1, at least), not an
> Apache::Session bug. I'll post to p5p after I post here.
I was surprised to find the "it's not a bug, it's a feature" defense on
p5p. So here's an update. The following is either a workaround, or the
proper fix, depending on what you think Perl's proper behavior should
be. ;-)
{
local *session;
tie %session, 'Apache::Session::MySQL', ...;
...
}
The "local *session;" is the important bit. It doesn't work to do
"local %session;", because %session will still be tied even after it
goes out of scope, and thus the hash data will never get written to
storage.
In a Mason context, which is where I'm using it, I do this in my
top-level autohandler (ignore the main:: subroutines, they're just for
pedagogy):
<%init>
# 'local' so it's available to lower-level components
local *session;
my $dbh = &::get_dbh;
my $session_id = &::get_cookie('_session_id');
tie %session, 'Apache::Session::MySQL', $session_id,
{Handle => $dbh, LockHandle => $dbh};
...
</%init>
===
To: "Ken Williams" <ken@mathforum.org>
From: "Perrin Harkins" <perrin@elem.com>
Subject: Re: Apache::Session getting DESTROYed in wrong
order
Date: Fri, 18 Jan 2002 01:44:06 -0500
> In a Mason context, which is where I'm using it, I do this in my
> top-level autohandler (ignore the main:: subroutines, they're just for
> pedagogy):
>
>
> <%init>
> # 'local' so it's available to lower-level components
> local *session;
>
> my $dbh = &::get_dbh;
> my $session_id = &::get_cookie('_session_id');
> tie %session, 'Apache::Session::MySQL', $session_id,
> {Handle => $dbh, LockHandle => $dbh};
> ...
> </%init>
Geez, that's awfully confusing to look at (local and typeglobs is not a
newbie-friendly combo). Isn't there a simpler way? What about putting
it in pnotes?
===
To: "Perrin Harkins" <perrin@elem.com>
From: "Ken Williams" <ken@mathforum.org>
Subject: Re: Apache::Session getting DESTROYed in wrong
order
Date: Fri, 18 Jan 2002 00:53:34 -0600
On Friday, January 18, 2002, at 12:44 AM, Perrin Harkins wrote:
>> In a Mason context, which is where I'm using it, I do this in my
>> top-level autohandler (ignore the main:: subroutines, they're just for
>> pedagogy):
>>
>>
>> <%init>
>> # 'local' so it's available to lower-level components
>> local *session;
>>
>> my $dbh = &::get_dbh;
>> my $session_id = &::get_cookie('_session_id');
>> tie %session, 'Apache::Session::MySQL', $session_id,
>> {Handle => $dbh, LockHandle => $dbh};
>> ...
>> </%init>
>
> Geez, that's awfully confusing to look at (local and typeglobs is not a
> newbie-friendly combo). Isn't there a simpler way? What about putting
> it in pnotes?
I don't think there's a simpler way. Putting it in pnotes means that
all other components will also have to use $r->pnotes('session'), rather
than just using %session.
Perhaps "local(*session)" is better than "local *session"? It at least
looks less like a "pointer to local". ;-)
===
To: "Perrin Harkins" <perrin@elem.com>
From: "Jay Lawrence" <Jay@Lawrence.Net>
Subject: Re: Apache::Session getting DESTROYed in wrong
order
Date: Fri, 18 Jan 2002 09:11:47 -0500
"Ken Williams" <ken@mathforum.org> wrote:
> On Friday, January 18, 2002, at 12:44 AM, Perrin Harkins wrote:
>
> >> In a Mason context, which is where I'm using it, I do this in my
> >> top-level autohandler (ignore the main:: subroutines, they're just for
> >> pedagogy):
> >>
> >>
> >> <%init>
> >> # 'local' so it's available to lower-level components
> >> local *session;
> >>
> >> my $dbh = &::get_dbh;
> >> my $session_id = &::get_cookie('_session_id');
> >> tie %session, 'Apache::Session::MySQL', $session_id,
> >> {Handle => $dbh, LockHandle => $dbh};
> >> ...
> >> </%init>
> >
> > Geez, that's awfully confusing to look at (local and typeglobs is not a
> > newbie-friendly combo). Isn't there a simpler way? What about putting
> > it in pnotes?
>
> I don't think there's a simpler way. Putting it in pnotes means that
> all other components will also have to use $r->pnotes('session'), rather
> than just using %session.
>
> Perhaps "local(*session)" is better than "local *session"? It at least
> looks less like a "pointer to local". ;-)
I register a clean up handler to explicitly untie the
session variable. I am not sure how to do this in the setup
you have running...so I can't be of much explicit help.
===
To: "Jay Lawrence" <Jay@Lawrence.Net>
From: "Perrin Harkins" <perrin@elem.com>
Subject: Re: Apache::Session getting DESTROYed in wrong
order
Date: Fri, 18 Jan 2002 11:00:38 -0500
> I register a clean up handler to explicitly untie the session variable.
I have found that it's safer to put things in pnotes than to use globals and
cleanup handlers. We used a lot of cleanup handlers at eToys to clear
globals holding various request-specific things, and we started getting
unpredictable segfaults. When I moved them to pnotes instead the segfaults
went away. I think it may have had something to do with cleanup handlers
running in an unpredictable order and some of them trying to use things that
were already cleaned up, so it was probably my fault, but pnotes just seems
a bit more foolproof.
===
To: modperl@apache.org
From: William White <whitew@library.ucsf.edu>
Subject: locking bug in Apache::Session::File
Date: Fri, 18 Jan 2002 13:17:05 -0800
I've been told this is the place to send questions related to apache perl
modules.
I believe I have discovered a locking bug in Apache::Session::File.
The following code should retrieve an existing session from the file system
and place an exclusive lock on the session file:
my $locking_args = { 'Directory' => '/tmp/sessions_dir',
'LockDirectory' => '/tmp/sessions_lock_dir',
'Transaction' => '1' };
tie(%session, 'Apache::Session::File', $session_id, $locking_args);
The 'locking_args' hash is used to pass parameters to the locking object
contained by the session object. According to the Apache::Session
documentation any true value of "Transaction" should force the object to
exclusively lock the session file. Unfortunately this does not appear to
work (at least not all the time).
Looking in the TIEHASH I think I've discovered the reason. The session
uses a locking object. In this case the locking object is
Apache::Session::Lock::File. This object has two methods which acquire
locks, aptly named acquire_read_lock and acquire_write_lock. The first
method uses flock to acquire a non-exclusive lock. The second method uses
flock to acquire an exclusive lock. TIEHASH checks the value of
'Transaction' and calls acquire_write_lock if the value is true. It then
calls a method named restore. It does this regardless of the value of
'Transaction'. The restore method calls acquire_read_lock. Again it does
this without examining the value of 'Transaction'.
Now according to the flock man page if a process requests a lock on a file
it already has locked, then the new lock will replace the old one. Thus
requesting a non-exclusive lock on file which the process already has an
exclusive lock for will cause the non-exclusive lock to replace the
exclusive one.
The call to acquire_read_lock in the restore method wipes out the exclusive
lock on the session file. This makes it impossible to maintain
transactional consistency with Apache::Session::File.
I was wondering if anyone else out there has run into this problem. Is
there a fix available? My version of Apache::Session is 1.54 which is the
newest version that I see on CPAN. Is there another version out there that
fixes this problem or should I bring this up with the author?
===