modperl-session_refresh_philosophy

This is part of The Pile, a partial archive of some open source mailing lists and newsgroups.



To: modperl@apache.org
From: Milo Hyson <milo@cyberlifelabs.com>
Subject: Session refresh philosophy
Date: Mon, 18 Feb 2002 18:22:23 -0800

Like my previous question on object caching, this one is
potentially a matter of style as well. When it comes to
implementing expirations on session data, I've encountered
two schools of thought on when is best to refresh the
timestamp/expiration. In that the general idea of expiration
is to discard information that hasn't been accessed in a
while, some feel that updating the timestamp is best done
during both loading and storing. After all, both are
considered accessing the data. However, taking into account
the general pattern of HTTP request processing, I feel that
updating only during storage is best, especially when using
a database for persistence.

Suppose one has a SQL table for saving session data. When a
request comes in, the session is loaded and its expiration
is examined. Assuming the session is still valid, one could
issue another statement to the database to refresh the
session's expiration time. That's two database ops before
the session is even used. If you count the one at the end
for storing the session back in the database it's a total of
three per request. My feeling is that if you're going to be
writing the session back within (hopefully) a fraction of a
second anyway, you might as well wait until then to refresh
the time-out.

The project I'm working on requires that I design a custom
application platform for current and future projects. My
proposed solution to the session management problem is as
follows:

1) A fix-up handler is called to extract the session ID from
a cookie.  Assuming a valid ID was found, the session is
loaded, de-serialized and checked for expiration. If all is
well, the a reference to the session is stored in pnotes for
use by the application.

1a) If for some reason no session was found (e.g. no cookie)
a new one is created and a new cookie is stuffed in the
outgoing headers.

2) During content-generation, the application obtains the
session reference from pnotes and uses it as necessary.

3) A clean-up handler is called to re-serialize the session
and stick it back in persistent storage (updating the
expiration in the process). The handler of course does
nothing if the application destroyed the session in step 2.

I'm still fairly new to mod_perl and haven't fully taken
apart all of the various application servers out there to
see how they do it. I would still appreciate any feedback
anyone may have on the above.

====


To: modperl@apache.org
From: Rob Nagler <nagler@bivio.biz>
Subject: Re: Session refresh philosophy
Date: Mon, 18 Feb 2002 20:29:26 -0700

Milo Hyson writes:
> 1) A fix-up handler is called to extract the session ID from a cookie. 
[snip]
> 1a) If for some reason no session was found (e.g. no cookie) a new one is 
[snip]
> 2) During content-generation, the application obtains the session reference 
[snip]
> 3) A clean-up handler is called to re-serialize the session and stick it back

I may be asking the wrong question: is there a need for sessions?
This seems like a lot of work when, for most applications, sessions
are unnecessary.

===

To: Rob Nagler <nagler@bivio.biz>, modperl@apache.org
From: Milo Hyson <milo@cyberlifelabs.com>
Subject: Re: Session refresh philosophy
Date: Mon, 18 Feb 2002 23:30:23 -0800

On Monday 18 February 2002 07:29 pm, Rob Nagler wrote:
> I may be asking the wrong question: is there a need for sessions?
> This seems like a lot of work when, for most applications, sessions
> are unnecessary.

I don't see how they could be unnecessary for what we're doing. Then again, 
maybe I'm just approaching the problem incorrectly. If one is doing a 
shopping-cart-style application (whereby someone selects/configures multiple 
items before they're ultimately committed to a database) how else would you 
do it? There has to be some semi-persistent (i.e. inter-request) data where 
selections are stored before they're confirmed.

===

To: modperl@apache.org
From: Rob Nagler <nagler@bivio.biz>
Subject: Re: Session refresh philosophy
Date: Tue, 19 Feb 2002 10:17:52 -0700

Milo Hyson writes:
> shopping-cart-style application (whereby someone selects/configures multiple 
> items before they're ultimately committed to a database) how else would you 
> do it? There has to be some semi-persistent (i.e. inter-request) data where 
> selections are stored before they're confirmed.

As I understand it, the session data is "state" which is committed to
the database on each request (possibly).  It would seem to me that
instead of denomalizing the state into a separate session table, you
should just store it in a normal table.  If the data needs to be
expired, then it can be time stamped when it is written.

The point is that it's always simpler to use the existing tables
directly rather than making a copy and storing it in the database
somewhere else.  This usually reduces the code by half or more,
because you don't have to worry about making the copy in the first
place.  Simpler code is more reliable and usually runs faster.

To me, sessions are negativist.  My expectation is that users will end
up clicking OK (making the purchase).  If that is the case, you are
much better off putting the data were belongs right from start.  You
may bind it to an ephemeral entity, such as a shopping cart, but when
the order is complete the only thing you have to do is free the cart
and replace it with an order.  The items, amounts, and special
considerations have already been stored.

If most of your users are filling shopping baskets and walking away
from them, it may be a problem with the software.  Checkout
http://www.useit.com for some ideas on how to improve the ratio.

Often you can avoid any server side persistence by using hidden fields
in the forms.  We use this technique extensively, and we have
encapsulated it so that it is easy to use.  For example, you might
have a sub form which asks the user to fill in an address.  When the
user clicks on the "fill in address" button, the server squirrels away
the context of the current form in the hidden fields of the address
form.  When the user clicks OK on the address form, the fields are
stuffed back into the original form including the new address.

If you have a performance problem, solve it when you can measure it.
Sessions can mitigate performance problems, but so can intelligent
caching, which avoids statefulness in the client-server protocol.

Rob

P.S. For sample sessionless sites, visit http://www.bivio.com and
     http://petshop.bivio.biz (which runs on a 3 year old 300mhz box
     running Apache and Postgres).

===

To: "Rob Nagler" <nagler@bivio.biz>, <modperl@apache.org>
From: "Perrin Harkins" <perrin@elem.com>
Subject: Re: Session refresh philosophy
Date: Tue, 19 Feb 2002 12:35:32 -0500

> As I understand it, the session data is "state" which is committed to
> the database on each request (possibly).  It would seem to me that
> instead of denormalizing the state into a separate session table, you
> should just store it in a normal table.

The typical breakdown I use for this is to put simple state information that
connects this browser to long-term data in the session, and everything else
in normal database tables.  So, I put the user's ID (if this session belongs
to an identified user), a flag telling whether or not this user has given a
secure login so far in this session, and not much else in the session.

Actually, even this stuff could be put into a normalized "sessions" table
rather than serialized to a blob with Storable.  It just means more work if
you ever change what's stored in the session.

===

To: "Milo Hyson" <milo@cyberlifelabs.com>,
From: "Perrin Harkins" <perrin@elem.com>
Subject: Re: Session refresh philosophy
Date: Tue, 19 Feb 2002 12:39:33 -0500

> In that the general idea of expiration is to discard
> information that hasn't been accessed in a while, some feel that updating
the
> timestamp is best done during both loading and storing.

If you don't always store the session (Apache::Session doesn't store unless
you modify data in the session) then you have to update the timestamp on
load to be accurate.  If you want to be frugal, you might make the store
operation update the timestamp, and then install a cleanup handler that will
update the timestamp only if the session was not stored.

- Perrin

===

To: modperl@apache.org
From: Rob Nagler <nagler@bivio.biz>
Subject: Re: Session refresh philosophy
Date: Tue, 19 Feb 2002 11:07:43 -0700

Perrin Harkins writes:
> Actually, even this stuff could be put into a normalized "sessions" table
> rather than serialized to a blob with Storable.  It just means more work if
> you ever change what's stored in the session.

This is a tough question.  If you store it in a blob, you can't query
it with an ad hoc SQL query.  If you store it in a table, you have to
deal with data evolution.  On the whole, I vote for tables over blobs.
My reasoning is that you have to deal with data evolution anyway.  We
have had about 200 schema changes in the last two years, and very few
of them have had anything to do with user/visitor state.

===

To: Milo Hyson <milo@cyberlifelabs.com>
From: Ged Haywood <ged@www2.jubileegroup.co.uk>
Subject: Re: [OT-ish] Session refresh philosophy
Date: Tue, 19 Feb 2002 11:42:59 +0000 (GMT)

Hi there,

On Mon, 18 Feb 2002, Milo Hyson wrote:

> maybe I'm just approaching the problem incorrectly. If one is doing a 
> shopping-cart-style application (whereby someone selects/configures multiple 
> items before they're ultimately committed to a database) how else would you 
> do it? There has to be some semi-persistent (i.e. inter-request) data where 
> selections are stored before they're confirmed.

You can for example send a hidden <form> object back and forth between
your Client and the app.

===

To: mod_perl Mailing List <modperl@apache.org>
From: Drew Taylor <drew@drewtaylor.com>
Subject: Re: [OT-ish] Session refresh philosophy
Date: Tue, 19 Feb 2002 17:50:31 -0500

At 11:42 AM 2/19/2002 +0000, Ged Haywood wrote:
>Hi there,
>
>On Mon, 18 Feb 2002, Milo Hyson wrote:
>
> > maybe I'm just approaching the problem incorrectly. If one is doing a
> > shopping-cart-style application (whereby someone selects/configures 
> multiple
> > items before they're ultimately committed to a database) how else would 
> you
> > do it? There has to be some semi-persistent (i.e. inter-request) data 
> where
> > selections are stored before they're confirmed.
>
>You can for example send a hidden <form> object back and forth between
>your Client and the app.

And that is what I am doing for a small project I'm working on now. In my 
case, I'm not sure about the capabilities of the remote server, and I know 
for sure that I don't have a database available, so session information is 
saved via hidden form fields. It's primitive, but was actually a bit of a 
challenge to make sure a (unused) hidden field and a visible form element 
don't appear in the same <form>. Not my first choice, but it definitely works.

===

To: "mod_perl Mailing List" <modperl@apache.org>,
From: "Perrin Harkins" <perrin@elem.com>
Subject: Re: [OT-ish] Session refresh philosophy
Date: Tue, 19 Feb 2002 17:55:09 -0500

> And that is what I am doing for a small project I'm working on now. In my
> case, I'm not sure about the capabilities of the remote server, and I know
> for sure that I don't have a database available, so session information is
> saved via hidden form fields. It's primitive, but was actually a bit of a
> challenge to make sure a (unused) hidden field and a visible form element
> don't appear in the same <form>. Not my first choice, but it definitely
works.

Incidentally, this is mostly the same thing as what Jeffrey Baker mentioned
a few days ago about storing state entirely inside a cookie with a message
digest.  The only difference is that by sticking it in a form element you're
attaching it to a specific page.

===

To: "Perrin Harkins" <perrin@elem.com>,
From: Drew Taylor <drew@drewtaylor.com>
Subject: Re: [OT-ish] Session refresh philosophy
Date: Tue, 19 Feb 2002 18:09:36 -0500

At 05:55 PM 2/19/2002 -0500, Perrin Harkins wrote:
>Incidentally, this is mostly the same thing as what Jeffrey Baker mentioned
>a few days ago about storing state entirely inside a cookie with a message
>digest.  The only difference is that by sticking it in a form element you're
>attaching it to a specific page.

True. I was very intrigued by his approach, and might use something like 
that to increase the security of my app by verifying the hidden form field 
contents. I suppose I could follow his approach, but the amount of data I 
need to store could possibly overwhelm the 4KB cookie limit. In this case, 
simple was better - simple application, simple session. And I know I can 
count on every browser implementing forms. :-)

===

To: "Perrin Harkins" <perrin@elem.com>,
From: Milo Hyson <milo@cyberlifelabs.com>
Subject: Re: [OT-ish] Session refresh philosophy
Date: Tue, 19 Feb 2002 15:09:25 -0800

On Tuesday 19 February 2002 02:55 pm, Perrin Harkins wrote:
> Incidentally, this is mostly the same thing as what Jeffrey Baker mentioned
> a few days ago about storing state entirely inside a cookie with a message
> digest.  The only difference is that by sticking it in a form element
> you're attaching it to a specific page.

That's not a bad idea. I guess if you're paranoid about snooping you could 
always encrypt the cookie.

===

To: "Drew Taylor" <drew@drewtaylor.com>,
From: "David Harris" <dharris@drh.net>
Subject: RE: [OT-ish] Session refresh philosophy
Date: Tue, 19 Feb 2002 22:08:23 -0500

Drew Taylor [mailto:drew@drewtaylor.com]:
> And that is what I am doing for a small project I'm working on now. In my
> case, I'm not sure about the capabilities of the remote server, and I know
> for sure that I don't have a database available, so session information is
> saved via hidden form fields. It's primitive, but was actually a bit of a
> challenge to make sure a (unused) hidden field and a visible form element
> don't appear in the same <form>. Not my first choice, but it definitely
works.

I built and use a module that encodes a session hash into a number of hidden
fields with a security MD5 sum. The encoded information is serialized,
gzipped, Base64 encoded, and then split into reasonable length hidden
fields. It looks like this:

<input type="hidden" name="_fc_part000"
value="eJx9zVEKwyAQhOEbhSiBhL2MrI1GW3VhR8n1a6HPeR6++ZNZCQd15YYYFIZWWEuQ2G/W4
IKqqDul">
<input type="hidden" name="_fc_part001"
value="cm5Ic7Ab3UneXNmL7ym3C33EuLykTmywE0IpLp9jHu/0l2ye5UZ+lM+v/gUagTUd">
<input type="hidden" name="_fc_security"
value="e99478182b7c579ce65dddb676bbe52e">

This way, you don't have to worry about creating hidden form fields in your
templates for every variable you need to encode. In your perl, simply call
the session encode and decode methods. You are also assured that nobody
messed with the data.

You can easily "pass" arbitrarily complex session information from one page
to another without using a database, and the session info is truly tied to
the *page*. Use of the back button, therefore, doesn't break anything.

I've attached some code. To use the code, you'll have to replace the module
FreezeThawLite with Storable. Also, beware the \r\n newlines. (I pulled this
out of CVS on my windows desktop.)

  ------=_NextPart_000_0020_01C1B992.4156B5D0
Content-Type: text/plain;
	name="FormContainer.pm.txt"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
	filename="FormContainer.pm.txt"


package Fusion::FormContainer;
use strict;

use Digest::MD5 ();
use MIME::Base64 ();
use FreezeThawLight ();
use Compress::Zlib ();

use Carp;

# this respresents a securty hole if we open-source this module.. the =
securtiy string
# needs to be passed as confguration at that point somehow.

sub _create_security_string
{
	my $string =3D shift;

	my $secret =3D <<EOT;
 --begin-secret--
enter your own secret binary of base64 encoded data here
enter your own secret binary of base64 encoded data here
enter your own secret binary of base64 encoded data here
enter your own secret binary of base64 encoded data here
-end-secret--
EOT

	my $ctx =3D Digest::MD5->new;
	$ctx->add($string);
	$ctx->add($secret);
	return $ctx->hexdigest;
}

sub encode
{
	my $info =3D shift;
	my $prefix =3D shift;

	my $string =3D =
MIME::Base64::encode(Compress::Zlib::compress(FreezeThawLight::freeze($in=
fo)));
	$string =3D~ s/\n$//;

	my @array;

	push @array, ("${prefix}_fc_security", =
_create_security_string($string));

	my $part_number =3D 0;
	foreach my $part ( split "\n", $string ) {
		my $part_number_string =3D sprintf("%.3d", $part_number);
		push @array, ("${prefix}_fc_part$part_number_string", $part);
		$part_number++;
	}

	if ( wantarray ) {
		return @array;
	} else {
		my $html;
		while ( @array ) {
			my $name =3D shift(@array);
			my $value =3D shift(@array);
			$html .=3D <<EOT;
<input type=3D"hidden" name=3D"$name" value=3D"$value">
EOT
		}
		return $html;
	}
}

sub decode
{
	my $apr =3D shift;
	my $prefix =3D shift;

	my $security =3D $apr->param("${prefix}_fc_security");

	my @string_parts;
	my $part_number =3D 0;
	while ( 1 ) {
		my $part_number_string =3D sprintf("%.3d", $part_number);
		my $part =3D $apr->param("${prefix}_fc_part$part_number_string");
		last if ( $part eq "" );
		push @string_parts, $part;
		$part_number++;
	}
	my $string =3D join "\n", @string_parts;

	croak "tampered or malformed FormContainer: securty string and string =
don't match"
		if ( _create_security_string($string) ne $security );

	return =
FreezeThawLight::thaw(Compress::Zlib::uncompress(MIME::Base64::decode($st=
ring)));
}

1;

  ------=_NextPart_000_0020_01C1B992.4156B5D0--



===



To: "David Harris" <dharris@drh.net>, "Drew Taylor"
From: "Perrin Harkins" <perrin@elem.com>
Subject: Re: [OT-ish] Session refresh philosophy
Date: Tue, 19 Feb 2002 22:51:06 -0500

> I built and use a module that encodes a session hash into a number of
hidden
> fields with a security MD5 sum.

Sounds a lot like CGI::SecureState.  Have you ever looked at it?

- Perrin

===

To: David Harris <dharris@drh.net>
From: Hans Juergen von Lengerke <lengerkeh@sixt.de>
Subject: Re: [OT-ish] Session refresh philosophy
Date: Wed, 20 Feb 2002 10:16:18 +0100 (CET)

David Harris <dharris@drh.net> on Feb 19, 2002:

> The encoded information is [...] split into reasonable length hidden
> fields.

Why not put everything in one field? Are there restrictions? Does it
make a difference when using POST?

===

To: "Perrin Harkins" <perrin@elem.com>, "Drew Taylor"
From: "David Harris" <dharris@drh.net>
Subject: RE: [OT-ish] Session refresh philosophy
Date: Wed, 20 Feb 2002 09:50:11 -0500

Perrin Harkins [mailto:perrin@elem.com] wrote:
> > I built and use a module that encodes a session hash into a
> > number of hidden fields with a security MD5 sum.
>
> Sounds a lot like CGI::SecureState.  Have you ever looked at it?

I just installed and played with CGI::SecureState (using the example in the
POD) and it is totally different than my module.

When I used CGI::SecureState it gave the client a non-versioning (more on
that later) key and stored the state information in the filesystem.

My module doesn't need to store any information in a database or in the
filesystem. The entire state is given to the client in hidden form fields,
and is passed back to the server on the next request.

In addition, CGI::SecureState does not tie the state information to the
*page*. With my module (or any method that stores the *data* in a hidden
form field, not just a non-versioning key), state information is tied to the
page. Let me explain:

Imagine a multi-step order process where the user works through pages A, B,
C, and D, (which contain forms) then uses the back button to go back to page
B, changes the form values, and submits the form. With CGI::SecureState,
page C will receive the state information stored by page D (that was
intended for use by page E, we presume), instead of the state originally
stored by page B (that was intended for page C). This is because all the
pages share the same key, and old "versions" of the state are overwritten by
the new "versions" and no longer available. When the back button is hit, a
newer version of state may be used where an older version was intended.

With my module, page C always gets the state information stored for it by
page B, since the state is stored in hidden form fields in page B. The
browser is actually storing the state and will always submit that same state
to page C.

(I have mentioned that CGI::SecureState uses a non-versioning key a few
times. A way to make CGI::SecureState tie the state information to the
actual page would be to change the key whenever the state changed, thus
creating a versioning key. The key could be a hash of the state itself. This
potentially means that a huge number of versions of the state would have to
be stored on disk. I think this method would only be helpful if the state is
large and it's not acceptable to pass it back and forth between the client.)

In addition, CGI::SecureState gets fuddled if the user opens a new window
(something I do often) and then starts performing different operations in
each window using the same state key! It has no way of knowing a new windows
exists and generating a new key.

If you just store a customer id, customer name, and other rarely changing
information in the state, these concerns may not matter to you. If you break
a long form or order process into multiple pages, gathering new information
on each page and storing it in the state so that you can process the order
at the end, then this is a likely problem.

David

===

To: mod_perl Mailing List <modperl@apache.org>
From: Rob Nagler <nagler@bivio.biz>
Subject: Re: [OT-ish] Session refresh philosophy
Date: Wed, 20 Feb 2002 07:55:46 -0700

Hans Juergen von Lengerke writes:
> Why not put everything in one field? Are there restrictions? Does it
> make a difference when using POST?

That's what we do.  There doesn't appear to be a restriction with
POST.

For while, we were encoding entire forms in URLs, but the limits got
to us for really large forms.

Rob
===

To: "Hans Juergen von Lengerke" <lengerkeh@sixt.de>
From: "David Harris" <dharris@drh.net>
Subject: RE: [OT-ish] Session refresh philosophy
Date: Wed, 20 Feb 2002 10:05:49 -0500

Hans Juergen von Lengerke [mailto:lengerkeh@sixt.de] wrote:
> David Harris <dharris@drh.net> on Feb 19, 2002:
> > The encoded information is [...] split into reasonable length hidden
> > fields.
>
> Why not put everything in one field? Are there restrictions? Does it
> make a difference when using POST?

The POST encoding dose not have a limit on data length. Heck, people use
<textarea> tags with huge amounts of content all the time.

However, I didn't feel comfortable assuming that the HTML parser used by the
browser could easily parse a potentially 20kb attribute. Basically, I didn't
want to make my production application a stress-test for my user's browsers.
:-)

It was easy to break the data up into multiple hidden fields, because Base64
encoding breaks the data into multiple lines by default. I simply encoded
each line in one hidden field.

I would *NOT* use my module with a GET form if you expect any size of data.
I've seen the query string get truncated at some arbitrary size limit.

===

To: "David Harris" <dharris@drh.net>, "Drew Taylor"
From: "Perrin Harkins" <perrin@elem.com>
Subject: Re: [OT-ish] Session refresh philosophy
Date: Wed, 20 Feb 2002 10:19:54 -0500

> When I used CGI::SecureState it gave the client a non-versioning (more on
> that later) key and stored the state information in the filesystem.

Okay, I only looked at it briefly and thought it stored the data on the
client.  Your module is actually more like CGI::EncryptForm I think, but
yours may make things a bit more transparent.  Maybe you should polish it up
for CPAN.

I'm well aware of the page-state vs. browser-state problem.  I was recently
bitten by it again when some consultants built a web app for my company that
puts the search results in a session keyed on a cookie.  As soon as the user
opens two windows, it's absolute mayhem.

- Perrin

===

To: "David Harris" <dharris@drh.net>
From: wsheldah@lexmark.com
Subject: RE: [OT-ish] Session refresh philosophy
Date: Wed, 20 Feb 2002 11:21:04 -0500


I can see how your approach adds functionality by performing as expected if
the user uses the Back button or opens the app. in more than one browser
window. The usual objection I've heard to using form fields is the security
risk of people changing hidden fields in ways unforseen before submitting
the form back, or of other people finding confidential data hidden in form
fields if the user walks away and leaves their browser open, or the web
page info gets hijacked somehow. Does your module address this, or is this
yet another tradeoff between security and functionality/convenience?

Wes Sheldahl

===

To: mod_perl Mailing List <modperl@apache.org>
From: dom@idealx.com
Subject: Re: [OT-ish] Session refresh philosophy
Date: Wed, 20 Feb 2002 17:48:28 +0100

> The usual objection I've heard to using form fields is the security
> risk of people changing hidden fields in ways unforseen before submitting
> the form back, or of other people finding confidential data hidden in form
> fields if the user walks away and leaves their browser open, or the web
> page info gets hijacked somehow. Does your module address this, or is this
> yet another tradeoff between security and functionality/convenience?

No, this just means that input must be validated once again when the
last 

the rest of The Pile (a partial mailing list archive)

doom@kzsu.stanford.edu