modperl_adaptor_scheduler

This is part of The Pile, a partial archive of some open source mailing lists and newsgroups.



To: modperl@apache.org
From: "Daniel Sully" <daniel@electricrain.com>
Subject: Using modperl as an 'adaptor' front end to an app
server.
Date: Thu, 1 Feb 2001 22:28:50 -0800

Is anyone using modperl in a way that it acts as an adaptor/scheduler in
front of an app server in a 3-tier application environment?

Basically I have a vendor provided (with source however) adaptor that
takes incoming requests to the webserver, and passes that request onto
an any number of applications via a socket running on a different
server. It handles failover from dead app instances, however not very
well, and is a big pile of C code. It also has problems in that because
Apache is not multithreaded, one child copy of that adaptor that has
marked an app instance as dead can't let another child know about that
dead instance.

Before I reinvent this C wheel in modperl, has anyone done so already?

===

To: "Daniel Sully" <daniel@electricrain.com>,
modperl@apache.org
From: Gunther Birznieks <gunther@extropia.com>
Subject: Re: Using modperl as an 'adaptor' front end to an
app server.
Date: Fri, 02 Feb 2001 18:44:07 +0800

Yes, we do this for several clients using SOAP as an RPC transport to Java 
middleware.

I think you could pretty easily just use socket timeout setting to say that 
if you don't get a response back within a period of time that you fail over 
to another server.

The downside for SOAP is that the connection is not a persistent socket. 
SOAP is a protocol running over HTTP so you may find it slower than 
sockets. However, for many applications it's fast enough.

I've also used PL/RPC, but I can't say that I was very happy with the 
stability of the PL/RPC server but that was over a year ago. And it could 
have been due to some XS modules that were persistent in the PL/RPC server.

Anyway, you should be able to implement your own custom handler to talk 
your proprietary socket protocol in mod_perl.

===

To: Gunther Birznieks <gunther@extropia.com>
From: "G.W. Haywood" <ged@www.jubileegroup.co.uk>
Subject: Re: Using modperl as an 'adaptor' front end to an
app server.
Date: Fri, 2 Feb 2001 10:57:53 +0000 (GMT)

Hi all,

On Fri, 2 Feb 2001, Gunther Birznieks wrote:

> Anyway, you should be able to implement your own custom handler to talk 
> your proprietary socket protocol in mod_perl.

I did this last year for a Client in the Big Smoke.  Only about twenty lines.
Sorry, can't give a way any source - it's their property.

===

To: modperl@apache.org
From: Martin Wood <mwood@orctel.co.uk>
Subject: Re: Using modperl as an 'adaptor' front end to an
app server.
Date: Fri, 2 Feb 2001 11:41:18 +0000

On Fri, Feb 02, 2001 at 06:44:07PM +0800, Gunther Birznieks wrote:
> Yes, we do this for several clients using SOAP as an RPC transport to Java 
> middleware.

Out of interest, are all these clients using different SOAP
implementations on different platforms?

We are designing a system that accepts XML messages from
disparate sources and routes them to their destination based
on content and are thinking of using SOAP to formalise the
encoding method (transportation is via HTTP), yet I keep
reading about issues with interoperability between different
SOAP implementations which defeats the purpose as far as I'm
concerened.

Any experiences in this area appreciated. (Off-list?)

===

To: Daniel Sully <daniel@electricrain.com>
From: Dave Rolsky <autarch@urth.org>
Subject: Re: Using modperl as an 'adaptor' front end to an
app server.
Date: Fri, 2 Feb 2001 09:33:20 -0600 (CST)

On Thu, 1 Feb 2001, Daniel Sully wrote:

> server. It handles failover from dead app instances, however not very
> well, and is a big pile of C code. It also has problems in that because
> Apache is not multithreaded, one child copy of that adaptor that has
> marked an app instance as dead can't let another child know about that
> dead instance.

You could use an IPC or DBM file (assuming you have 1 apache machine) to
communicate this info.  With multiple webservers, you'd need a database or
NFS or something.

===

To: "Daniel Sully" <daniel@electricrain.com>,
modperl@apache.org
From: kyle dawkins <kyle@centralparksoftware.com>
Subject: Re: Using modperl as an 'adaptor' front end to an
app server.
Date: Fri, 2 Feb 2001 12:02:58 -0500

Daniel

A wild guess here but are you talking about WebObjects?

On Fri, 02 Feb 2001 01:28, Daniel Sully wrote:
> Is anyone using modperl in a way that it acts as an adaptor/scheduler in
> front of an app server in a 3-tier application environment?

I wrote a perl adaptor last year to sniff the traffic that the WebObjects 
adaptor was sending to the application instances.  I don't have the source 
any more but as far as I can remember, it was only a few screensful of code.  
However, it wasn't a full implementation of the adaptor and didn't work with 
KeepAlive requests and so forth.

> Basically I have a vendor provided (with source however) adaptor that
> takes incoming requests to the webserver, and passes that request onto
> an any number of applications via a socket running on a different
> server. It handles failover from dead app instances, however not very
> well, and is a big pile of C code. It also has problems in that because
> Apache is not multithreaded, one child copy of that adaptor that has
> marked an app instance as dead can't let another child know about that
> dead instance.

I have often dreamt of writing a drop-in mod_perl replacement for the WO 
adaptor... I have never worked anywhere where they didn't have trouble 
configuring the Apache adaptor, particularly if their webservers were running 
Linux.   If you're interested in pursuing this, drop me a line and we can jam 
on the topic for a bit.  I also know the guy who wrote the hairy C code and 
he might have some insight into how to reimplement it.

===

To: Dave Rolsky <autarch@urth.org>
From: "Daniel Sully" <daniel@electricrain.com>
Subject: Re: Using modperl as an 'adaptor' front end to an
app server.
Date: Fri, 2 Feb 2001 17:21:50 -0800

Once upon a time Dave Rolsky shaped the electrons to say...

> On Thu, 1 Feb 2001, Daniel Sully wrote:
> 
> > server. It handles failover from dead app instances, however not very
> > well, and is a big pile of C code. It also has problems in that because
> > Apache is not multithreaded, one child copy of that adaptor that has
> > marked an app instance as dead can't let another child know about that
> > dead instance.
> 
> You could use an IPC or DBM file (assuming you have 1 apache machine) to
> communicate this info.  With multiple webservers, you'd need a database or
> NFS or something.

That would be a no. Socket communication only. Shared filesystems in
production are bad, mmkay.

===

To: kyle dawkins <kyle@centralparksoftware.com>
From: "Daniel Sully" <daniel@electricrain.com>
Subject: Re: Using modperl as an 'adaptor' front end to an
app server.
Date: Fri, 2 Feb 2001 17:24:37 -0800

Once upon a time kyle dawkins shaped the electrons to say...

> A wild guess here but are you talking about WebObjects?

Yes - I've hacked up the WO 4.5 adaptor pretty bad, and it's a pile already.

> I wrote a perl adaptor last year to sniff the traffic that the WebObjects 
> adaptor was sending to the application instances.  I don't have the source 
> any more but as far as I can remember, it was only a few screensful of code.  
> However, it wasn't a full implementation of the adaptor and didn't work with 
> KeepAlive requests and so forth.

Hmm.. given the source to the adaptor, and being able to turn debugging
on, it looks pretty simple.

> I have often dreamt of writing a drop-in mod_perl replacement for the WO 
> adaptor... I have never worked anywhere where they didn't have trouble 
> configuring the Apache adaptor, particularly if their webservers were running 
> Linux.   If you're interested in pursuing this, drop me a line and we can jam 
> on the topic for a bit.  I also know the guy who wrote the hairy C code and 
> he might have some insight into how to reimplement it.

Will do. Would that happen to be Steve Quirk?

===
To: Daniel Sully <daniel@electricrain.com>
From: Dave Rolsky <autarch@urth.org>
Subject: Re: Using modperl as an 'adaptor' front end to an
app server.
Date: Fri, 2 Feb 2001 23:24:35 -0600 (CST)

On Fri, 2 Feb 2001, Daniel Sully wrote:

> > You could use an IPC or DBM file (assuming you have 1 apache machine) to
> > communicate this info.  With multiple webservers, you'd need a database or
> > NFS or something.
>
> That would be a no. Socket communication only. Shared filesystems in
> production are bad, mmkay.

I'm not sure which you're referring to, my suggestion that you use a DBM
file (with locking, of course) on 1 machine or NFS.  I have no experience
with NFS myself but I've heard of other using it for such things (or
trying to, at least).  I would think a database would end up being the
simplest way to do this sort of information passing.


===

To: Dave Rolsky <autarch@urth.org>
From: "Daniel Sully" <daniel@electricrain.com>
Subject: Re: Using modperl as an 'adaptor' front end to an
app server.
Date: Fri, 2 Feb 2001 23:43:38 -0800

Once upon a time Dave Rolsky shaped the electrons to say...

> > That would be a no. Socket communication only. Shared filesystems in
> > production are bad, mmkay.
> 
> I'm not sure which you're referring to, my suggestion that you use a DBM
> file (with locking, of course) on 1 machine or NFS.  I have no experience
> with NFS myself but I've heard of other using it for such things (or
> trying to, at least).  I would think a database would end up being the
> simplest way to do this sort of information passing.

There is too much overhead for using a locking a database across a
shared filesystem with potential latency problems and blocking/deadlock
potential. A direct socket connection, or even tossing a multicast
packet out onto the wire is the best for inter-machine communication,
where there may be N machines that are available to answer that request.

===

To: "Daniel Sully" <daniel@electricrain.com>
From: "Kyle Dawkins" <kyle@centralparksoftware.com>
Subject: Re: Using modperl as an 'adaptor' front end to an
app server.
Date: Sun, 4 Feb 2001 22:27:24 -0500

Yo ho ho

> > However, it wasn't a full implementation of the adaptor and didn't work
with
> > KeepAlive requests and so forth.
>
> Hmm.. given the source to the adaptor, and being able to turn debugging
> on, it looks pretty simple.

Absolutely... we'd need to sift thru the adaptor source and piece together
something that emulates it.  My version was purely to sniff traffic to see
what was going between Apache and the WO apps.

> > I have often dreamt of writing a drop-in mod_perl replacement for the WO
> > adaptor... I have never worked anywhere where they didn't have trouble
> > configuring the Apache adaptor, particularly if their webservers were
running
> > Linux.   If you're interested in pursuing this, drop me a line and we
can jam
> > on the topic for a bit.  I also know the guy who wrote the hairy C code
and
> > he might have some insight into how to reimplement it.
>
> Will do. Would that happen to be Steve Quirk?

You got it.

I am quite serious about getting it to work.  WO 4.5 is a different beast
from 4.0, though, with the introduction of wotaskd.  Still, it's all
XML-based so maybe we can leverage a lot of what's out there to get it up
and running.  Man, it'd be great to have a mod_perl adaptor.  Minimise the
number of postings to the WO mailing list about getting the stupid adaptor
to work, at any rate. :-)

====


the rest of The Pile (a partial mailing list archive)

doom@kzsu.stanford.edu