modperl-conniptions_to_get_ProxyIOBufferSize_feature_with_current_cvs_apache_1.3.24

This is part of The Pile, a partial archive of some open source mailing lists and newsgroups.



Date: Sun, 24 Mar 2002 01:01:24 +0000
From: Pedro Melo Cunha <melo@isp.novis.pt>
To: modperl@perl.apache.org
Subject: Be carefull with apache 1.3.24

Hi,

a bit of history: I read on the documentation the new mod_proxy feature
called ProxyIOBufferSize which allows for even better handling of lasrge
mod_perl httpd's. This baby, and another little patch that also made it
(closing the backend connection as soon as possible) allows for me to
have a split of 100 modproxy process for only 10 backend mod_perl
process... neat! :) (BTW, using this and mod_gzip, and you can save a
bundle instead of buying those nice redline boxes :)

the problem is that it's only available on 1.3.24 (at the time, 2 or 3
days ago, it wasnt available yet), so I used the CVS version.

Looking at the change log, they mention a bug that multiple set-cookie's
will fail (only the last one will be sent to the client, the proxy will
"eat" the others). And it was true... The problem is that 1.3.24 final
also has that bug: only the last set-cookie will reach your browser.

This was a show-stopper for my application. So I sat down, read the
proxy code, and made a patch to solve the bug. I then sent the patch to
the apache bug database (i really hate gnats ;), but they drop it (as a
duplicate). I admit i could not attach my patch to the existing bug
report on this subject (I said I hated gnat's :). So my patch did not
made it the 1.3.24...

Anyway, I attached here. Maybe someone here can push it into apache for
the next release, and in the meantime you can use it, if you want.

I also attach a cgi that set's two cookies, to demonstrate the problem.

Best regards,

PS: mod_perl totally rocks... serving well over 70.000.000 pageviews
    with mason...

===

Date: Sun, 24 Mar 2002 12:29:31 +0800
From: Stas Bekman <stas@stason.org>
To: Pedro Melo Cunha <melo@isp.novis.pt>
Cc: modperl@perl.apache.org
Subject: Re: Be carefull with apache 1.3.24

Pedro Melo Cunha wrote:

> a bit of history: I read on the documentation the new mod_proxy feature
> called ProxyIOBufferSize which allows for even better handling of lasrge
> mod_perl httpd's. This babby, and another little patch that also made it
> (closing the backend connection as soon as possible) allows for me to
> have a split of 100 modproxy process for only 10 backend mod_perl
> process... neat! :) (BTW, using this and mod_gzip, and you can save a
> bundle instead of buying those nice redline boxes :)
> 
> the problem is that it's only available on 1.3.24 (at the time, 2 or 3
> days ago, it wasnt available yet), so I used the CVS version.
> 
> Looking at the change log, they mention a bug that multiple set-cookie's
> will fail (only the last one will be sent to the client, the proxy will
> "eat" the others). And it was true... The problem is that 1.3.24 final
> also has that bug: only the last set-cookie will reach your browser.
> 
> This was a show-stopper for my application. So I sat down, read the
> proxy code, and made a patch to solve the bug. I then sent the patch to
> the apache bug database (i really hate gnats ;), but they drop it (as a
> duplicate). I admit i could not attach my patch to the existing bug
> report on this subject (I said I hated gnat's :). So my patch did not
> made it the 1.3.24...

Pedro, please send it to dev @ httpd.apache.org.

> Anyway, I attached here. Maybe someone here can push it into apache for
> the next release, and in the meantime you can use it, if you want.
> 
> I also attach a cgi that set's two cookies, to demonstrate the problem.
> 
> Best regards,
> 
> PS: mod_perl totally rocks... serving well over 70.000.000 pageviews
>     with mason...
> 
> 
> ------------------------------------------------------------------------
> 
> Index: proxy_http.c
> ===================================================================
> RCS file: /home/cvspublic/apache-1.3/src/modules/proxy/proxy_http.c,v
> retrieving revision 1.88
> diff -u -r1.88 proxy_http.c
> --- proxy_http.c	21 Mar 2002 11:38:03 -0000	1.88
> +++ proxy_http.c	22 Mar 2002 01:21:39 -0000
> @@ -136,6 +136,20 @@
>      return url;
>  }
>  
> +
> +/* copies header key/value into table p if it's a cookie header.
> + * this is a callback function for a ap_table_do below.
> + */
> +int ap_proxy_copy_cookie_headers(void *p, const char *key, const char *value)
> +{
> +    table *d = (table *)p;
> +
> +    if (!strcasecmp(key, "Set-Cookie") || !strcasecmp(key, "Set-Cookie2"))
> +      ap_table_add(d, key, value);
> +
> +    return 1;
> +}
> +
>  /*
>   * This handles http:// URLs, and other URLs using a remote proxy over http
>   * If proxyhost is NULL, then contact the server directly, otherwise
> @@ -529,8 +543,20 @@
>          ap_proxy_write_headers(c, ap_pstrcat(p, "HTTP/1.1 ", r->status_line, NULL), resp_hdrs);
>      }
>  
> -    /* Setup the headers for our client from upstreams response-headers */
> +    /* Setup the headers for our client from upstreams response-headers.
> +     * We use resp_hdrs to keep all the Set-Cookie and Set-Cookie2 headers,
> +     * and then call ap_overlap_table, because it keep's only the last one.
> +     * The we copy all the cookies back.
> +		 */
> +    ap_table_do(ap_proxy_copy_cookie_headers, resp_hdrs, r->headers_out, NULL);
> +
>      ap_overlap_tables(r->headers_out, resp_hdrs, AP_OVERLAP_TABLES_SET);
> +
> +    ap_table_unset(r->headers_out, "Set-Cookie");
> +    ap_table_unset(r->headers_out, "Set-Cookie2");
> +
> +    ap_table_do(ap_proxy_copy_cookie_headers, r->headers_out, resp_hdrs, NULL);
> +
>      /* Add X-Cache header - be careful not to obliterate any upstream headers */
>      ap_table_mergen(r->headers_out, "X-Cache",
>                    ap_pstrcat(r->pool, "MISS from ",
> 
> 
> ------------------------------------------------------------------------
> 
> #!/usr/local/bin/perl -w
> 
> use CGI qw( :standard );
> use CGI::Cookie;
> 
> my $c1 = new CGI::Cookie(-name=>'ID',-value=>123456);
> my $c2 = new CGI::Cookie(-name=>'preferences',
>                          -value=>{ font => Helvetica, size => 12 });
> 
> print header(-cookie=>[$c1,$c2]);
> 
> print "Done";

===

Date: Sun, 24 Mar 2002 09:44:01 +0000 (GMT)
From: Ged Haywood <ged@www2.jubileegroup.co.uk>
To: Pedro Melo Cunha <melo@isp.novis.pt>
Subject: Re: Be carefull with apache 1.3.24

Hi Pedro,

On Sun, 24 Mar 2002, Pedro Melo Cunha wrote:

[snip,snip]
> Looking at the change log, they mention a bug that multiple set-cookie's
> will fail (only the last one will be sent to the client, the proxy will
> "eat" the others). And it was true... The problem is that 1.3.24 final
> also has that bug: only the last set-cookie will reach your browser.
> 
> This was a show-stopper for my application. So I sat down, read the
> proxy code, and made a patch to solve the bug. [snip]
> 
> Anyway, I attached here. Maybe someone here can push it into apache for
> the next release, and in the meantime you can use it, if you want.
> 
> I also attach a cgi that set's two cookies, to demonstrate the problem.


Pedro, thanks a real bunch!  You just saved me a load of work!

If we ever meet, I'll buy you a beer, I promise!



===

Date: Mon, 25 Mar 2002 11:14:01 -0500
Subject: Re: Be carefull with apache 1.3.24
From: John Siracusa <siracusa@mindspring.com>
To: Mod Perl Mailing List <modperl@perl.apache.org>

Does anyone know how I can put my ProxyIOBufferSize config line in a
conditional that'll keep it from blowing up if I'm using a version of apache
earlier than 1.3.24?

===

Date: Mon, 25 Mar 2002 17:31:17 +0100 (CET)
From: Hans Juergen von Lengerke <lengerkeh@sixt.de>
Subject: Re: Be carefull with apache 1.3.24
To: John Siracusa <siracusa@mindspring.com>

John Siracusa <siracusa@mindspring.com> on Mar 25, 2002:

> Does anyone know how I can put my ProxyIOBufferSize config line in a
> conditional that'll keep it from blowing up if I'm using a version
> of apache earlier than 1.3.24?

You could use <IfDefine> but that would mean that the command that
starts apache needs to be aware of which version you are using and
modifying startup params accordingly.

See http://httpd.apache.org/docs/mod/core.html#ifdefine



===

Date: Mon, 25 Mar 2002 11:35:26 -0500
To: siracusa@mindspring.com, Mod Perl Mailing List <modperl@perl.apache.org>
From: Robert Landrum <rlandrum@capitoladvantage.com>
Subject: Re: Be carefull with apache 1.3.24

At 11:14 AM -0500 3/25/02, John Siracusa wrote:
>Does anyone know how I can put my ProxyIOBufferSize config line in a
>conditional that'll keep it from blowing up if I'm using a version of apache
>earlier than 1.3.24?
>
>-John

In /path/to/apache1.3.24/bin/apachectl add a -DAPACHE_1_3_24 to the 
httpd command.

In your httpd.conf add

<IfDefined APACHE_1_3_24>
ProxyIOBufferSize 1000000000
</IfDefined>

I think that should work....

===

Date: Mon, 25 Mar 2002 11:39:22 -0500
From: Geoffrey Young <geoff@modperlcookbook.org>
Subject: Re: Be carefull with apache 1.3.24


> In /path/to/apache1.3.24/bin/apachectl add a -DAPACHE_1_3_24 to the
> httpd command.
> 
> In your httpd.conf add
> 
> <IfDefined APACHE_1_3_24>
> ProxyIOBufferSize 1000000000
> </IfDefined>
> 
> I think that should work....

you could also do something hackish like this to avoid command line switches

<Perl>
  my ($version) = Apache::Constants::SERVER_VERSION =~ m/(\d{2})/;
  $ProxyIOBufferSize = 10000 if $version >= 24
</Perl>

===

Date: Mon, 25 Mar 2002 12:47:13 -0500
Subject: Re: Be carefull with apache 1.3.24
From: John Siracusa <siracusa@mindspring.com>
To: Mod Perl Mailing List <modperl@perl.apache.org>

On 3/25/02 11:39 AM, Geoffrey Young wrote:
>> In /path/to/apache1.3.24/bin/apachectl add a -DAPACHE_1_3_24 to the
>> httpd command.
>> 
>> In your httpd.conf add
>> 
>> <IfDefined APACHE_1_3_24>
>> ProxyIOBufferSize 1000000000
>> </IfDefined>
>> 
>> I think that should work....
> 
> you could also do something hackish like this to avoid command line switches
> 
> <Perl>
> my ($version) = Apache::Constants::SERVER_VERSION =~ m/(\d{2})/;
> $ProxyIOBufferSize = 10000 if $version >= 24
> </Perl>

I don't want to use defines if I can help it, and it's a proxy server
without mod_perl, so the <Perl> section is out.  Hm, you'd think apache
would have a directive for this, especially since it pukes when it sees
unknown conf directives.  Maybe I'll go nag the apache devs... :)

===

Date: Fri, 29 Mar 2002 13:11:57 -0500
Subject: Re: Be carefull with apache 1.3.24
From: John Siracusa <siracusa@mindspring.com>
To: Pedro Melo Cunha <melo@isp.novis.pt>,

On 3/23/02 8:01 PM, Pedro Melo Cunha wrote:
> Looking at the change log, they mention a bug that multiple set-cookie's
> will fail (only the last one will be sent to the client, the proxy will
> "eat" the others). And it was true... The problem is that 1.3.24 final
> also has that bug: only the last set-cookie will reach your browser.
> 
> This was a show-stopper for my application. So I sat down, read the
> proxy code, and made a patch to solve the bug.

Applied the attached patch to 1.3.24, but it complained:

 ---

% patch -i ../mod_proxy_patch.diff src/modules/proxy/proxy_http.c
patching file src/modules/proxy/proxy_http.c
Reversed (or previously applied) patch detected!  Assume -R? [n] y

 ---

I'm not sure what that means, but I forced it through anyway, and the bug
remains!  (As tested by your attached cookie.cgi script.)

Just to make sure I wasn't crazy, I also re-tried my unpatched version of
1.3.24 and it showed the bug too.

So, what's the status on this?  Is 1.3.25 due out soon to fix this?  Does
anyone have a working patch?

===

Date: Fri, 29 Mar 2002 13:27:21 -0500 (EST)
From: Larry Leszczynski <larryl@furph.com>
To: mod_perl list <modperl@apache.org>
Subject: Re: Be carefull with apache 1.3.24


On 3/23/02 8:01 PM, Pedro Melo Cunha wrote:

> The problem is that 1.3.24 final also has that bug: only the last
> set-cookie will reach your browser.

Another problem that bit me is that 1.3.24 mod_proxy is returning HTTP/1.1
chunked encoding responses to HTTP/1.0 clients:
   http://nagoya.apache.org/bugzilla/show_bug.cgi?id=7513

(For those who might not know it if they saw it, chunked encoding adds a
length (specified in hex) at the very beginning of a response, and a zero
at the end.)



===

Date: Fri, 29 Mar 2002 13:26:43 -0500
Subject: Re: Be carefull with apache 1.3.24
From: John Siracusa <siracusa@mindspring.com>
To: Pedro Melo Cunha <melo@isp.novis.pt>,

On 3/29/02 1:11 PM, John Siracusa wrote:
> On 3/23/02 8:01 PM, Pedro Melo Cunha wrote:
>> Looking at the change log, they mention a bug that multiple set-cookie's
>> will fail (only the last one will be sent to the client, the proxy will
>> "eat" the others). And it was true... The problem is that 1.3.24 final
>> also has that bug: only the last set-cookie will reach your browser.
>> 
>> This was a show-stopper for my application. So I sat down, read the
>> proxy code, and made a patch to solve the bug.
> 
> Applied the attached patch to 1.3.24, but it complained:
> 
> ---
> 
> % patch -i ../mod_proxy_patch.diff src/modules/proxy/proxy_http.c
> patching file src/modules/proxy/proxy_http.c
> Reversed (or previously applied) patch detected!  Assume -R? [n] y
> 
> ---
> 
> I'm not sure what that means

Heh, n e v e r m i n d... :)

Anyway, now that I have patched, working versions, what's the status on
apache.org fixing this bug in the real distribution?

===

Date: Fri, 29 Mar 2002 13:39:23 -0500
Subject: Re: Be carefull with apache 1.3.24
From: John Siracusa <siracusa@mindspring.com>
To: Mod Perl Mailing List <modperl@perl.apache.org>

On 3/29/02 1:26 PM, John Siracusa wrote:
> Anyway, now that I have patched, working versions...

Scratch that: the patch doesn't seem to work when I build a server with
mod_ssl in it.  Hrm...

===

the rest of The Pile (a partial mailing list archive)

doom@kzsu.stanford.edu