Beefy Boxes and Bandwidth Generously Provided by pair Networks
go ahead... be a heretic
 
PerlMonks  

mod_perl bug when reading post content with $|?

by Flexx (Pilgrim)
on Aug 30, 2002 at 13:43 UTC ( [id://194104]=perlquestion: print w/replies, xml ) Need Help??

Flexx has asked for the wisdom of the Perl Monks concerning the following question:

print "got a strange problem" if mod_perl::VERSION == 1.21 and ($| or client_is_printing("\015\012 +"))

Brothers and Sisters, fellow Monks!

After four days of meditating about this, I step before you, humbly asking for some insight. I really don't know if my problem is reproducable on other setups, maybe I'm just an unlucky guy. Note also that I (unfortunately) can't change my setup, since this is beyond my control.

PerlMonks is great. If you want to ask a qualified question, you have to sit down and write a small script to reproduce your problem, and that quite often leads to a solution "all by itself"... That's what happened to me, with my current problem. I didn't see that what I am about to describe was related to $| before I stripped down my server script to present only the relevant parts here...

Usually I'd bang my head onto my keyboard now, fix the real thing, and exhale... But this time I'm courious, since I found a workaround but no explanation for my problem, so I modified my post and put it on here... I was wondering whether the SOPW was the right place, or if this might be enough for some meditation -- to be on the secure side, I chose the former. Any yes, I know this is not the place to tell the story of a lifetime, but that's me... ;)

But first, let me describe my configuration:

Server: Apache 1.3.12, mod_perl 1.21, Apache::Registry 2.01 on IBM AIX 4.3
Clients: various (RedHat 7.1, SuSE 8.0, Win32)

In my httpd.conf you'll find:

<IfModule mod_perl.c> PerlRequire /apache/advinst_test/conf/startup.pl PerlModule Apache::StatINC Alias /cgi-registry /apache/advinst_test/cgi-bin <Location /cgi-registry> SetHandler perl-script PerlHandler Apache::Registry Options ExecCGI PerlSendHeader On PerlInitHandler Apache::StatINC PerlSetVar StatINCDebug On order deny,allow allow from all </Location> </IfModule>

Now put the following script on the server (I named it echo_post_data.pl):

#!/usr/bin/perl -w use strict; run_server(); sub debug_post_data() { warn "DEBUG: the content length is $ENV{CONTENT_LENGTH}\n"; my $returncode = read(STDIN, my $post_content, $ENV{CONTENT_LENGTH}) +; warn "DEBUG: the read returncode was $returncode\n"; warn "DEBUG: Finished.\n"; } sub run_server() { $| = 1; print "Content-Encoding: iso-8859-1\015\012", "Content-Type: text/plain\015\012", "\015\012"; debug_post_data(); }

Then put the following script on your client (I named it client_info_postdata.pl)

#!/usr/bin/perl -w use strict; use LWP::UserAgent; my $server = shift; my $content_length = shift || 1024; my $pattern = shift || '#'; my $useragent = LWP::UserAgent->new; my $http_request = HTTP::Request->new(POST => $server); $http_request->content_type('text/plain'); $http_request->content($pattern x ($content_length / length($pattern +)) ); print "\nRequest Content Length is ", length $http_request->conte +nt,".\n"; my $http_response = $useragent->request($http_request); print "\n<response>\n", $http_response->as_string, "</response>\n";

Now do perl client_info_postdata.pl http://pacman.host.magwien.gv.at:8080/cgi-registry/echo_post_data.pl 11680 +

Everything will work fine:

Request Content Length is 11680. <response> HTTP/1.1 200 OK Connection: close Date: Fri, 30 Aug 2002 12:05:38 GMT Server: Apache/1.3.12 (Unix) mod_perl/1.21 NTLM/2.1 Content-Encoding: iso-8859-1 Content-Type: text/plain Client-Date: Fri, 30 Aug 2002 12:07:11 GMT Client-Response-Num: 1 Client-Transfer-Encoding: chunked </response>

Then say perl client_info_postdata.pl http://pacman.host.magwien.gv.at:8080/cgi-registry/echo_post_data.pl 11681 +

If it works fine, do it a few times. You should see:

Request Content Length is 11681.

On my setup this will hang until it times out:

<response> 500 (Internal Server Error) read timeout Client-Date: Fri, 30 Aug 2002 12:10:36 GMT </response>

Apache's errlog log will show (after the timeout):

DEBUG: the content length is 11681 DEBUG: the read returncode was 11680 DEBUG: Finished. at /foo/bar/cgi-bin/echo_post_data.pl line 14.

Note the one-byte difference? Seemingly something chomped (?) the last byte of my post data!! Who was it? Come out, Bastard!

Now for the strange part of this: If I run the same script against /cgi-bin/ (mod_cgi instead of Apache::Registry), it works fine. When I comment out $| = 1 it works fine, too. When I don't print the final "\015\012" it works too, but then I get:

Request Content Length is 11681. <response> 500 (Internal Server Error) EOF instead of reponse status line Client-Date: Fri, 30 Aug 2002 12:20:00 GMT </response>

And, much to my surprise it appears to work once whenever Apache::StatINC reloads some of my libraries I load on that server. AFAIK I don't do anything I should better not do in mod_perl, or do I? Or, did I discover something here? Is this a known bug in mod_perl 1.21 or Apache::Registry 2.01 I haven't heard know about?

It seems that I can simply leave $| off to solve my problem for now, but I'd really want my hot pipes back! I already tried chunked reading, BTW, using

sub read_post_data_chunked(;$$) { my $content_length = shift || $ENV{CONTENT_LENGTH}; my $chunk_size = shift || $DEFAULT_CHUNK_SIZE; my $raw_data; while($content_length > 0) { warn read(STDIN, my $chunk, $chunk_size < $content_length ? $chu +nk_size : $content_length) . " chars read.\n"; $raw_data .= $chunk; $content_length -= $chunk_size; } return $raw_data; }

instead of reading in everything at once, but that didn't help. The script hung just the same (It does 2 loops than hangs in the third.

Any ideas?

Next thing I am going to do, is to look inside Apache::Registry, maybe I can find an explanation for this. I guess CORE::read is overloaded, maybe there's a glitch somewhere.

So long, Flexx

Additional Info

perl -v says:

Summary of my perl5 (5.0 patchlevel 5 subversion 3) configuration: Platform: osname=aix, osvers=4.3.1.0, archname=aix uname='aix hercules 3 4 00001427e800 ' hint=recommended, useposix=true, d_sigaction=define usethreads=undef useperlio=undef d_sfio=undef Compiler: cc='cc', optimize='-O', gccversion= cppflags='-D_ALL_SOURCE -D_ANSI_C_SOURCE -D_POSIX_SOURCE -qmaxmem= +8192 -I/usr/local/include' ccflags ='-D_ALL_SOURCE -D_ANSI_C_SOURCE -D_POSIX_SOURCE -qmaxmem= +8192 -I/usr/local/include' stdchar='unsigned char', d_stdstdio=define, usevfork=false intsize=4, longsize=4, ptrsize=4, doublesize=8 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=8 alignbytes=8, usemymalloc=n, prototype=define Linker and Libraries: ld='ld', ldflags =' -L/usr/local/lib' libpth=/usr/local/lib /lib /usr/lib /usr/ccs/lib libs=-lnsl -lgdbm -ldbm -ldl -lld -lm -lc -lcrypt -lbsd -lPW libc=, so=a, useshrplib=false, libperl=libperl.a Dynamic Linking: dlsrc=dl_aix.xs, dlext=so, d_dlsymun=undef, ccdlflags='-bE:perl.ex +p' cccdlflags=' ', lddlflags='-bhalt:4 -bM:SRE -bI:$(PERL_INC)/perl.e +xp -bE:$(BASEEXT).exp -b noentry -lc -L/usr/local/lib' Characteristics of this binary (from libperl): Built under aix Compiled at Feb 4 2000 10:18:39 @INC: /usr/local/lib/perl5/5.00503/aix /usr/local/lib/perl5/5.00503 /usr/local/lib/perl5/site_perl/5.005/aix /usr/local/lib/perl5/site_perl/5.005 .

Replies are listed 'Best First'.
Re: mod_perl bug when reading post content with $|?
by perrin (Chancellor) on Aug 30, 2002 at 15:48 UTC
    I don't know the problem with the missing byte, but I do know that $| doesn't make much difference under mod_perl and that you won't find the answer in Apache::Registry. I'd suggest posting to the mod_perl mailing list for help with this, because it's kind of obscure.
Re: mod_perl bug when reading post content with $|?
by RMGir (Prior) on Aug 30, 2002 at 15:28 UTC
    Note, I'm pretty clueless about mod_perl. But looking at the changelog for mod_perl 1.xxx, there are a few items that look like they might be relevant somehow, like:
    fix $r->read() so it will not block if all data has already been read and so that Apache will not hang during ap_discard_request_body() on error or redirect after all data has been read
    Is there any way you could try it under the current version (1.27)?

    In any case, browse that changelog, I'm sure those items will mean more to you than they mean to me.
    --
    Mike

Re: mod_perl bug when reading post content with $|?
by Flexx (Pilgrim) on Aug 30, 2002 at 16:46 UTC

    UPDATE

    Thanks for your anwers, monks!

    Now this becomes even stranger... Since you guys think it sounds like a mod_perl bug, too, I installed mod_perl 1.26 and apache 1.3.23 on my SuSE 8.0 i686 Box, so I think I'm fairly up to date now.

    Next I ran the client script against http://localhost/perl/tmp/echo_post_data.pl and guess what? It hangs with any content length I tried (1, 10, 10000, 11680, 11681, make your pick). Flip $| to zero, and data is flowing... %-|

    It also works with mod_cgi (usually via /cgi-bin/), but it does not work with Apache::PerlRun (/cgi-perl/ on my box)

    I'd really appreciate if somebody could crosscheck this on the same / other platforms.

    I wonder how this can be a mod_perl bug. Am I the only one to use $| = 1 with mod_perl? 1.21 was out in 1998, how could this have been undiscovered until now?

    You could break your CGI apps (or someone else's - HARHAR >;) with just one small change. Or, imagine a common CPAN module would start to set $| nonzero in a new verison... wow... 8|

    So long,
    Flexx

      Am I the only one to use $| = 1 with mod_perl?

      You might be. Is it ever necessary? You said you want to be as lean as possible, but autoflush ($|) makes scripts slower.

      Or, imagine a common CPAN module would start to set $| nonzero in a new verison...

      That's why one should always use local when setting special variables. Imagine what would happen if one changes $/ without localizing it first. Even not localizing $_ can be problematic. Under mod_perl you should always use local with special variables. (Don't local $ENV{key} as this is buggy: localize the entire hash)

      - Yes, I reinvent wheels.
      - Spam: Visit eurotraQ.
      

        Actually, regarding my problem, it doesn't make a difference to localize $|.

        My server now looks like this, but has the same hang

        #!/usr/bin/perl -w use strict; local $| = 1; print "Content-Encoding: iso-8859-1\015\012", "Content-Type: text/plain\015\012", "\015\012"; warn "DEBUG: the content length is $ENV{CONTENT_LENGTH}\n"; my $returncode = read(STDIN, my $post_content, $ENV{CONTENT_LENGTH +}); warn "DEBUG: the read returncode was $returncode\n"; warn "DEBUG: Finished.";

        Stas Bekman has summed up why one would use $| = 1 under mod_perl quite well in mod_perl: Performance Tuning.

Re: mod_perl bug when reading post content with $|?
by fglock (Vicar) on Aug 30, 2002 at 14:17 UTC

    can't change my setup

    Do you mean you can't use a CGI module? You could at least try using CGI for testing purposes.

    update:  binmode solves some problems related to newlines in windows. It might have some relation to your problem.

      I was almost expecting this... ;)

      This is not a browser application, I don't use forms. I see that there's no excuse not to use CGI on typical CGI applications, but in my project HTTP/CGI is merely a communications gateway (hehe, yes I know that HTTP/CGI is nothing more than that).

      Besides that, using CGI would put me over the edge perfomance-wize. I'm under rapid fire from clients, and I need to be as lean as possible.

      However, I don't want to start yet another use [no] CGI discussion, that wasn't my point in the first place. I simply wonder how setting $| nonzero could block those reads (!). To me that's something that simply must not happen (and under mod_cgi it does not happen)...

      With "can't change my setup" I meant I can't update my Apache/mod_perl/OS (at least unless someone tells me that this is a bug that has been fixed in some later version, so I can apply pressure on the server/perl packaging folks -- I can understand they don't want to invest in AIX 4.3 any longer when we're migrating to AIX 5 anyway).

      So long,
      Flexx

        With "can't change my setup" I meant I can't update my Apache/mod_perl/OS (at least unless someone tells me that this is a bug that has been fixed in some later version, so I can apply pressure on the server/perl packaging folks...)
        If this is an apache/mod_perl bug (which seems likely, given how old your apache is), it seems so esoteric that I'd despair of finding a reference to it in the changelog. Have you tried a newer version of apache yourself? You can build one in a nonstandard location without needing root access. Surely you must have a suitable development machine somewhere you can try this out on.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://194104]
Approved by rattusillegitimus
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others making s'mores by the fire in the courtyard of the Monastery: (7)
As of 2024-03-28 11:39 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found