Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked
 
PerlMonks  

request timing out on Perl Web

by jpcardoso (Initiate)
on May 30, 2014 at 15:04 UTC ( #1088003=perlquestion: print w/replies, xml ) Need Help??

jpcardoso has asked for the wisdom of the Perl Monks concerning the following question:

Hi guys,

I'm not sure if this is a Perl question, but I'm developing a website in Perl and it contains a line that calls for a grep in a bzcat of a .bz2 file.

It's something like this:
my $var = `bzcat $path/file.bz2 | grep -i word`; print "$var";

The issue is that when I load the perl in the web browser, the page stops loading before the bzcat ends. The bzcat is taking around 5 minutes to complete but the browser "cuts" the connection and the page does not load the content.

Running the perl in the terminal it prints the content without problems.

My question is: Is there a way to increase the timeout of the bzcat process running on web?

Thanks in advance.

Replies are listed 'Best First'.
Re: request timing out on Perl Web
by flexvault (Monsignor) on May 30, 2014 at 15:18 UTC

    Welcome jpcardoso,

    What is the error message in the web server log?

    Usually the default is around 5 minutes, but the parameter is in seconds, so you could just be missing it.

    Regards...Ed

    "Well done is better than well said." - Benjamin Franklin

      Hi Ed,

      The error message in the webserver log is "Timeout waiting for output from CGI script".

      You opened my mind and I set the TimeOut value to 360 in the apache configuration, I was missing that. :)

      It seems to be working fine now, thank you very much for the quick help!

        jpcardoso,

        I'm glad I could help you, but I also have to agree with the other monks about waiting 5 minutes for a response. That's a really long time on the web today. Most users expect responses in seconds, not minutes. Doing this lookup with 'ajax' would be nice, but it would also be a lot of work.

        Maybe, let us help you with the real problem -- having to 'grep' on a 'bz2' file. Perl is very good at reading a file and searching ( 'index' ). Several generic fixes come to mind, but some more information about the data(bz2 file) would help us give you some better options.

        But just in case you can't get around the 'grep', you could schedule it and immediately return a HTML page informing the user that it will take 'x' minutes to get a response and to check back in a few minutes. Provide a link to check on the status. Once the response is available, return the current page that you are returning now.

        You may also want to look at the problem of denial of service (DOS) with longer time-outs. The apache web site describes this in detail. Better to plan now, than get the web site finished and then have a DOS attack.

        Best of Luck...Ed

        "Well done is better than well said." - Benjamin Franklin

Re: request timing out on Perl Web
by choroba (Archbishop) on May 30, 2014 at 15:45 UTC
    Similar questions have been already posted and answered. Have you tried to search the site?

    Search for: CGI long running

    لսႽ ᥲᥒ⚪⟊Ⴙᘓᖇ Ꮅᘓᖇ⎱ Ⴙᥲ𝇋ƙᘓᖇ
      unfortunately newbies always need to be informed to click on search after following the link.

      (Would be nice if Link to preload this search: would automatically produce a red search button on top with bold text telling people to click )

      Cheers Rolf

      ( addicted to the Perl Programming Language)

      Actually I searched, but probably used the wrong words. Anyway, thanks for the one who helped me.
Re: request timing out on Perl Web
by taint (Chaplain) on May 30, 2014 at 17:14 UTC
    My first thought reading this was to answer suggesting some while {...}, emitting some form of progress information -- a scale, or a repeating "Please wait..." output, ...

    This would have served 2 purposes; 1) overcome "timeout thresholds"; both the web server, as well as the web client (browser). 2) provide feedback to the user; the threshold of users attention span has greatly diminished, since the dawn of the internet.

    All the best.

    --Chris

    λɐp ʇɑəɹ⅁ ɐ əʌɐɥ puɐ ʻꜱdləɥ ꜱᴉɥʇ ədoH

Re: request timing out on Perl Web
by Anonymous Monk on May 31, 2014 at 00:22 UTC
Re: request timing out on Perl Web
by sundialsvc4 (Abbot) on May 31, 2014 at 01:47 UTC

    I usually make the suggestion that “a web-page should be thought of as a user-interface, nothing more.”   If there is “more” to be done, then the web-page interface ought to be used to initiate it, to monitor it, and to retrieve the results from it (whenever the user cares to stop by again and (s)he sees that the work is now complete), but the web page should never be the one to actually do it.   Some other background-processing engine, and there are a great many to choose from, should be doing that part of it.   Ideally, everything that a single web-page HTTP transaction should be asked to do, should take only a few milliseconds between request and response.   Even if the user checks back hours later or the next day, the results should be there waiting to be picked up.

Re: request timing out on Perl Web
by Laurent_R (Canon) on May 30, 2014 at 17:10 UTC
    I might have misunderstood, but do you seriously expect your user to wait more that 5 minutes for your page to be displayed?

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://1088003]
Approved by mr_mischief
Front-paged by mr_mischief
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others pondering the Monastery: (4)
As of 2021-10-21 11:10 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    My first memorable Perl project was:







    Results (83 votes). Check out past polls.

    Notices?