| [reply] |
I had this same problem a few months ago with a script I built for Warner Bros. The script would load 70,000 surveys, calculate statistical means and medians for each question, then output a master report. The problem was, was that the calculations were SLOW. Sometimes the CGI script took a half hour to run.
The beta testers (heh, myself really) would hit reload after a minute or two, thinking that the script had crashed. Why no response from the server? *click* *click* The server stopped responding completely, it crashed from the concurrent requests (CPU: 14.30). That night I started looking for another job.
Two months of work down the drain.
After I calmed down a bit, I realized that it was okay that the request was slow to execute. That's a lot of surveys we're talking about, users just need a status screen updating them on the status of their request. That's all. No big deal. So I built a minor caching system that cached the reports, and then opened Dreamweaver and built a template. Here's a screenshot of the final status screen in action. (Note the progress bar at the bottom in HTML tables. I think it's cute.)
Now all I had to do was send the browser a redirect to this status page (which has a meta tag requesting itself to be refreshed every 5 seconds), then periodically update it as I continued processing surveys, until I finish, and then I'll redirect it to the master report. Easy as pie. Right?
WRONG!!!
It worked fine in Opera and Netscape but IE wouldn't execute the redirect until AFTER the script finished running!!!! Bastard program. I knew there was a good reason to never trust IE. Evil manipulating bastards, I say. And at this point I was behind schedule, I should have delivered the application days before.
My intuition is that Apache doesn't relinquish the TCP/IP connection (server side) until the perl process exits. That's logical right? IE received the redirect just fine, but it waited for that TCP/IP connection to drop before actually executing it. (There's even a Microsoft knowledge base article on the problem, but I switched computers and don't have it bookmarked any more.)
I'm not a perl guru or anything, so instead of wasting more time figuring out how to tell Apache to drop the connection, and probably learning something in the process, I hacked a crappy fix together that works:
my $pid;
#
# Fork the current process.
#
if (!defined ($pid = fork)) {
DieNice("Unable to fork: $!\n");
} elsif (! $pid) {
# This is the child fork, for processing.
close(STDIN); close(STDOUT); close(STDERR);
} else {
# This is the parent fork, for the browser.
print $query->redirect("http://$servername/wbnorm/cache/status.h
+tml");
# Apache, you will answer to me!
exit;
}
#
# Calculations continue as normal here
#
*ding*!
(I registered after a year of lurking just to participate in this very topic. As a new user, I'd like to say, merlyn -- you've saved my ass dozens of times, and I thank you for it.) | [reply]
[d/l] |
... and of course, I end up posting as anonymous monk. *doh*
| [reply] |
You have to add the following line at the beginning of your script:
$|=1;
It is setting printing into direct mode (turning buffer off), so perl will flush out everything in realtime.
--
tune | [reply]
[d/l] |
The general problem you're facing is that of returning a response (in the form of an HTML page) to the user while work continues on the server side.
There are a couple of strategies for this. merlyn points to one of them, which is to have the CGI fork, and continue work in the child process while the parent process exits. Works great on Unix systems, works very poorly on Win32 (IIS).
Another approach is to write a separate server process to handle work requests. The CGI queues up a request to this new worker process, either by dropping a file into a well-known directory, or by opening a socket to it and pushing the request along. The CGI pushs and HTML page back to the browser and exits, while the worker process grinds through requests.
This latter approach works well when you have control of the server. Some ISPs forbid running your own server processes.
| [reply] |
On Unix boxen you may be able to use a simple quick and dirty solution:
open AT, "|at now" or die "Failed piping to 'at': $!";
print AT "$some $commandline $to_execute\n";
print AT "$or $maybe $two\n";
close AT
Of course I didn't tell you that. :-)
Makeshifts last the longest. | [reply]
[d/l] |
Redirect the browser to your homepage? or a thank-you page?
print "Location: $return_page\n\n";
| [reply]
[d/l] |
