Someone sent me that excellent article in response to another question. The problem is not the browser timing out, but rather that the process waits until the end to display the results.

I've actually used the exact code that's in the article you reference:

my $command = "make -i -j 10 -f upgrade.make";
  
  unless (open F, "-|")
  {
    open STDERR, ">&=1";
    exec "$command";
  }
  
  while (<F>)
  {
    $buffer .= $_;
    $cache->set($session, [0, $buffer]);
  }
[download]

And it still waits until the process completes to give me back any output whereas if I run $command from the command line it gives me real-time feedback.

CT

Charles Thomas
Madison, WI

The problem is not the browser timing out, but rather that the process waits until the end to display the results.

You may have a different problem, but merlyn's article is still a possible solution. Have your lengthy process fork off and write its output to a file, then have the CGI refresh every X seconds, reading that output file, and displaying the results. Perhaps you could even parse the output file and make the results a bit more user friendly (this is just speculation -- the output may already be user friendly).

You may also find that your lengthy process buffers its output writing to the file, though. In that case, you can investigate some of the solutions sgifford recommended.

Have you tried setting $| = 1; at the top of your script? That sets the buffer to flush immediately, and it's a standard part of all of my CGI scripts...

Second, it could be in the $my_long_process program. Many programs, and the default stdio library, buffer output unless it's to a tty. If that's the case with $my_long_process program, it wouldn't buffer output when you ran it from the command-line (because it was connected to a tty), but would when run from your program. You might be able to test by running:

$my_long_program |cat
[download]

If that's the case, finding an option to cause the program to display output immediately is the easiest solution; the other option is to connect it to a pseudo-tty, with something like Expect.

make sure whenever your actually print the output that $| is set to a true value.

I have done this. Like you, this is something I used in most of my scripts.

Second, it could be in the $my_long_process program. Many programs, and the default stdio library, buffer output unless it's to a tty. If that's the case with $my_long_process program, it wouldn't buffer output when you ran it from the command-line (because it was connected to a tty), but would when run from your program. You might be able to test by running: $my_long_program |cat

It still gave me real-time feedback when run piped into cat as you suggested.

CT

Charles Thomas
Madison, WI

Changing the code as follows:

  unless (open F, "-|")
  {
    open STDERR, ">&=1";
    select(STDIN);
    $| = 1;
    select(STDERR);
    # also tried select(F) here
    $| = 1;

  
    exec "$command";
  }
  
  while (<F>)
  {
    $buffer .= $_;
    $cache->set($session, [0, $buffer]);
  }
[download]

Does not make any difference.

CT

Charles Thomas
Madison, WI

Try with second entry.... select RBITS,WBITS,EBITS,TIMEOUT

True, but I'm throwing a <br> tag after every line, so I don't believe that's the issue.

CT

Charles Thomas
Madison, WI

I think the issue is in the $cache->set call. What type of object is $cache?

It's a Cache::FileCache object as described in the paper by Merlyn referenced earlier in the thread.

Basically the cgi keeps calling itself every five seconds, and when it revisits itself it prints the contents of the cache object. So I store all output of the script in this cache object.

Once the long process completes, all its output goes into the cache object as it should. It's just that it takes until it completes to see what's going on.

CT

Charles Thomas
Madison, WI

OK. merlyn's article deals with how to save the information between invocations to a CGI (using a persistent cache and session key to get the data back). You want to do something different. You want real timeish results. That's different.

You do not need to put the data in the cache; in fact, it is what is storing or buffering the output. You need to set $| to 1 and send the data directly to the user. One big issue, though, is that browsers sometimes "timeout": as noted above if they don't see data in n seconds, they disconnect. I have solved this in the past using timeouts. For now, I'd just write the data directly to the browser with $|=1 and without the cache, and see what happens.

You could use a solution like merlyn's by extending the key to be $session-$line and then retreiving each line individually.