http://qs321.pair.com?node_id=55241

odie has asked for the wisdom of the Perl Monks concerning the following question:

I am using a bidirectional pipe to another process. Readline will block the program until a line is recieved. How do I read in a non blocking way?
Is there perhaps a smart (and nifty. must be nifty.) way to check if there is data waiting?


--
I am a manual signature virus. Copy me please!

Replies are listed 'Best First'.
Re: Non blocking read on a filehandle
by tadman (Prior) on Jan 30, 2001 at 19:34 UTC
    If you're willing to get down and dirty, you can use select() like the rest of them. Earlier, reyjrar was asking about doing the same sort of thing on a socket. The same principles apply since file-handles and socket handles are really quite the same. That post is here.

    I've extracted the relevant bits of that code and posted it here as an example on how to check up on a filehandle without blocking:
    my ($block_size) = 10_000_000; # 10MB block size my ($timeout) = 0; # Don't wait. my ($rfd) = ''; # Initialize with string only # SYNOPSIS: open ($filehandle, "FILE or PIPE"); CheckHandle ($filehandle); sub CheckHandle { my ($filehandle) = @_; vec ($rfd, fileno($filehandle), 1) = 1; # Wait for something to happen, and make sure # that it happened to the right filehandle. if (select ($rfd, undef, undef, $timeout) >= 0 && vec($rfd, fileno($filehandle), 1)) { # Something came in! my ($buffer); read ($filehandle, $buffer, $block_size); return $buffer; } return; }
    Explanation:
    POSIX calls such as select() use a slightly different view of the world when compared with Perl. Most of the time Perl will convert for you automatically, but there are a few cases where you have to help out.

    select() polls a number of file-handles (or sockets) and can report if they are ready to read, write, or if they have experienced an error. The documentation on select() is a little thin, and it will take a bit of experimentation to get a good handle on it. Note that select() can be configured to block or to be non-blocking, the trick is in the "$timeout" parameter. A zero timeout will put it into non-blocking mode.

    In order to tell select which filehandles you want to monitor, vec() is used to create a bit-mask with bit number zero representing filehandle 0, bit 1 for filehandle 1, etc. These filehandles are a bit different from Perl filehandles, so a quick conversion with fileno() will get things on track.

    This code uses "scalar"-style filehandles here because they are easier to pass between functions, and they work the same in most circumstances.
      You set $timeout to 0, this will cause select to block until it can read, it doesn't set select to nonblocking like you sugested. You can set $timeout to 1 and then select will wait up to 1 second to see if anything comes in, and then will continue on with the next line of code. But, this will slow down your program as it will wait up to 1 second each time your loop is run.
Re: Non blocking read on a filehandle
by Gloom (Monk) on Jan 30, 2001 at 19:31 UTC
    Try to use select with a small timeout value.
    Maybe you can use also the perl implementation of ioctl that allow you to do non-blocking i/o ( among many other things )

    Update :

    It's also possible to handle the "IO" signal by linking some function to $SIG{IO} sighandler, and play with the filehandle's flags with fcntl

    No more details : searching is the way of understanding...

    Hope this help :)

Re: Non blocking read on a filehandle
by boo_radley (Parson) on Jan 30, 2001 at 19:32 UTC
    non-blocking reads from the cookbook.
    use Fcntl; sysopen(MODEM, "/dev/cua0", O_NONBLOCK|O_RDWR) or die "Can't open modem: $!\n";
    be warned : once the file handle's non blocking, statements that try to switch back will return undef and set $!.

    --
    Hope this helps -- my OS doesn't have any of this fancy blocking or nonblocking hoo-hah, so I can't test properly ;-)

      But this is only for files, right? Not processes... =)

      --
      I am a manual signature virus. Copy me please!
Re: Non blocking read on a filehandle
by c-era (Curate) on Jan 30, 2001 at 19:35 UTC
    If you are on Win32, you are just out of luck. But if you are on unix/linux you can set the filehandle to non-blocking. This will let you read from the handle if something is there. If nothing is there it will continue on to the next line of code.
    use Fcntl; my $flags; fcntl(FILEHANDLE, F_GETFL, $flags) || die $!; # Get the current flags +on the filehandle $flags |= O_NONBLOCK; # Add non-blocking to the flags fcntl(FILEHANDLE, F_SETFL, $flags) || die $!; # Set the flags on the f +ilehandle
      This seems to work. (so far =)
      It is also quite nifty. I like it.

      --
      I am a manual signature virus. Copy me please!
      my $flags = 0; works for me
Re: Non blocking read on a filehandle
by InfiniteSilence (Curate) on Jan 30, 2001 at 19:31 UTC
    You need to create separate threads, one of input and the other for output. I learned this from Lincoln Stein's new Network Programming with Perl book. There are great examples there of this for different platforms (Win32/Unix).

    Celebrate Intellectual Diversity

      Perl does not have a reliable threading model. You cannot rely on this for production level code.
Re: (reyjrar) Non blocking read on a filehandle
by reyjrar (Hermit) on Jan 31, 2001 at 00:24 UTC
    Here's something I've started throwing together.. might as well post it here to get the feedback from it and get it all corrected for my use and for your use.(question marks mean I'm unsure on these)
    UPDATED!!
    Function Read/Write? Blocking? Buffering?
    send WRITE Yes Yes
    recv READ Yes Yes
    print WRITE Yes Yes
    read READ Yes Yes
    syswrite WRITE Yes No
    sysread READ Yes No
    <FILEHANDLE> READ Yes Yes


    I think that blocking also depends on the flags set on the file handle.. like I said, this is probly all wrong, but it would be nice for someone to correct me and it could prove useful for perl programmers met with the frustration I've experienced trying to understand socket/file I/O ;)

    -brad..

      Your color select sucks when viewed with my color scheme. Please don't use color.

      The "Blocking?" field should be "Yes" for all seven items (even the ones you have marked as "No"). Under "Buffering?", the two "No (?)"s should be "Yes"s.

              - tye (but my friends call me "Tye")
      Pretty much everything you have there is buffered in some way or another. Don't forget that you can turn this off using $. It's also worth mentioning that's its a bad idea to use send or recv on a filehandle. :-D For syswrite and sysread, buffering is also optional.
      AgentM Systems nor Nasca Enterprises nor Bone::Easy nor Macperl is responsible for the comments made by AgentM. Remember, you can build any logical system with NOR.

        sysread and syswrite are not buffered, other than that the operating system might buffer things such that what you syswrite() might not be flushed to disk immediately. But it will be visible to other processes immediately.

                - tye (but my friends call me "Tye")
Re: Non blocking read on a filehandle
by daveola (Sexton) on Sep 24, 2008 at 09:53 UTC
    I've found this following routine useful. Basically it allows me to just get nonblocking full lines of input from a filehandle (using the select/vec solution - and heavily based on tadman's code above).
    # An non-blocking filehandle read that returns an array of lines read # Returns: ($eof,@lines) my %nonblockGetLines_last; sub nonblockGetLines { my ($fh,$timeout) = @_; $timeout = 0 unless defined $timeout; my $rfd = ''; $nonblockGetLines_last{$fh} = '' unless defined $nonblockGetLines_last{$fh}; vec($rfd,fileno($fh),1) = 1; return unless select($rfd, undef, undef, $timeout)>=0; # I'm not sure the following is necessary? return unless vec($rfd,fileno($fh),1); my $buf = ''; my $n = sysread($fh,$buf,1024*1024); # If we're done, make sure to send the last unfinished line return (1,$nonblockGetLines_last{$fh}) unless $n; # Prepend the last unfinished line $buf = $nonblockGetLines_last{$fh}.$buf; # And save any newly unfinished lines $nonblockGetLines_last{$fh} = (substr($buf,-1) !~ /[\r\n]/ && $buf =~ s/([^\r\n]*)$//) ? $1 : ''; $buf ? (0,split(/\n/,$buf)) : (0); }
    I've tested it with a pipe I opened using an IO::File, YMMV. Example usage:
    $fh = new IO::File; open($fh,"$cmd 2>&1 |"); do { ($eof,@lines) = nonblockGetLines($fh); foreach my $line ( @lines ) { print "Pipe saw: [$line]\n"; } } until $eof; close $fh;
    I don't know enough about vec() to know why tadman called it twice, don't know if this is a race condition or if it's unneeded, and at this point don't care enough to figure it out - I've got other code to write! :) Enjoy!
      > # I'm not sure the following is necessary?
      > return unless vec($rfd,fileno($fh),1);

      select() allows you to check for many file descriptors at once by adding each fd into the $rfd vector. The $rfd gets modified by select() to let you know which file descriptors are available.

      The second line of code here verifies that the file descriptor you asked about is one that's ready.