Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling
 
PerlMonks  

Re: IO::Handle->getline() partial reads from pipe

by wazat (Scribe)
on Dec 19, 2013 at 06:14 UTC ( #1067782=note: print w/replies, xml ) Need Help??


in reply to IO::Handle->getline() partial reads from pipe

I used to know something about pipes, but that was a long time ago. At the kernel level a pipe as a maximum size (once upon a time I think 16K may have been typical). If you writes are larger than that then at the system level the child may get at most that max amount. I would think that getline() should buffer the read and read again until it reads data with an newline. It sounds like this isn't happening in your case. I'd expect that kind of behaviour from sysread(), but not getline().

I'm curious if the long line is split between getline() calls. If it was you could check if the returned line was newline terminated. If it wasn't terminated then call getline() again and concatenate the data yourself. (I know this is UGLY.)

You might also see if flushing the write side makes any difference. I wouldn't expect it to help but it may be worth a try.

I am ASSUMING that your situation is as simple as described---a single parent writes and a single child reads, no other interaction between the two processes.

  • Comment on Re: IO::Handle->getline() partial reads from pipe

Replies are listed 'Best First'.
Re^2: IO::Handle->getline() partial reads from pipe
by saintmike (Vicar) on Dec 19, 2013 at 17:32 UTC
    I'm curious if the long line is split between getline() calls. If it was you could check if the returned line was newline terminated. If it wasn't terminated then call getline() again and concatenate the data yourself. (I know this is UGLY.)
    Yeah, I was curious about that as well, but haven't tried re-reading it because I think that'd be a bug in getline() and should be fixed there. getline() should always read until a newline (or EOF), and neither seems to be the case (unless the kernel erroneously sends EOF for some reason).

    The problem is that I can't reproduce it in a standalone script, it only happens randomly on a busy production server, so my options in debugging this are limited.

    You might also see if flushing the write side makes any difference. I wouldn't expect it to help but it may be worth a try.
    That one I've tried, using printflush() instead of print on the IO::Handle of the writer makes no difference.

    And yes, it's as simple as described, a single writer and a single reader.

      I suspect I'm of little help, but I'll throw out some dumb ideas.

      You could try a series of partial writes of a long line to see if that reproduces the problem.

      Is there any CRLF or UTF8 processing at play?

      Are you able to test using the exact production data?

      Is the failure a matter of a truncated line, or premature end of file?

      And did you test printflush() return value? The print, printflush, flush, close methods all return true for success.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1067782]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others examining the Monastery: (2)
As of 2020-07-03 23:20 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?