Re: IO::Handle->getline() partial reads from pipe

I used to know something about pipes, but that was a long time ago. At the kernel level a pipe as a maximum size (once upon a time I think 16K may have been typical). If you writes are larger than that then at the system level the child may get at most that max amount. I would think that getline() should buffer the read and read again until it reads data with an newline. It sounds like this isn't happening in your case. I'd expect that kind of behaviour from sysread(), but not getline().

I'm curious if the long line is split between getline() calls. If it was you could check if the returned line was newline terminated. If it wasn't terminated then call getline() again and concatenate the data yourself. (I know this is UGLY.)

You might also see if flushing the write side makes any difference. I wouldn't expect it to help but it may be worth a try.

I am ASSUMING that your situation is as simple as described---a single parent writes and a single child reads, no other interaction between the two processes.

Comment on Re: IO::Handle->getline() partial reads from pipe

Replies are listed 'Best First'.
Re^2: IO::Handle->getline() partial reads from pipe by saintmike (Vicar) on Dec 19, 2013 at 17:32 UTC
I'm curious if the long line is split between getline() calls. If it was you could check if the returned line was newline terminated. If it wasn't terminated then call getline() again and concatenate the data yourself. (I know this is UGLY.) Yeah, I was curious about that as well, but haven't tried re-reading it because I think that'd be a bug in getline() and should be fixed there. getline() should always read until a newline (or EOF), and neither seems to be the case (unless the kernel erroneously sends EOF for some reason). The problem is that I can't reproduce it in a standalone script, it only happens randomly on a busy production server, so my options in debugging this are limited. You might also see if flushing the write side makes any difference. I wouldn't expect it to help but it may be worth a try. That one I've tried, using printflush() instead of print on the IO::Handle of the writer makes no difference. And yes, it's as simple as described, a single writer and a single reader.	[reply]
Re^3: IO::Handle->getline() partial reads from pipe by wazat (Monk) on Dec 19, 2013 at 20:49 UTC
I suspect I'm of little help, but I'll throw out some dumb ideas. You could try a series of partial writes of a long line to see if that reproduces the problem. Is there any CRLF or UTF8 processing at play? Are you able to test using the exact production data? Is the failure a matter of a truncated line, or premature end of file?	[reply]
Re^3: IO::Handle->getline() partial reads from pipe by oiskuu (Hermit) on Dec 19, 2013 at 20:01 UTC
And did you test printflush() return value? The print, printflush, flush, close methods all return true for success.	[reply]


good chemistry is complicated, and a little bit messy -LW
	PerlMonks