Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling
 
PerlMonks  

Perl ignores lines with bold characters from a pipe on Windows

by kingkongrevenge (Scribe)
on Mar 03, 2007 at 02:20 UTC ( [id://603010]=perlquestion: print w/replies, xml ) Need Help??

kingkongrevenge has asked for the wisdom of the Perl Monks concerning the following question:

I'm trying to pipe in text from a command on Windows; we'll call the command "mycmd". Some output lines from mycmd end in bold characters and no matter what I try perl completely ignores these lines. Binmode does not help.
> cmd > test_shell > perl my $res = `cmd`; open (TEST, ">test_perl1"); print TEST $res; close TEST; open (TEST, ">test_perl2"); open (CMD, "cmd |"); binmode CMD; #Tried with and without this. while (<CMD>) { print TEST "$_"; } close CMD; close TEST; #Let dos shell do io? The 'cmd /C' bit should be superfluous.(?) #End result is the same either way. `cmd /C "mycmd > test_perl3"`; EOF > wc -l test_shell test_perl1 test_perl2 test_perl3 111 test_shell 107 test_perl1 107 test_perl2 107 test_perl3
What the heck is going on here? How can Perl force the shell to throw away all lines with bold characters? I get the same results with both Perl 5.005 and Perl 5.8.8.

Replies are listed 'Best First'.
Re: Perl ignores lines with bold characters from a pipe on Windows (source not dest)
by tye (Sage) on Mar 03, 2007 at 03:14 UTC

    I strongly suspect that the answer that explains this will actually be to the question "Why doesn't mycmd write the lines that would contain bold characters when I launch it these ways from Perl?".

    The third case is especially indicative as Perl isn't doing any of the "problem" I/O so how can Perl be ignoring any lines? Perl isn't between mycmd and test_perl3 so the contents of test_perl3 have nothing to with Perl's I/O features nor could Perl "ignore" things and have any impact on what ends up in test_perl3.

    You need to discard your theory that Perl is ignoring lines and look at the problem again. There are other test cases you can try (running mycmd vs. perl -e"system('mycmd')", for example). You should look for more information about mycmd also.

    - tye        

      Probably correct. Unfortunately I have no idea how mycmd might be analyzing its parent process. To my knowledge no relevant environment variables are set either from the shell or from Perl. Mycmd is closed source, so I'm likely at a dead end.

      How can I check that Perl isn't setting any environment variables?

        How can I check that Perl isn't setting any environment variables?

        AFAIK, it doesn't. If you don't, that is. OTOH under *NIX it is possible to detect whether a file descriptor is connected to a tty or not, which is often used to "understand" whether a program's STDOUT is being piped or redirected to a file or not. (This is what e.g. ls(1) does under recent releases of Linux, at least: it acts like ls -1 if you pipe it to something else, and similarly if you set --color=auto it will print coloured output on the terminal, but will disable it when piping or redirecting to a file - in fact the switch is set by default in many distros.) Maybe under windows it's the same. Let me see:

        C:\temp>perl -le "open F, '>F'; print -t *F; print -t *STDOUT" 1
Re: Perl ignores lines with bold characters from a pipe on Windows
by almut (Canon) on Mar 03, 2007 at 04:25 UTC

    Just a wild guess: your mycmd might be mixing output via redirectable streams with "direct" console output via WriteConsole. AFAIK, the latter would typically be used to render highlighted text etc., making use of console "attributes". In particular, note that

    Both WriteConsole and WriteFile can be used for console I/O. While WriteConsole supports writing Unicode characters to a console screen buffer, WriteFile does not. However, WriteConsole fails if it is used with a standard handle that is redirected to a file. (... more)

    (emphasis added)

    This might explain why some lines get lost when capturing redirectable output only (stdout).

    As to why the console output does end up in the file test_shell when redirecting output from the shell command line, I don't know. Maybe some additional magic is going on in this case...(?)

    Anyway, what kind of program is this? Do you have the sources? If not, you might try a tool like StraceNT to find out what API calls are being used... (in particular look for WriteConsole, WriteFile, etc.)

      This is a port from Unix (cleartool), so I doubt it's MS api oddities.

        I think it is the Win32 API oddity, as the Win32 console has no ANSI or terminal emulation. If you want to output any color or weirdo formatting, you have to output these by directly writing to the console window yourself, bypassing the STDOUT filehandle. Hence I guess that the programmers wanted to keep their fancy feature of bold output and hence destroyed the ability to redirect the whole output.

Re: Perl ignores lines with bold characters from a pipe on Windows
by pKai (Priest) on Mar 03, 2007 at 09:34 UTC

    As proposed by tye you should be able to test the different theories mentioned in other replies.

    Check different scenarios on a plain cmd command line.

    commandoutput seen in/oninterpretation
    outerrscreen
    1.mycmd >out 2>err+-+almut
    2.++-Joost
    other?post it, and someone will explain
Re: Perl ignores lines with bold characters from a pipe on Windows
by Joost (Canon) on Mar 03, 2007 at 02:27 UTC

      Definitely nothing going to STDERR.

      In case it matters, the bold lines look this:
      C:\a\file [AFLAG]

      Perhaps the bolded '[' means something to perl IO? I'm grasping at straws.

        In case it matters, the bold lines look this:
        C:\a\file [AFLAG]

        Well, text is text, and it knows no intrinsic notion of charachter attributes, like bold. However one early, conceptually simple and long-lasting strategy to achieve them was and is by means of ANSI escape codes: support for them is built in under Linux. Back in the days of good 'ol DOS, you would use the ANSI.SYS device driver to enable them. (And at some point ANSI art flourished.) This remained possible up to the 9.x Windows releases, but NT based ones do not support it anymore, which is the reason why dedicated ANSI art viewers and, speaking of Perl, a module like Win32::Console::ANSI exist. Thus a program that wants to output charachters with attributes under recent Windowsen needs to dialogue directly with the console APIs, e.g. in Perl by means of Win32::Console upon which Win32::Console::ANSI relies. Chances are that it is not perl that "ignores" lines with bold charachters, but that those lines are not printed to STDOUT at all.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://603010]
Approved by Joost
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others examining the Monastery: (5)
As of 2024-04-16 06:57 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found