http://qs321.pair.com?node_id=1158649

StuLong has asked for the wisdom of the Perl Monks concerning the following question:

I have a problem that is driving me to the brink.

I am calling an external script from a daemonized perl script. The external perl script is used to connect to a NetApp filer and get the major and minor API release level, and return in to the calling daemon.

Here's my problem. Sometimes the daemon script collects the stdout of the external, and in other cases it doesn't. However, when the external script is run from the command line, it works in all cases.

Here's my code snippet where I call the external.

my $get_version_prog = 'na_get_version.pl'; my $major_minor = qx($Bin/$get_version_prog $filer)

Both the daemon and the external use syslog to write to the application log file. Here's the final processing from the external:

syslog('notice', "Version info for $filer $major_minor"); print STDOUT $major_minor; exit 0;

and here's what I log from the daemon:

syslog('debug', "OnTap major/minor for $filer is $major_minor");

and finally, here's what shows up in the log file:

Mar 23 15:38:51 sppubu01 snapaidd[639]: [ID 702911 local7.notice] Gett +ing version info for SP1STONC02 Mar 23 15:38:53 sppubu01 snapaidd[639]: [ID 702911 local7.notice] Vers +ion info for SP1STONC02 1.31 Mar 23 15:38:53 sppubu01 snapaidd[636]: [ID 702911 local7.debug] OnTap + major/minor for SP1STONC02 is

So, the external logs the proper version info, but the print to STDOUT is not captured by the qx.

As I said before, running the external stand-alone prints the proper version info. More weird is that the daemon works as expected if the returned OnTap version from the NetApp is 1.21; it only fails if the returned OnTap version is 1.31, and only fails when being invoked through the daemon. However, the external is connecting to the NetApp, getting the proper API release, and logging it.

Replies are listed 'Best First'.
Re: qx not always capturing stdout
by dasgar (Priest) on Mar 23, 2016 at 21:50 UTC

    Using qx or backticks will only capture the STDOUT. I'm wondering if the calls where you are not capturing anything could be related to the "output" of the external command is going to the STDERR instead.

    Whenever I have used backticks and I don't get what I expect, a common debugging technique that I use is to store the external command in a variable and print that out. That has helped me find mistakes in constructing the external command that I wanted to run.

    One suggestion that you could try is to look at using Capture::Tiny, which will let you capture STDOUT, STDERR and the exit code of an external command.

Re: qx not always capturing stdout
by graff (Chancellor) on Mar 24, 2016 at 04:29 UTC
    Following up on dasgar's explanation: when you run a program at the command line in a terminal window, the default behavior (if you don't use any redirection) is to have both stdout and stderr outputs printed to the terminal.

    I don't know about the "cmd.exe" (windows command-line shell), but in any *n*x shell (e.g. bash), you can redirect stdout and stderr independently.

    When you use perl to run a shell command via backticks or qx, you could (if you want) include redirection, to have both stdout and stderr print to stream, like this:

    my $major_minor = qx($Bin/$get_version_prog $filer 2>&1)
    (not tested)

      The redirection works for Windows too.

      Premature optimization is the root of all job security
Re: qx not always capturing stdout
by Paladin (Vicar) on Mar 23, 2016 at 21:42 UTC
    With no full program to examine, it's hard to say what's going on. Would it be possible to create a small sample program that shows the same issue and post that here? Is there any pattern to when it does and doesn't capture the output (time of day, etc.)?

      I have been trying to write a test program that will re-create the issue. So far, no luck. Everything I've done to call the external in my test programs works as expected. I've been trying to make the test programs more complex, adding in more modules, etc., but so far nothing exhibits the symptoms but the main daemon. I will continue to try and replicate.

      My next plan of attack is to un-daemonize the daemon and try running it through the debugger.

Re: qx not always capturing stdout
by Anonymous Monk on Mar 23, 2016 at 22:19 UTC
    Try Capture::Tiny
    #!/usr/bin/perl -- use strict; use warnings; use Capture::Tiny qw/ capture /; my @cmd = ( "$Bin/$get_version_prog", $filer ); my( $stdout, $stderr, $exit ) = capture { system { $cmd[0] } @cmd; };; $exit and die "it failed with $exit and $stderr ";
Re: qx not always capturing stdout
by Marshall (Canon) on Mar 23, 2016 at 21:16 UTC
    Update: by popular demand, I strike this question myself...This qx thing is relatively new. What happens when you use the more traditional "backticks"?

      `...` is just short for qx`...`, which is short for readpipe(qq`...`).

      $ perl -MO=Concise,-exec -e'`...`' 1 <0> enter 2 <;> nextstate(main 1 -e:1) v:{ 3 <$> const[PV "..."] s 4 <1> backtick[t1] vK 5 <@> leave[1 ref] vKP/REFC -e syntax OK $ perl -MO=Concise,-exec -e'qx`...`' 1 <0> enter 2 <;> nextstate(main 1 -e:1) v:{ 3 <$> const[PV "..."] s 4 <1> backtick[t1] vK 5 <@> leave[1 ref] vKP/REFC -e syntax OK $ perl -MO=Concise,-exec -e'readpipe(qq`...`)' 1 <0> enter 2 <;> nextstate(main 1 -e:1) v:{ 3 <$> const[PV "..."] s 4 <1> backtick[t2] vK/1 5 <@> leave[1 ref] vKP/REFC -e syntax OK

      This is by no means new; this has been the case for at least 18 years. (The oldest documentation I could find is perl5.005_53 from Oct 31st, 1998.)

      I'm not sure what your definition of relatively new is, but qx has existed since at least 5.8.8 (oldest Perldoc readily available on perldoc.perl.org) and 5.8.8 was released over 10 years ago (2006-Jan-31).
      No change, same output. Or rather lack thereof.
        Found an example, where the daemonization closes STDOUT and STDIN in order to not create files. This could confuse the child process (which has no stdout). So either, maybe this works:

        use Daemon::Simple; open(SAVEIN, "<&STDIN"); open(SAVEOUT, "<&STDOUT"); Daemon::Simple::init("daemon"); open(STDIN, "<&SAVEIN"); open(STDOUT, "<&SAVEOUT"); print qx("/external/command"); close(STDIN); close(STDOUT);

        or you will have to redirect your output to a file

        qx("/external/command > /tmp/file")

        and read it back using open(), diamond and close()