http://qs321.pair.com?node_id=11120244

Special_K has asked for the wisdom of the Perl Monks concerning the following question:

For reference, this is somewhat of a follow up question to a question I asked previously: https://perlmonks.org/?node_id=11120004

What is the impact on the value of $? if the output of an application is redirected to /dev/null? This is running under RHEL7.

For example, suppose I have a perl script that calls a compiled executable that seg faults:

`my_executable_that_seg_faults`; my $exit_value = $? >> 8; my $signal_num = $? & 127; my $dumped_core = $? & 128; printf("return value is %d\n", $?); printf("exit_value = $exit_value\n"); printf("signal_num = $signal_num\n"); printf("dumped_core = $dumped_core\n");

Running the above code produces the following output:

return value is 11 exit_value = 0 signal_num = 11 dumped_core = 0

This output is consistent with what I would expect based on information found here related to the $? variable: https://perldoc.perl.org/perlvar.html

However if I change the executable call above to the following:

`my_executable_that_seg_faults > /dev/null`;

The following output is produced:

return value is 35584 exit_value = 139 signal_num = 0 dumped_core = 0

The above output is significantly different even though the application still seg faulted. Here are my questions:

1. Why did the value of $? completely change when I added the output redirect to /dev/null but kept everything else the same?

2. Does the value of $? in the second case still contain the same information but encoded differently?

I created a different application with an intentional segmentation fault and called it using the above perl script. It produced the exact same output, suggesting that this behavior is not dependent on the behavior of the called application.

Replies are listed 'Best First'.
Re: Effect of redirecting output to /dev/null on $? value
by ikegami (Patriarch) on Aug 03, 2020 at 00:26 UTC

    `$cmd` usually involves running /bin/sh with -c and the value of $cmd as arguments.

    The use of the shell is optimized away if the command is simple enough. That is apparently the case in the first snippet.

    In the second snippet, you are getting the exit status of /bin/sh. It returns 0x80 | signal_num when the program it runs is killed by a signal.

    You could use the following to execute the program in the child process instead of creating a grandchild:

    `exec my_executable_that_seg_faults >/dev/null`;

    And you can avoid the shell entirely using IPC::Run.

    Update: Added exec (before I noticed jcb mentioning it).

      Thanks for the explanation. I ran a different, more complex application with a deliberate seg fault added again using my method above and now see the following as the output:

      sh: line 1: 26902 Segmentation fault my_complex_executable_that_s +egfaults > /dev/null return value is 35584 exit_value = 139 signal_num = 0 dumped_core = 0
      In the above case, 0x80 | signal_num = 0x80 | 0x0B = 0x8B = 139, which matches the listed exit_value. 35584 is just the exit value shifted up 8 bits, which is consistent with the definition of $?.

      What does the 26902 value indicate? Is that the 16-bit status int set by wait(2) from running my_complex_executable_that_segfaults that can be parsed by perl to assemble the fields of the $? variable, and can also be accessed directly in C/C++ using the macros WTERMSIG, WEXITSTATUS, etc. (source: https://linux.die.net/man/2/wait)?

        What does the 26902 value indicate?

        The shell generated that message to report that a subprocess with PID 26902 exited with a segmentation fault. The hint is that it starts with "sh: line 1:". The shell sees your command as a one-line shell script.

        What does the 26902 value indicate?

        pid? I don't think I've ever seen a message like that.

        It's not a exit status. 26902 = 0x6916. As a status, this would mean the process exited with code 0x69 and it was killed by signal 0x16, but those are mutually exclusive.

      How is the command executed if the shell is not invoked?
      Why does adding a redirect to /dev/null increase the complexity enough to invoke the shell?
      Finally, what method is used to execute the command if I had instead used perl's system() function instead of the backticks?

        This is pretty much all covered in the docs for system().

        Why does adding a redirect to /dev/null increase the complexity enough to invoke the shell?

        Because it's the shell that does the redirection.


        🦛

        How is the command executed if the shell is not invoked?

        By asking to system execute the specified program instead of asking the system to execute /bin/sh.

        Why does adding a redirect to /dev/null increase the complexity enough to invoke the shell?

        There's no point in including an entire shell in Perl. I don't know exactly what part of shell command parsing is implemented in Perl.

        Finally, what method is used to execute the command if I had instead used perl's system() function instead of the backticks?

        system($cmd), exec($cmd), open(my $pipe, '-|', $cmd) (and its 2-arg form) and open(my $pipe, '|-', $cmd) (and its 2-arg form) work the same way as `$cmd` (aka qx`$cmd` aka readpipe($cmd)).

        system($prog, LIST) where LIST returns at least one scalar won't invoke the shell.[1]

        system({ $prog } $prog, LIST) won't invoke the shell.


        1. It can in Windows.
Re: Effect of redirecting output to /dev/null on $? value
by jcb (Parson) on Aug 03, 2020 at 00:34 UTC

    As ikegami mentioned, the effect on $? is due to running a shell to perform the I/O redirection. If you can assume a Bourne shell, (probably a safe assumption on POSIX systems, which redirection to /dev/null already assumes) then you can use the exec shell command like so:

    `exec my_executable_that_seg_faults > /dev/null`;

    Note that this is the shell exec inside the qx//, not the Perl exec. The shell exec command causes the shell to replace itself with the desired executable and thus should cause $? to reflect the executable's exit status, instead of the shell's exit status.

      Backticks always uses the Bourne shell except on Windows. Well, it always uses /bin/sh. While that might not be the Bourne shell, it's going to be something that supports exec.

      `exec my_executable_that_seg_faults > /dev/null`;

      exec at the start of a command is one of the few things that are treated specially in Perl_do_exec3(), see Re^2: Improve pipe open? (redirect hook). It seems to disable the default optimization and forces the use of the default shell.

      There is a way to stay mentally sane on Unix/POSIX systems for I/O-redirection. Don't use system; instead, fork and exec manually:

      # (untested) my $pid=fork(); my @cmd_and_args=('my_executable_that_seg_faults'); # maybe push @cmd_and_args,qw( some arguments for the program ); defined($pid) or die "Can't fork: $!"; if ($pid) { # parent process waitpid $pid; #<-- sets $? } else { # child process open STDOUT,'>','/dev/null' or die "Can't redirect STDOUT: $!"; exec { $cmd_and_args[0] } @cmd_and_args; die "exec failed: $!"; }

      Note that ONLY indirect object notation on exec prevents all attempts of perl to be smart. See The problem of "the" default shell.

      Alexander

      --
      Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)

        I have used this before, but our questioner seemed to be asking for a minimal solution. This strategy is the only way if you need more than the basic pipes, but (having just checked) IPC::Open3 is actually implemented using system, presumably for compatibility on Windows, where fork must be emulated.

        That's horrible advice.

        The OP is complaining about errors being misreported, and you introduce a new such error.

        exec failures should not appears as exit code from a successfully launched child. Even system and open3 get that right. (They use a close-on-exec pipe to communicate the errno of failures to the parent.)