Backticked commands and the readpipe blues

grinder has asked for the wisdom of the Perl Monks concerning the following question:

Suppose I want to do something like:

my @output = `/bin/foo -p stuff`;

If I want, I could build a list of arguments up on the fly, and pass the list to the command to be run, and everything would be funky.

    my @args = map{ ($_, foo($arg{$_})) } keys %args;
    my @output = `/bin/foo @args`;
[download]

I have a couple of problems with this, in that the @args variable doesn't really serve any purpose, apart from allowing me to avoid the rather cumbersome:

my @output = `/bin/foo @{[map{ ($_, foo($arg{$_})) } keys %args]}`;

But in any event it just so happens that foo() can return strings containing $ characters, which are then munged by the shell, and the called program gets confused. I can't use the single quoted variant of qx//, because then @args won't be interpolated, which is even worse.

In Programming Perl 3^rd Edition, it explains that "backticks don't have a list-argument form that is proof against the shell," but it goes on to say "a multi-argument form of the underlying readpipe operator is in development; but as of this writing, it isn't quite ready for prime time".

In terms of system, you can choose, or not, to pass through the underlying shell by doing:

  system( '/bin/foo    -p    stuff' ); # passes by the shell
  system( '/bin/foo', '-p', 'stuff' ); # bypasses the shell
[download]

Backticked commands are always fed to the shell. This poses a problem if the command line in question contains a dollar sign, because the shell will try and interpolate an evironment variable in its place and all hell breaks loose.

That is, there is no syntax to do:

  my @out = `/bin/foo`, qw/-p stuff/;
[download]

You have to go the whole hog, and fork and read back from your child. Which gives something like:

my $pid;
if( !defined($pid = open( KIDP, '-|' ))) {
    print child fork error: $!\n";
}
elsif( $pid ) {
    while( <KID> ) {
        chomp;
        # digest $_
    }
    close KID;
}
else {
     $ENV{PATH} = '/bin:/usr/bin:/usr/local/bin';
     my @args = map{ (foo, foo($_)) } keys %data;
     exec '/usr/local/bin/check-netscreen', @args;
     die "ping child died \$!=$! \$@=$@\nargs = @args";
}
[download]

So I wondered, given PP's hint, about using readpipe but the results were less than glorious.

Running the following code...

#! /usr/bin/perl -w

use strict;

my @out = readpipe( '/bin/foo', qw/one two three/ );
[download]

... produces the expected output except that the first line returned is the name of the command (a problem we can get around easily enough, if it's strictly reproducible). Unfortunately, the last element of @args is not processed correctly. It does not appear to be passed to the called program, and the script emits the following to STDERR:

Can't exec "three": No such file or directory

On the other hand, the shell is bypassed, which is what I wanted. This is perl 5.6.1. Hence my main questions, is readpipe (finally) still not ready for prime time? and/or, is there a better way to that I have overlooked to allow you to perform backticked commands that bypass the shell? That is, replacing 17 lines of code by 1. This looks like a case where Perl doesn't make the difficult things very easy, but like I said, I'm probably missing something obvious.

print@_{sort keys %_},$/if%_=split//,'= & *a?b:e\f/h^h!j+n,o@o;r$s-t%t#u'

Comment on Backticked commands and the readpipe blues Select or Download Code

Replies are listed 'Best First'.
Re: Backticked commands and the readpipe blues by Zaxo (Archbishop) on Sep 23, 2002 at 14:41 UTC
The readpipe function is questionable in other ways: $ perl -e'print prototype("CORE::readpipe")' Can't find an opnumber for "readpipe" at -e line 1. $ [download] Source diving shows that backticks and iirc readpipe, too, are implemented by `do_backticks`. After Compline, Zaxo	[reply] [d/l]
Re: Backticked commands and the readpipe blues by Flexx (Pilgrim) on Sep 23, 2002 at 15:25 UTC
Hi, grinder! Maybe I completely miss your point, but what's wrong about my @args = map { ($_, foo($arg{$_})) } keys %args; # either: prepend \ to all $ in @args s/\$/\\\$/g foreach @args; # or maybe: quote all single quotes, then single-quote all args s/'/\\'/g foreach @args; @args = map "'$_'", @args; my @output = `/bin/foo @args`; [download] Am I missing something? So long, Flexx	[reply] [d/l]
Re:x2 Backticked commands and the readpipe blues by grinder (Bishop) on Sep 23, 2002 at 16:37 UTC
What you're missing is the fact that I want to bypass the shell. I don't want to subject to its quoting rules and I don't want it to see 'foo;rm -rf /'. I don't want to second-guess the shell by escaping metacharacters and hope I get them all... I just don't want to have to worry at all. And from my dim, dark memories of shell programming, to quote a single quote, e.g. `don't`, you have to say `'don'"'"'t'`, which means that your code would produce incorrect results. This stuff is tricky! Backticking is a concise construct for extracting output from external programs, but as things stand, you cannot avoid bringing the shell into the picture. If you could be certain that you were calling the child program directly, a lot of the security hassle would magically evaporate. It's another "Doctor, Doctor" story. Patient: Doctor, doctor, when I stick my arm out the car window, it gets smashed to pieces by a passing truck. Doctor: Well, don't do that then. If I knew I didn't have to stick my arm out, I'd be fine. But the only way to do that in the current scenario is to go through all the hoops of forking and listening to my child. In my books this is One More Damned Thing To Go Wrong. Programming Perl, the book, hinted that in the future such a thing was going to be possible, but on the face of things it looks like it was an idea never went anywhere. print@_{sort keys %_},$/if%_=split//,'= & *a?b:e\f/h^h!j+n,o@o;r$s-t%t#u'	[reply] [d/l] [select]
Re^3: Backticked commands and the readpipe blues by Flexx (Pilgrim) on Sep 23, 2002 at 21:13 UTC
Hi grinder! Thanks for the clarification.. I did indeed not see that you'd actually really, really want to bypass the shell.. ;) Under that aspect, in a way, it's just fair you'll have to cope with what the shell usually does for you... ;) If I recall correctly, system() calls execvp(), maybe you could do that yourself. But that would, of course, involve XS hacking, something I didn't want to touch so far (which might turn out to be a fear of nothing once I do). Then again, maybe there's something out there already... Umm... I wish I could be of more help... Good luck & so long, Flexx	[reply]
Re: Backticked commands and the readpipe blues by Aristotle (Chancellor) on Sep 23, 2002 at 17:38 UTC
If I'm not completely missing the point, it sounds as though IPC::Open2 or IPC::Open3 will do what you want with a little more red tape required than the backticks need. Update: I would appreciate if whoever --ed this could tell me if they thought I was wrong or rude. I don't see either in my post. Makeshifts last the longest.	[reply]
Re: Backticked commands and the readpipe blues by blakem (Monsignor) on Sep 23, 2002 at 13:36 UTC
oops... nevermind... ~~I looks like readpipe() is expecting an EXPR not a LIST. Have you tried:~~ ~~`readpipe( '/bin/foo one two three' );`~~ ~~[download]~~ -Blake	[reply] [d/l]


P is for Practical
	PerlMonks