|
vr has asked for the wisdom of the Perl Monks concerning the following question:
I thought Perl would take care of that for me.
On Linux:
~$ perl -E 'system $^X, "-E", "say for \@ARGV", 1, qq(\x27), 2'
1
'
2
~$ perl -E 'system $^X, "-E", "say for \@ARGV", 1, qq(\x22), 2'
1
"
2
But on Windows (no test for single quote, because irrelevant and works OK):
>perl -E "system $^X, '-E', 'say for @ARGV', 1, qq(\x22), 2"
1
2
So I have to do this:
>perl -E "system $^X, '-E', 'say for @ARGV', 1, qq(\\\x22), 2"
1
"
2
What was the space character? Just wondering if this behaviour was some oversight on porters' side? Of course not much trouble to escape all '"' 's myself, once I know about it. Are there other known similar surprises to watch out for?
|
Re: Having to manually escape quote character in args to "system"?
by haukex (Archbishop) on Sep 12, 2017 at 12:38 UTC
|
AFAIK it isn't as easy for Perl to avoid the shell on Windows, and the system docs do say "On Windows, only the system PROGRAM LIST syntax will reliably avoid using the shell; system LIST, even with more than one element, will fall back to the shell if the first spawn fails." So you might want to try system {$^X} $^X, ...;. Otherwise, there is Win32::ShellQuote, and I wrote an article about various other methods for running system commands, some of which also help you avoid the shell, here. In particular, newer versions of IPC::Run3 will automatically use Win32::ShellQuote on Windows.
| [reply]
[d/l]
[select] |
|
|
AFAIK it isn't as easy for Perl to avoid the shell on Windows,
The issue is not the shell.
A windows program is not called with an array of arguments (as in Unix) but with a single command line and it is the program (not the shell) the one that breaks the command line into an array of arguments.
To make things worse, every program may use its own rules to process the command line. Nowadays things are more or less standardized (see CommandLineToArgvW, introduced with Windows 2000 and Windows XP), but historically, every language supporting library would use its own variation (for instance, see C++), so in order to quote a command properly, you should take into account the program implementation language!
In summary, the real issue is that in order to call a program in Windows with a list of arguments you have to quote and combine those arguments into a single command line and that quoting in Windows can be really tricky.
Update: An interesting read: Everyone quotes command line arguments the wrong way.
| [reply] |
|
|
Everyone quotes command line arguments the wrong way is quite funny, in a sad way, and it is wrong. As wrong as any other program attempting to quote on Windows. It is a game that you simply can not win.
You explained the basic problem: Arguments are passed to programs as a single string on systems derived from CP/M (i.e. DOS, Windows, OS/2), and programs (or the underlying runtime libraries) decide how to split that single string into arguments (see also Re^3: Perl Rename). Backwards compatibility to ancient DOS and WinNT, including bugs in command.com and cmd.exe, have lead to a ridiculous amount of complex rules for quoting and escaping.
The CommandLineToArgV convention mentioned in "Everyone quotes command line arguments the wrong way" is just that - a convention. All programs are free to use different quoting rules, and at least legacy programs do have different rules. (I did not look up or test, but I would not be surprised if cygwin-based programs would implement very different quoting rules, or even use a cygwin-only way to pass argv[] around, with a command line string only as fallback for non-cygwin programs.)
Pretending that this convention is universal for all programs, and claiming that code that escapes and quotes according to the convention is the only correct solution, would be really funny, if it was posted by a noob in some dusty corner of the internet or our local universal expert. Posting that at microsoft.com is just sad.
Unix has gone a long way, but the authors got argument passing right at the first attempt (i.e. fork() and exec()). And based on that lucky API, they made argument-splitting a problem of the shell, so you can use exactly the same quoting for all invoked programs. Over time, the shells got rid of most argument-splitting and argument-passing problems. That made quoting rules on Unix quite simple (but still far from being perfect). The best thing is that on Unix, you don't have to invoke the shell at all, so you don't have to quote at all. You pass a list of arguments to exec(), and main() will get exactly that list in argv[].
Alexander
--
Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)
| [reply]
[d/l]
[select] |
|
|
|
|
|
|
Thank you very much for the details! I hardly ever run external commands on Windows, so I haven't gotten into the details very much, other than that Win32 apparently doesn't have an equivalent of execvp(3), and that Perl actually does its own quoting internally, which unfortunately doesn't seem to be perfect. But I have heard a few good things about Win32::ShellQuote, and I haven't had any problems with IPC::Run3 on Windows (although I may just have not yet run into a case of really complicated quoting).
| [reply] |
Re: Having to manually escape quote character in args to "system"?
by salva (Canon) on Sep 12, 2017 at 12:54 UTC
|
But on Windows (no test for single quote, because irrelevant and works OK):
>perl -E "system $^X, '-E', 'say for @ARGV', 1, qq(\x22), 2"
1
2
Tracing the script with Process Monitor shows that the system call becomes...
C:\Strawberry\perl\bin\perl.exe -E "say for @ARGV" 1 " 2
It seems that multi-arg system doesn't quote arguments that already have quotes... maybe some poor application of DWIM principles or a work-around for keeping compatibility with old scripts written before the automating quoting was introduced. | [reply]
[d/l]
[select] |
Re: Having to manually escape quote character in args to "system"?
by huck (Prior) on Sep 12, 2017 at 12:37 UTC
|
I think its about the windows cmd shell and its quoting needs.
These give me a good clue but i cant quite explain what i mean
perl -E "$,='|'; say $^X, '-E', 'say for @ARGV', 1, qq(\\\x22), 2"
C:\Perl\bin\perl.exe|-E|say for @ARGV|1|\"|2
perl -E "$,='|'; say $^X, '-E', 'say for @ARGV', 1, qq(\x22), 2"
C:\Perl\bin\perl.exe|-E|say for @ARGV|1|"|2
perl -E "say for @ARGV" 1 \" 2
1
"
2
perl -E "say for @ARGV" 1 " 2
1
2
In the last case I SUSPECT windows is seeing " 2 as a single arg with an unterminated quote and its terminating it for us | [reply]
[d/l]
[select] |
Re: Having to manually escape quote character in args to "system"?
by Anonymous Monk on Sep 12, 2017 at 14:35 UTC
|
| [reply] |
|
|
Thank you, everyone, for very enlightening answers, I've learned quite a few things from them.
Unfortunately, ShellQuote::Any::Tiny doesn't do it right:
>perl -MShellQuote::Any::Tiny=shell_quote -E "system $^X, '-E', 'say f
+or @ARGV; <STDIN>', 1, shell_quote(qq(\x5c \x5c)), 2"
1
\\ \
2
because it escapes every backslash. But only terminating one, which precedes the closing double-quote (if any), should be escaped -- as follows from "Everyone quotes command line arguments the wrong way".
Win32::ShellQuote appears to handle everything correctly.
Actually, I lived happily before this, relying only on Perl's built-in ability to double-quote arguments, which contain spaces. Until someone, in her wisdom, decided that a double-quote in the middle of template name, to be passed as argument to another script, is a good idea.
| [reply]
[d/l] |
|
|
> Unfortunately, ShellQuote::Any::Tiny doesn't do it right:
Bug reported.
($q=q:Sq=~/;[c](.)(.)/;chr(-||-|5+lengthSq)`"S|oS2"`map{chr |+ord
}map{substrSq`S_+|`|}3E|-|`7**2-3:)=~y+S|`+$1,++print+eval$q,q,a,
| [reply]
[d/l] |
|
|
|
|
The bug has been fixed (see my other comment for the link). Can you check?
($q=q:Sq=~/;[c](.)(.)/;chr(-||-|5+lengthSq)`"S|oS2"`map{chr |+ord
}map{substrSq`S_+|`|}3E|-|`7**2-3:)=~y+S|`+$1,++print+eval$q,q,a,
| [reply]
[d/l] |
|
|
|
|
