Disclaimer: I don't have enough inside knowledge to answer your real question, nor do I have anything to prove my following notion.

What could also be the case is that msg.exe does its own command line argument parsing. On my machine (Windows XP, Dutch) I get this:

C:\>msg /?
Een bericht naar de gebruiker sturen.

MSG {gebruikersnaam | sessiennaam | sessie-ID | @bestandsnaam | *}
    [/SERVER:servernaam] [/TIME:seconds] [/V] [/W] [bericht]
[download]

Liberally translated, this says that msg.exe takes the following arguments:

1. Either a user name, a session name, a session ID, a "@" + file name, or a "*"
2. Optional /SERVER:, /TIME:, /V, and /W switches
3. Optionally a message

¹ Or, more precisely, that there is something that could be interpreted as a "/w" before the message, even though that wasn't the intended meaning.

OK, I understand what you're saying. And it is an interesting observation, which I didn't take into account so far. On the other hand, I observed this behavior also with other executables. In fact, my actual problem is related to another (specific) program, and I used the msg.exe really just to have something simple to demonstrate the issue in a post in this forum. In particular, I noticed that double quotes characters inside one of the passed arguments can destroy the whole execution. Which makes me think, that there is "something" interpreting the arguments, instead of purely passing the argument string "as is" to the executable, at the desired argument position. Anyway, thanks for your feedback!

Which makes me think, that there is "something" interpreting the arguments, instead of purely passing the argument string "as is" to the executable, at the desired argument position.

Yeah, its called

$ perl -V:sh
sh='cmd /x /c';
[download]

Minor, OT Nit: "As an example, let's use a simple executable, that (I think) is available on every windows machine: msg.exe."

Actually, msg.exe was not part of Win2k. It was in an early (initial?) release of XP.

It's not on my Vista machine either.

Microsoft Windows [Version 6.0.6002]
Copyright (c) 2006 Microsoft Corporation.  All rights reserved.

C:\Users\wfsp>msg
Send a message to a user.

MSG {username | sessionname | sessionid | @filename | *}
    [/SERVER:servername] [/TIME:seconds] [/V] [/W] [message]

  username            Identifies the specified username.
  sessionname         The name of the session.
  sessionid           The ID of the session.
  @filename           Identifies a file containing a list of usernames
+,
                      sessionnames, and sessionids to send the message
+ to.
  *                   Send message to all sessions on specified server
+.
  /SERVER:servername  server to contact (default is current).
  /TIME:seconds       Time delay to wait for receiver to acknowledge m
+sg.
  /V                  Display information about actions being performe
+d.
  /W                  Wait for response from user, useful with /V.
  message             Message to send.  If none specified, prompts for
+ it
                      or reads from stdin.
[download]

update: I dread to think what you've done to your vista box :-)

I recommend to create a small Perl script:

# file joined-echo.pl
print join('||', @ARGV), "\n";
[download]

And use system($^X, 'joined-echo.pl', @rest_of_your_args) for testing - that way you know exactly that the command you're testing doesn't do any magical argument processing on its own.

Yes, tried that. And guess what... I get arguments split (by spaces) before the executable specified in the system() call is run.

So if this is the content of my "main" perl script:

my $txt = '/w my > text >= [with] \ symbols " /wabc/def (xyz)';
system($^X, 'joined-echo.pl', $txt, 'ARG2');
[download]

...Then this is the output:

/w||my||>||text||>=||[with]||\||symbols|| /wabc/def (xyz) ARG2
[download]

As one can see, the perl called by system() does not get 2 arguments - as I would expect - but much more. Mhmm.

See exec always invokes the shell? win32, the situation is brainfuck-cubed

The way I get around it, is to always pretend like I'm typing in cmd.exe

#!/usr/bin/perl --
use strict;
use warnings;

if( @ARGV ){
    print join "\n", map({"( $_ )"} @ARGV), "\n";
} else {
    my $txt = '/w my > text >= [with] \ symbols " /wabc/def (xyz)';
    my( @args ) = win32_quote(  'perl', __FILE__, $txt, 'ARG2', );
    print "YOU CAN TYPE THIS AT THE cmd.exe PROMPT\n   @args\n\n";
## command.com doesn't like it (it wants "perl" to be perl)
## i don't know what powershell does :)
    system @args;
}

sub win32_quote {
    my( @args ) = @_;
    s~
(
 [%><|&^"]
)
    ~
{
 '%' => '^%',
 '>' => '^>',
 '<' => '^<',
 '"' => '\\"',
 '&' => '^&',
 '|' => '^|',

}->{$1}
    ~gex  for @args;
    $_=qq["$_"] for @args;
    return @args;
}

__END__
D:\>perl win32.quote.pl
YOU CAN TYPE THIS AT THE cmd.exe PROMPT
   "perl" "win32.quote.pl" "/w my ^> text ^>= [with] \ symbols \" /wab
+c/def (xyz)" "ARG2"

( /w my ^> text ^>= [with] \ symbols " /wabc/def (xyz) )
( ARG2 )


D:\>"perl" "win32.quote.pl" "/w my ^> text ^>= [with] \ symbols \" /wa
+bc/def (xyz)" "ARG2"
( /w my ^> text ^>= [with] \ symbols " /wabc/def (xyz) )
( ARG2 )
[download]

Glancing at your post, I see

my $txt = '/w sample text with some / special /w characters > xyz';
system('msg', 'username', $txt);
[download]

Is that the one that's giving you problems? That's suppose to be equivalent to

msg username "/w sample text with some / special /w characters > xyz"
[download]

Which is obviously incorrect. Does that answer your question? I gotta run without looking at your post in detail at this time.

And the sad truth is there is no way to do it on Windows, due to how CreateProcess works.

Many Thanks to all who contributed. Especially this post: http://www.perlmonks.org/?node_id=887534 to which someone has pointed, was extremely helpful.

Thank you!

And the sad truth is there is no way to do it on Windows, due to how CreateProcess works.

Sure there is, just quote it like CreateProcess expects, as shown in Re^3: system() implementation on Windows (again)

There are two problems here, and other nodes have covered parts of them.

First is that argument lists are always passed as a single string in Windows, as opposed to arrays on other systems. This is less of a problem than it appears, because 95% of programs use the same rules for parsing that string into an array. Roughly speaking, the rules are that arguments can be quoted with double quotes, and backslashes can escape any character.

The second issue is that cmd.exe uses different quoting rules than the normal parsing routine. It uses caret as the escape character instead of backslash.

The result of this is that you can't create a string that will be treated the same for both of these cases. This means you have to quote strings differently depending on if they have shell meta-characters or not. And there isn't any good way to check that without reimplementing the code to detect them that exists inside perl. So here is a routine that will quote arguments correctly to use with system:

sub quote_list {
    my (@args) = @_;

    my $args = join ' ', map { quote_literal($_) } @args;

    if (_has_shell_metachars($args)) {
        # cmd.exe treats quotes differently from normal argument parsi
+ng.
        # just escape everything using ^.
        $args =~ s/([()%!^"<>&|])/^$1/g;
    }
    return $args;
}

sub quote_literal {
    my ($text) = @_;

    # basic argument quoting.  uses backslashes and quotes to escape
    # everything.
    if ($text ne '' && $text !~ /[ \t\n\v"]/) {
        # no quoting needed
    }
    else {
        my @text = split '', $text;
        $text = q{"};
        for (my $i = 0; ; $i++) {
            my $bs_count = 0;
            while ( $i < @text && $text[$i] eq "\\" ) {
                $i++;
                $bs_count++;
            }
            if ($i > $#text) {
                $text .= "\\" x ($bs_count * 2);
                last;
            }
            elsif ($text[$i] eq q{"}) {
                $text .= "\\" x ($bs_count * 2 + 1);
            }
            else {
                $text .= "\\" x $bs_count;
            }
            $text .= $text[$i];
        }
        $text .= q{"};
    }

    return $text;
}

# direct port of code from win32.c
sub _has_shell_metachars {
    my $string = shift;
    my $inquote = 0;
    my $quote = '';

    my @string = split '', $string;
    for my $char (@string) {
        if ($char eq q{%}) {
            return 1;
        }
        elsif ($char eq q{'} || $char eq q{"}) {
            if ($inquote) {
                if ($char eq $quote) {
                    $inquote = 0;
                    $quote = '';
                }
            }
            else {
                $quote = $char;
                $inquote++;
            }
        }
        elsif ($char eq q{<} || $char eq q{>} || $char eq q{|}) {
            if ( ! $inquote) {
                return 1;
            }
        }
    }
    return;
}
[download]

Well... Thanks for providing detailed workarounds. But that's exactly my point: the same system() call cannot apparently be used on both platforms, win and unix.

And that's my problem. I have a piece of perl code which needs to execute an external program, and I want/need to make it portable across both platforms. It is not really satisfactory for me, when I have to introduce additional 30 lines of code for the windows part, just to catch all the possible quirks that can happen to me because of quotes or other special characters that might be contained in the arguments to my external program.

I had the hope, that the system() call in the argument array form, would do what the documentation says: pass the arguments _diretcly_ to the executable. But thanks to the discussion in this thread, I understand now that it's simply not possible on Windows. And I have to live with that (and obviously implement one or another workaround for this). Thanks again.