Unexpected behavior with "map"

Clovis_Sangrail has asked for the wisdom of the Perl Monks concerning the following question:

Hello Esteemed Monks,

I'm trying to do stuff the Perl way, not the "old 'C' programmer futzing with Perl" way, so I decided to try and strip off the path portion of a directory listing with the Perl map function. I have a list like:

my @oldlist = qw X /etc/passwd /etc/group /etc/shadow X;
[download]

And I thought to map it into a new list without the "/etc/" part via:

my @newlist = map  s/\/etc\/// ,  @oldlist;
[download]

But that is nothing like what happened! Here is a sample program that runs this stuff (with an extra, non-matching input list element) and prints the output:

$         
$ cat t.pl
#!/usr/bin/perl

use strict;
use warnings;

my @oldlist = qw X /etc/passwd /etc/group /egg/drop /etc/shadow X;

my @newlist = map  s/\/etc\/// ,  @oldlist;

print "\nAfter map (newlist)\n";
print "=========\n";
foreach (@newlist) {
    print "$_\n";
}

$ ./t.pl

After map (newlist)
=========
1
1

1
$
[download]

WTF?? Now, actually I *am* an old 'C' programmer futzing with Perl, so instead of messing with the debugger I just printed everything out before and after the map:

$ cat ta.pl
#!/usr/bin/perl

use strict;
use warnings;

my @oldlist = qw X /etc/passwd /etc/group /egg/drop /etc/shadow X;

print "\nBefore map (oldlist)\n";
print "==========\n";
foreach (@oldlist) {
    print "$_\n";
}
my @newlist = map  s/\/etc\/// ,  @oldlist;

print "\nAfter map (oldlist)\n";
print "=========\n";
foreach (@oldlist) {
    print "$_\n";
}
print "\nAfter map (newlist)\n";
print "=========\n";
foreach (@newlist) {
    print "$_\n";
}

$ ./ta.pl

Before map (oldlist)
==========
/etc/passwd
/etc/group
/egg/drop
/etc/shadow

After map (oldlist)
=========
passwd
group
/egg/drop
shadow

After map (newlist)
=========
1
1

1
$
[download]

How can this be?? The documentation in the "intermediate Perl" book (where I'm try'ing to do chap1 ex2) says that the map function loads each member of the source list into $_ , evaluates the mapping expression in list context (which I figured would make a 1-element list, substituting a null string for the "/etc/" part if matched in $_) , and then puts (shifts?) the resulting list into the new resulting list. But *MY* map function is altering @oldlist !?!? I guess that something a little more complicated than the text is going on, something involving references instead of values being loaded in $_, so I decided I'd try using the block form of map, and make a separate local variable in the block to work the substitution upon, like so (after realizing I had to lose the ','):

my @newlist = map {
                    my $l = $_;
                    $l =~ s/\/etc\///;
                  }  @oldlist;
[download]

With a new string, I should have nothing more whatsoever to do with @oldlist, right? And yet:

$ ./tb.pl 

Before map (oldlist)
==========
/etc/passwd
/etc/group
/egg/drop
/etc/shadow

After map (oldlist)
=========
/etc/passwd
/etc/group
/egg/drop
/etc/shadow

After map (newlist)
=========
1
1

1
$
[download]

Now I'm not messing with the source list any more, but I am still getting a boolean or numeric context out of my block! Page 54 of my "Learning Perl" book ends the discussion of "forcing scalar context" with "...there's no corresponding function to force list context. It turns out never to be needed." I'm not so sure anymore... I have to create yet another local variable in the block to get the behavior I'm looking for ('before' print removed for brevity):

$ cat tc.pl 
#!/usr/bin/perl

use strict;
use warnings;

my @oldlist = qw X /etc/passwd /etc/group /egg/drop /etc/shadow X;

my @newlist = map {
                    my $l = $_;
                    $l =~ s/\/etc\///;
                    my $l2 = $l;
                  }  @oldlist;

print "\nAfter map (oldlist)\n";
print "=========\n";
foreach (@oldlist) {
    print "$_\n";
}
print "\nAfter map (newlist)\n";
print "=========\n";
foreach (@newlist) {
    print "$_\n";
}

$ ./tc.pl 

After map (oldlist)
=========
/etc/passwd
/etc/group
/egg/drop
/etc/shadow

After map (newlist)
=========
passwd
group
/egg/drop
shadow
$
[download]

So, why do I have to go thru all this? Why doesn't my original map function just put out the final value of the $_ variable?

Comment on Unexpected behavior with "map" Select or Download Code

Replies are listed 'Best First'.
Re: Unexpected behavior with "map" by runrig (Abbot) on Mar 12, 2014 at 18:59 UTC
`s///` operates on `$_`. It returns the number of substitutions. In map, each list argument is aliased to `$_`. `my @newlist = map { my $l = $_; $l =~ s/\/etc\///; my $l2 = $l; } @oldlist;` [download] You just need to return at the end, not assign (and you can save some backwhacking in your s/// by picking a different delimiter): `my @newlist = map { my $l = $_; $l =~ s\|/etc/\|\|; $l; } @oldlist;` [download] Or you can use Filter() from Algorithm::Loops. Update: I forgot about the 'r' modifier for s///. It makes s/// return the new value. So if you have a new enough perl, you can do: `my @newlist = s\|/etc\|\|r, @oldlist;` [download]	[reply] [d/l] [select]
Re: Unexpected behavior with "map" by toolic (Bishop) on Mar 12, 2014 at 19:03 UTC
s///r with perl 5.14+ (I think): `my @newlist = map s/\/etc\///r , @oldlist;` [download] Stupid mistakes I repeatedly make	[reply] [d/l]
Re: Unexpected behavior with "map" by kcott (Archbishop) on Mar 12, 2014 at 21:37 UTC
G'day Clovis_Sangrail, That's a mistake that I also seem to make from time to time. [I see ++runrig has provided an explanation.] I usually use `map` with braces so when I get a list of boolean values returned from something like: `map { s/match/replace/ } @array` [download] It's easily fixed like this: `map { s/match/replace/; $_ } @array` [download] As you've been delving into the map documentation to understand more closely how `map` operates, you might also be interested to know that you can return nothing from a `map` expression by causing the expression to be evaluated as an empty list. For example, this code: `my @x = qw{a b a c bab aca}; my @y = map { /(.?a.?)/ ? $1 : () } @x; print "\|$_\|\n" for @y;` [download] outputs: `\|a\| \|a\| \|bab\| \|ac\|` [download] Also note how that differs from grep which you could emulate with that `map` expression by changing `$1` to `$_`. This code: `my @x = qw{a b a c bab aca}; my @y = map { /(.?a.?)/ ? $_ : () } @x; print "\|$_\|\n" for @y;` [download] and this code `my @x = qw{a b a c bab aca}; my @y = grep { /(.?a.?)/ } @x; print "\|$_\|\n" for @y;` [download] both produce identical output: `\|a\| \|a\| \|bab\| \|aca\|` [download] Obviously, there's really no point in writing that extra code (and, presumably, causing the extra processing) to make `map` emulate `grep`. However, it's feasible that you could have a function that operated as either based on a flag — I can't think of a situation where I've ever needed to do that but, if it did arise, it could be easily implemented like this: `$ perl -e ' my $use_grep = 0; my @x = qw{a b a c bab aca}; my @y = map { /(.?a.?)/ ? $use_grep ? $_ : $1 : () } @x; print "\|$_\|\n" for @y; ' \|a\| \|a\| \|bab\| \|ac\|` [download] `$ perl -e ' my $use_grep = 1; my @x = qw{a b a c bab aca}; my @y = map { /(.?a.?)/ ? $use_grep ? $_ : $1 : () } @x; print "\|$_\|\n" for @y; ' \|a\| \|a\| \|bab\| \|aca\|` [download] -- Ken	[reply] [d/l] [select]
Re: Unexpected behavior with "map" by fishmonger (Chaplain) on Mar 12, 2014 at 19:12 UTC
Have you considered using a module designed for parsing paths? `use File::Basename; use Data::Dumper; my @oldlist = qw X /etc/passwd /etc/group /egg/drop /etc/shadow X; my @newlist = map {basename $_} @oldlist; print Dumper \@newlist;` [download] outputs: `$VAR1 = [ 'passwd', 'group', 'drop', 'shadow' ];` [download]	[reply] [d/l] [select]
Re: Unexpected behavior with "map" by TomDLux (Vicar) on Mar 13, 2014 at 03:14 UTC
Having regexes like `s/\/etc\///` is referred to as Leaning Toothpick Syndrome, aka LTS. You already know you can use your preferred delimiter with `qw`; same thing with `s///`. YOu can even use delimiters that come in pairs, to differentiate the search part from the replace part: `s{/etc/}{}` [download] I generally like to move the search string and the replace components out of the regex operation, so the operation is just operating on predefined variables. `my $search_for = qr{/etc/}; my replace_with = qr{}; s{$search_for}{$replace_with}` [download] But in this case, you're dropping a single, identical prefix from all the paths. Perhaps that's simply a contrived example for the post. But in such a situation, I would use `substr()` to discard the prefix, which I already know the length of. While `basename()` is a more robust, reliable, portable solution, you could also use `rindex` to locate the rightmost '/' separator, and use substr to extract what comes after that. As Occam said: Entia non sunt multiplicanda praeter necessitatem.	[reply] [d/l] [select]
Re: Unexpected behavior with "map" by Clovis_Sangrail (Beadle) on Mar 12, 2014 at 21:06 UTC
Thanks for the advice! The /usr/bin/perl on this AIX box is 5.8 so /r does not work, but I have 5.16 in an alternate directory, using it and adding /r to my first listing above works. I was not thinking modules because these are exercises for learning, but I guess the two are not mutually exclusive.	[reply]


Your skill will accomplish what the force of many cannot
	PerlMonks