http://qs321.pair.com?node_id=1119628

builat has asked for the wisdom of the Perl Monks concerning the following question:

Good day\night collective mind.
Somehow I ask myself question. What method faster for large vector data.
All my life I believe that "map" faster. But..
So here results.
This is perl 5, version 20, subversion 2 (v5.20.2) built for x86_64-linux-gnu-thread-multi

intel I3, 2024 mb mem.
-------------------------------------------------
my @a; $a[$_] = int(rand (100)) for(0..10000000);

-------------------------------------------------

foreach(@a){$_+=1;};

real 0m3.687s
user 0m3.556s
sys 0m0.120s

--------------------------------------------------

@a = map{$_+= 1} @a;

real 0m7.719s
user 0m7.104s
sys 0m0.556s


So question. In what kind of situations map function is more applicable then for\foreach\while?

Replies are listed 'Best First'.
Re: map vs for\foreach. (perf)
by tye (Sage) on Mar 11, 2015 at 15:49 UTC

    Picking your syntax based on benchmarks of trivial operations is total folly.

    Use map if you want to get back a list. Don't otherwise.

    Now, there can be non-trivial performance implications in similar syntax choices between for() and while() when reading from a (large) file, for example. But you also likely would not discover those from running benchmarks of trivial operations.

    The benchmark results presented so far mostly illustrate differences in the trivial operations being done, not any performance difference that is likely to actually matter if you were changing useful code between map/for() [for cases of code where for() made sense].

    - tye        

      Here's some trivial benchmarks that at least compare comparable operations and ones that are just slightly more than trivial:

      #!/usr/bin/perl -w use strict; use Benchmark 'cmpthese'; my @a = 1..1000; sub p { my( $a ) = @_; if( $a & 1 ) { return $a*$a; } else { return ($a-1)*($a+1); } } cmpthese( -2, { for => sub { my $t = 0; for( @a ) { $t += p($_) } return $t }, map => sub { my $t = 0; map $t += p($_), @a; return $t }, } ); cmpthese( -2, { map => sub { my @b = map p($_), @a; return $b[-1] }, for => sub { my @b; for( @a ) { push @b, p($_) } return $b[-1] }, } );

      And here are my results:

      Rate for map for 2101/s -- -1% map 2127/s 1% -- Rate map for map 1981/s -- -0% for 1987/s 0% --

      So I don't even have to give my explanation of why Benchmark.pm saying "20% faster" almost never has any practical meaning.

      - tye        

Re: map vs for\foreach.
by MidLifeXis (Monsignor) on Mar 11, 2015 at 14:27 UTC

    I strictly reserve map for coercion of a set of FOO items into a set of BAR items (@lista -> map -> @listb), for/foreach for iteration (do_something for @list), and while for terminating on a condition (do_something while sometest()). I don't use map unless I am consistently transforming ALL (typically) of the original items to the new set, although I may combine it with a leading grep if I only want a subset.

    --MidLifeXis

      "I strictly reserve map for coercion of a set of FOO items into a set of BAR items (@lista -> map -> @listb)..."

      And i don't understand why you insist on this.

      Just because there must be an example:

      #!/usr/bin/env perl use strict; use warnings; use Data::Dump; my @foo = ( 1 .. 10 ); dd \@foo; my @bar = map { ++$_ } @foo; dd \@foo; dd \@bar;

      So why is map { ++$_ } @foo; wrong?

      I vaguely remember that there where some performance issues...patched?

      Best regards, Karl

      «The Crux of the Biscuit is the Apostrophe»

        It isn't wrong, it is just not saying what you mean. map is a transformation. for is a loop. If I use map, it is understood that there are to be no modifications to the original data set (and conversely, if there are side effects, I did it wrong). If I use for, then there is the potential for modification of the original data.

        It is, for me, about clarity of intent. That is why I insist (in my code) on this usage.

        --MidLifeXis

        So why is map { ++$_ } @foo; wrong?

        If for no other reasons: it’s a little harder to follow and four characters more typing than $_++ for @foo. Superstition makes me post-increment; I think it’s not slower anymore but was back in the dinosaur age. (Update: map $_++, @foo is a little shorter too…)

      Is it good practice to mix map with lambda functions? like:
      map{ sub{...} } @list;

        That is essentially what a map is - although your syntax is wrong. Unless you do something like: map { sub {...}->(...) } @foo to force the execution of the sub, you will simply get a set of sub {...} anonymous subroutines the same size as @foo.

        The contents of the BLOCK in the map call are what I think you are thinking of as the sub. OTOH, something like:

        @subs = map { my $something = $_; sub { blah( $something, ... ); } } @data;

        would give you a set of subroutines that each operate on a specific item in @data. In this case, you are still transforming @data into a set of subs working on a closures.

        --MidLifeXis

Re: map vs for\foreach.
by sn1987a (Deacon) on Mar 11, 2015 at 14:33 UTC

    You are not comparing apples to apples. You map loop assigns (copies) the entire array in addition to incrementing each element. If you drop the assignment, map beats foreach.

    My results

    foreach(@a){$_+=1;}; real 0m4.941s user 0m1.096s sys 0m3.604s
    -----------------------
    @a = map{$_+= 1} @a; real 0m6.248s user 0m2.652s sys 0m2.900s
    ------------------------
    map{$_+= 1} @a; real 0m3.224s user 0m2.908s sys 0m0.232s

    Note: It is generally considered poor form to use map only for its side effects.

    Updated to clean up and add note

    Update 2:

    More rigorous timing show that foreach is indeed faster --> Re^3: map vs for\foreach.

      In my case:
      foreach(@a){$_+=1;}; real 0m3.756s user 0m3.588s sys 0m0.120s

      map{$_+= 1} @a; real 0m5.233s user 0m5.056s sys 0m0.164s

        My previous timings where just from a single run of each program. Using Benchmark and repeating to verify consistent results, I find that foreach is indeed faster.

        use strict; use warnings; use Benchmark; my @a; $a[$_] = int( rand(100) ) for ( 0 .. 10000000 ); cmpthese( -30, { 'foreach' => sub { foreach (@a) { $_ += 1; } }, 'map (assign)' => sub { @a = map { $_ += 1 } @a; }, 'map (bare)' => sub { map { $_ += 1 } @a; }, } );

        Results:

        s/iter map (assign) map (bare) foreach map (assign) 2.06 -- -40% -76% map (bare) 1.23 67% -- -59% foreach 0.501 311% 146% --

Re: map vs for\foreach.
by LanX (Saint) on Mar 11, 2015 at 14:22 UTC
    update

    sorry I misunderstood your question, looked like you are comparing $a[$_] = int(rand (100)) for(0..10000000);

    /update

    for(0..10000000) iterates dynamically but map{...}@a flattens the list which results in shuffling memory...

    Better try @a in both cases and the results should level.

    Cheers Rolf
    (addicted to the Perl Programming Language and ☆☆☆☆ :)

    PS: Je suis Charlie!

      You right. But..
      foreach(@a){$_+=1;}; real 0m3.756s user 0m3.588s sys 0m0.120s

      map{$_+= 1} @a; real 0m5.233s user 0m5.056s sys 0m0.164s
        Sorry I misunderstood your question (see update in meantime), but it can still be the same effect that for tries to be clever and iterates @a, while map is flattening.

        Output with perl -MO=Terse seems to indicate this...

        Cheers Rolf
        (addicted to the Perl Programming Language and ☆☆☆☆ :)

        PS: Je suis Charlie!

Re: map vs for\foreach.
by eyepopslikeamosquito (Archbishop) on Mar 12, 2015 at 01:44 UTC

    In what kind of situations map function is more applicable then for\foreach\while?
    Effective Perl Programming item 20, "Use foreach, map and grep as appropriate", provides a nice summary of when to use foreach, map and grep:
    • Use foreach to iterate read-only over each element of a list
    • Use map to create a list based on the contents of another list
    • Use foreach to modify elements of a list
    • Use grep to select elements in a list

Re: map vs for\foreach.
by AppleFritter (Vicar) on Mar 11, 2015 at 19:56 UTC

    If you'll forgive some general remarks only tangentially related to your question -- to add to what tye said, they say that premature optimization is the root of all evil. I think it's generally better to write a program in the way that most clearly and naturally expresses what you want it to do, that's most maintainable and so on; optimize only when it's necessary and worthwhile, and then profile to see where the hotspots are and focus on those, not on what looks most ripe for optimization. And always: measure, measure, measure.

    Disclaimer: I'm not a programmer or in fact any kind of IT pro, I just occasionally use Perl because it helps me solve problems.

Re: map vs for\foreach.
by stevieb (Canon) on Mar 11, 2015 at 14:22 UTC

    map() is a very powerful transformation tool. If I see map() in code, I know the intent of the code is to do something with each element of the structure passed in. That's not always as easy to spot in a for() loop.

    Besides, most map() work is contained on a single line, therefore making the code easier to understand at a glance.

    Just my $.02.

    -Steve

Re: map vs for\foreach.
by Anonymous Monk on Mar 12, 2015 at 01:36 UTC
    Don't "diddle" code to make it faster: find a better algorithm."

    -- Kernighan & Plauger, The Elements of Programming Style.
Re: map vs for\foreach.
by crusty_collins (Friar) on Mar 11, 2015 at 14:11 UTC
    I read somewhere that map is just a for loop anyway.
      If so, why it shows different result with foreach?