Beefy Boxes and Bandwidth Generously Provided by pair Networks
We don't bite newbies here... much
 
PerlMonks  

perl 5.36 and the for_list feature - a simple speed comparison

by swl (Vicar)
on Jun 21, 2022 at 09:32 UTC ( #11144892=perlmeditation: print w/replies, xml ) Need Help??

Perl 5.36 has a new feature that iterates over multiple values at a time in a for loop (perldelta entry here). Any number of values can be specified, but the pairwise case is the focus here.

The for_list feature is a useful alternative to a while-each idiom over a hash when working with key-value pairs. It also allows one to natively work with arrays of key-value pairs without converting them to a hash (and thus losing values associated with duplicate keys).

Speed is one factor in the decision to updated code, so I figured I would see how fast the new feature is compared with while-each and some other approaches using List::Util functions. Some benchmark code is below (inside readmore tags), with results following.

Benchmark labels starting with a_ operate on array data while those with h_ operate on hash data.

I note that it is fairly well documented that the while-each idiom is best avoided because the hash iterator has global state and thus can cause action at a distance. In the benchmark code the hash is only accessed this way in one sub.

The summation of the hash values in the loops is merely something for the loop to do that would have relatively low overhead compared with the looping itself.

The hash_vals sub is done as a point of comparison. There is no reason why one would iterate over both keys and values when only the values are needed.

Code was run under Ubuntu 20.04 under WSL2 for Windows.

The main conclusion is that the new feature is faster than all the others when an array is used. It is faster than all of the hash approaches that use both keys and values. Profiling shows the cause of the slowdown when using List::Util::pairs is the dereference of the pair array, which is not surprising. Using the declared_refs feature does not help in this case.

Overall I quite like the new for_list feature. Aside from being faster than many of the alternatives, it is also much cleaner.

use 5.036; use strict; use warnings; use List::Util qw /pairs pairvalues/; use Benchmark qw /cmpthese/; use experimental qw/for_list declared_refs/; my @vals = (1..100000); my %val_hash = @vals; say lu_pairs(); say lu_pairs_refalias(); say lu_pairvals(); say for_list_a(); say for_list_h(); say while_each(); say hash_vals(); say hash_by_key(); cmpthese(-3, { a_pairs => \&lu_pairs, a_pairs_alias => \&lu_pairs_refalias, a_pair_vals => \&lu_pairvals, a_for_list => \&for_list_a, h_for_list => \&for_list_h, h_each => \&while_each, h_vals => \&hash_vals, h_by_key => \&hash_by_key, }); sub lu_pairs { my $i; for my $pair (pairs @vals) { $i += $pair->[1]; } return $i; } sub lu_pairs_refalias { my $i; for my \@pair (pairs @vals) { $i += $pair[1]; } return $i; } sub lu_pairvals { my $i; for my $val (pairvalues @vals) { $i += $val; } return $i; } sub for_list_a { my $i; for my ($key, $value) (@vals) { $i += $value; } return $i; } sub for_list_h { my $i; for my ($key, $value) (%val_hash) { $i += $value; } return $i; } sub while_each { my $i; while (my ($key, $value) = each %val_hash) { $i += $value; } return $i; } sub hash_vals { my $i; foreach my $value (values %val_hash) { $i += $value; } return $i; } sub hash_by_key { my $i; foreach my $key (keys %val_hash) { $i += $val_hash{$key}; } return $i; }

Results:

2500050000 2500050000 2500050000 2500050000 2500050000 2500050000 2500050000 2500050000 Rate a_pairs_alias a_pairs h_by_key h_each h_for_list +h_vals a_pair_vals a_for_list a_pairs_alias 87.9/s -- -9% -19% -35% -51% + -82% -83% -89% a_pairs 96.5/s 10% -- -11% -28% -47% + -80% -81% -88% h_by_key 109/s 24% 13% -- -19% -40% + -77% -78% -87% h_each 135/s 53% 40% 24% -- -25% + -72% -73% -84% h_for_list 181/s 106% 88% 66% 34% -- + -62% -64% -78% h_vals 477/s 443% 394% 338% 254% 164% + -- -6% -42% a_pair_vals 506/s 476% 424% 364% 275% 180% + 6% -- -39% a_for_list 827/s 841% 757% 659% 513% 357% + 73% 63% --

Replies are listed 'Best First'.
Re: perl 5.36 and the for_list feature - a simple speed comparison
by duelafn (Parson) on Jun 21, 2022 at 11:46 UTC

    List::Util::pairmap would be an existing solution very similar to the new feature. It also seems to run quite quickly (but I can't compare it to for_list since I don't have a new-enough perl).

    use List::Util qw/ pairmap /; sub a_pairmap { my $i; pairmap { $i += $b } @vals; return $i; }

    Good Day,
        Dean

      G'day Dean,

      "List::Util::pairmap would be an existing solution very similar to the new feature. It also seems to run quite quickly (but I can't compare it to for_list since I don't have a new-enough perl)."

      No problem. I have v5.36.0 installed. I added your code to the OP benchmark code.

      Mine starts as follows. Refer to "perl5360delta: use v5.36" for why I removed the strict and warnings pragmata.

      #!/usr/bin/env perl use v5.36; use List::Util qw /pairs pairvalues pairmap/;

      I then added:

      • say a_pairmap(); after the existing eight say statements.
      • a_pairmap     => \&a_pairmap, to the end of the cmpthese hashref.
      • Your sub a_pairmap { ... } verbatim, at the end of the script.

      I ran the benchmark five times. All results were pretty much the same; here's a median set:

      2500050000 2500050000 2500050000 2500050000 2500050000 2500050000 2500050000 2500050000 2500050000 Rate a_pairs_alias a_pairs h_each h_by_key h_for_list +h_vals a_pair_vals a_pairmap a_for_list a_pairs_alias 48.8/s -- -4% -67% -73% -83% + -90% -90% -92% -94% a_pairs 51.0/s 4% -- -66% -71% -82% + -90% -90% -91% -94% h_each 150/s 207% 193% -- -16% -47% + -69% -71% -74% -82% h_by_key 178/s 265% 249% 19% -- -37% + -64% -65% -69% -78% h_for_list 284/s 482% 457% 90% 59% -- + -42% -45% -51% -66% h_vals 490/s 903% 860% 227% 175% 72% + -- -5% -16% -41% a_pair_vals 513/s 951% 906% 243% 188% 81% + 5% -- -12% -38% a_pairmap 582/s 1091% 1041% 289% 226% 105% + 19% 13% -- -30% a_for_list 827/s 1594% 1521% 453% 364% 191% + 69% 61% 42% --

      In all runs, a_pairmap was second fastest; and, h_by_key was faster than h_each (which differs from the OP results).

      My environment:

      $ perl -v | head -2 | tail -1 This is perl 5, version 36, subversion 0 (v5.36.0) built for cygwin-th +read-multi $ uname -a CYGWIN_NT-10.0 titan 3.2.0(0.340/5/3) 2021-03-29 08:42 x86_64 Cygwin

      — Ken

        Thanks for this. Updated code and results are in 11144924.

      Thanks for that. List::Util::pairmap is a good suggestion.

      Below is some updated code to add your array code and a hash variant. The results of three runs follow.

      The conclusions wrt pairmap are pretty much the same as those of kcott in 11144922. A point of difference is that h_by_key remains slower in my case.

      I wonder if there are differences due to the compiler used to build the perls or there's something cygwin vs linux. My system details are at the end.

      Edit - the original code used say a_pairmap() twice. kcott++ for noting.

      Rate a_pairs_alias a_pairs h_by_key h_pairmap h_each h +_pair_vals h_for_list h_vals a_pair_vals a_pairmap a_for_list a_pairs_alias 83.5/s -- -7% -13% -33% -34% + -41% -53% -79% -83% -84% -90% a_pairs 89.5/s 7% -- -6% -28% -29% + -37% -50% -78% -82% -83% -89% h_by_key 95.5/s 14% 7% -- -23% -25% + -33% -46% -76% -81% -81% -88% h_pairmap 125/s 49% 39% 30% -- -2% + -12% -30% -69% -75% -76% -85% h_each 127/s 52% 41% 33% 2% -- + -11% -29% -68% -74% -75% -84% h_pair_vals 142/s 70% 59% 49% 14% 12% + -- -20% -65% -71% -72% -82% h_for_list 178/s 113% 99% 86% 43% 41% + 25% -- -56% -64% -65% -78% h_vals 401/s 380% 348% 320% 222% 217% + 182% 125% -- -18% -22% -50% a_pair_vals 490/s 487% 448% 413% 294% 287% + 245% 175% 22% -- -4% -39% a_pairmap 512/s 513% 472% 436% 311% 304% + 260% 188% 28% 4% -- -36% a_for_list 804/s 863% 799% 742% 546% 535% + 466% 352% 101% 64% 57% -- Rate a_pairs_alias a_pairs h_by_key h_each h_pair_vals + h_pairmap h_for_list h_vals a_pair_vals a_pairmap a_for_list a_pairs_alias 73.2/s -- -17% -25% -31% -49% + -49% -54% -82% -85% -86% -91% a_pairs 87.7/s 20% -- -10% -17% -39% + -39% -45% -79% -82% -83% -89% h_by_key 97.0/s 33% 11% -- -8% -32% + -32% -39% -77% -81% -82% -88% h_each 106/s 44% 21% 9% -- -26% + -26% -34% -75% -79% -80% -87% h_pair_vals 143/s 96% 63% 47% 35% -- + -0% -11% -66% -71% -73% -82% h_pairmap 143/s 96% 63% 48% 36% 0% + -- -11% -65% -71% -73% -82% h_for_list 160/s 119% 83% 65% 52% 12% + 12% -- -61% -68% -70% -80% h_vals 415/s 467% 373% 328% 292% 190% + 189% 159% -- -17% -22% -48% a_pair_vals 498/s 581% 468% 413% 371% 248% + 248% 211% 20% -- -6% -38% a_pairmap 531/s 626% 506% 448% 403% 271% + 271% 231% 28% 7% -- -33% a_for_list 797/s 990% 809% 722% 654% 457% + 456% 398% 92% 60% 50% -- Rate a_pairs_alias a_pairs h_by_key h_each h_pairmap h +_pair_vals h_for_list h_vals a_pair_vals a_pairmap a_for_list a_pairs_alias 75.9/s -- -18% -29% -41% -41% + -52% -58% -77% -83% -86% -89% a_pairs 92.4/s 22% -- -13% -28% -29% + -41% -49% -72% -80% -82% -87% h_by_key 107/s 40% 16% -- -17% -18% + -32% -41% -67% -77% -80% -85% h_each 128/s 69% 39% 20% -- -1% + -18% -29% -60% -72% -76% -82% h_pairmap 130/s 71% 40% 21% 1% -- + -17% -28% -60% -72% -75% -82% h_pair_vals 157/s 106% 70% 47% 22% 21% + -- -13% -52% -66% -70% -78% h_for_list 180/s 137% 95% 69% 40% 39% + 15% -- -45% -61% -66% -74% h_vals 325/s 328% 252% 204% 153% 151% + 107% 80% -- -29% -38% -54% a_pair_vals 459/s 504% 397% 330% 257% 254% + 193% 155% 41% -- -13% -35% a_pairmap 527/s 594% 470% 394% 311% 306% + 236% 192% 62% 15% -- -25% a_for_list 705/s 828% 663% 560% 449% 444% + 350% 291% 117% 54% 34% --
      perl -v | head -2 | tail -1 This is perl 5, version 36, subversion 0 (v5.36.0) built for x86_64-li +nux uname -a Linux L-933C0F3 5.10.102.1-microsoft-standard-WSL2 #1 SMP Wed Mar 2 00 +:30:59 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux gcc -v 2>&1 | tail -2 Thread model: posix gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.1)

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlmeditation [id://11144892]
Approved by Athanasius
Front-paged by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others studying the Monastery: (7)
As of 2023-02-08 11:36 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    I prefer not to run the latest version of Perl because:







    Results (42 votes). Check out past polls.

    Notices?