Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses
 
PerlMonks  

Re^3: looping efficiency (Benchmark Example)

by eyepopslikeamosquito (Archbishop)
on Dec 30, 2020 at 02:48 UTC ( [id://11125969]=note: print w/replies, xml ) Need Help??


in reply to Re^2: looping efficiency
in thread looping efficiency

Don’t Optimize Code -- Benchmark It

-- from Ten Essential Development Practices by Damian Conway

The real problem is that programmers have spent far too much time worrying about efficiency in the wrong places and at the wrong times; premature optimization is the root of all evil (or at least most of it) in programming

-- Donald Knuth

Without good design, good algorithms, and complete understanding of the program's operation, your carefully optimized code will amount to one of mankind's least fruitful creations -- a fast slow program.

-- Michael Abrash

Wow, I assumed they were complaining about clarity rather than efficiency! :) For cheap thrills, assuming all you need are the file names 0000 .. 9999 in order, I benchmarked the three offered solutions as shown below.

use strict; use warnings; use Benchmark qw(timethese); sub orig { for my $i ( 0 .. 9 ) { for my $j ( 0 .. 9 ) { for my $k ( 0 .. 9 ) { for my $l ( 0 .. 9 ) { my $filename = "$i$j$k$l"; } } } } } sub yourmum { for ( 0 .. 9999 ) { my $filename = sprintf "%04d", $_; } } sub marshall { for ('0000' ... '9999') { my $filename = $_; } } orig(); yourmum(); marshall(); timethese 50000, { Orig => sub { orig() }, YourMum => sub { yourmum() }, Marshall => sub { marshall() }, };

On my machine, the results were:

Benchmark: timing 50000 iterations of Marshall, Orig, YourMum... Marshall: 25 wallclock secs (25.16 usr + 0.00 sys = 25.16 CPU) @ 19 +87.52/s (n=50000) Orig: 39 wallclock secs (38.83 usr + 0.00 sys = 38.83 CPU) @ 12 +87.73/s (n=50000) YourMum: 40 wallclock secs (40.08 usr + 0.00 sys = 40.08 CPU) @ 12 +47.57/s (n=50000)

If all you need are the file names (not the individual digits), it's no surprise that Marshall's suggestion was the fastest. I also think it is the simplest and clearest if all you need are the file names.

Update: see also perlperf - Perl Performance and Optimization Techniques. Added this node to Performance References.

Replies are listed 'Best First'.
Re^4: looping efficiency
by LanX (Saint) on Dec 30, 2020 at 03:03 UTC
    None of the methods shown will have more than a negligible impact compared to creating/accessing those files.

    Harddisks are extremely slow compared to CPU and RAM.

    Cheers Rolf
    (addicted to the Perl Programming Language :)
    Wikisyntax for the Monastery

      Oh I totally agree! I was shocked they would argue about efficiency without first benchmarking the whole program and identifying that code as a chronic bottleneck! (I only did the benchmark because I was bored and curious on a slow Christmas/New Year holiday).

      On Coding Standards and Code Reviews summarises my opinion:

      • Correctness, simplicity and clarity come first. Avoid unnecessary cleverness. If you must rely on cleverness, encapsulate and comment it.
      • Don't optimize prematurely. Benchmark before you optimize. Comment why you are optimizing.

      For much more detail on this topic see:

Re^4: looping efficiency
by afoken (Chancellor) on Dec 31, 2020 at 13:45 UTC

    Don’t Optimize Code -- Benchmark It

    -- from Ten Essential Development Practices by Damian Conway

    The real problem is that programmers have spent far too much time worrying about efficiency in the wrong places and at the wrong times; premature optimization is the root of all evil (or at least most of it) in programming

    -- Donald Knuth

    No one has yet linked to Devel::NYTProf. It's a great tool to find out where your code really wastes CPU and time. Just run your script like this ...

    perl -d NYTProf yourscript.pl foo bar baz

    ... followed by ...

    nytprofhtml --open

    ... and look for the code that takes the longest time to run. That's where you want to start optimizing.

    Commit your current version to git/SVN/CVS/whatever, modify, run through NYTProf again until speed improves. Revert and retry if speed goes down. Commit, re-profile, opimize the next top problem from NYTProf. Stop when the code is sufficiently fast.

    Alexander

    --
    Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)

      Oh yes, Devel::NYTProf is a fantastic tool!

      Just for completeness, you can examine the OP codes generated for different snippets of Perl code via B::Terse. For example:

      > perl -MO=Terse -e "sprintf '%04d', $_" LISTOP (0x254e078) leave [1] OP (0x2753080) enter COP (0x254e0b8) nextstate LISTOP (0x254e118) sprintf [2] OP (0x254e158) pushmark SVOP (0x254e1f8) const [3] PV (0x254f200) "%04d" UNOP (0x254e188) null [14] PADOP (0x254e1c0) gvsv GV (0xd6af50) *_ -e syntax OK > perl -MO=Terse -e "qq{$i$j$k$l}" LISTOP (0x2806210) leave [1] OP (0x28061e0) enter COP (0x2806250) nextstate UNOP_AUX (0x28062b0) multiconcat [8] OP (0x28062f0) null [3] UNOP (0x664fe0) null [14] PADOP (0x665018) gvsv GV (0x65fbb0) *i UNOP (0x664f70) null [14] PADOP (0x664fa8) gvsv GV (0x6549e8) *j UNOP (0x664ec0) null [14] PADOP (0x664ef8) gvsv GV (0x65ff10) *k UNOP (0x2806360) null [14] PADOP (0x2806398) gvsv GV (0x654b68) *l -e syntax OK > perl -MO=Terse -e "$i.$j.$k.$l" LISTOP (0x27d23b0) leave [1] OP (0x27d2380) enter COP (0x27d23f0) nextstate UNOP_AUX (0x27d2450) multiconcat [7] UNOP (0x25d2d70) null [14] PADOP (0x25d2da8) gvsv GV (0x25c4d78) *i UNOP (0x25d2d00) null [14] PADOP (0x25d2d38) gvsv GV (0x25c49b8) *j UNOP (0x25d2c50) null [14] PADOP (0x25d2c88) gvsv GV (0x25d01b0) *k UNOP (0x27d2490) null [14] PADOP (0x27d24c8) gvsv GV (0x25c4e38) *l -e syntax OK

      No great surprise to see qq{$i$j$k$l} and $i.$j.$k.$l generating the same OP codes under the covers.

Re^4: looping efficiency
by Bod (Parson) on Dec 30, 2020 at 12:01 UTC
    For cheap thrills, assuming all you need are the file names 0000 .. 9999 in order, I benchmarked the three offered solutions as shown below.

    WoW!!!
    Isn't it amazing what random things can be learnt in the Monastery?

    I had no idea it is so easy to benchmark different ways of doing the same thing...although, thinking about it, it's pretty certain that someone would have thought about doing it and created a module to make it easy.

    That is one of the beauties of Perl - we are standing on the shoulders of giants (or at least, following the CPAN road).

Re^4: looping efficiency
by Anonymous Monk on Dec 30, 2020 at 03:51 UTC
    Thanks especially for that. I really appreciated the benchmarking (and seeing how to benchmark). Thanks to Marshall, too. I have taken on all of the comments about efficiency, and agree that the exact circumstances deserve most of the weight when considering various alternatives (but I also don't want to "learn" bad techniques in the first place. :) ).

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://11125969]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others pondering the Monastery: (7)
As of 2024-04-19 09:57 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found