comment on

I've spent the last few weeks optimizing and reoptimizing and re(n)optimizing a math module (Math::Combinator) I've been working on (to be released here soon), and had to wonder exactly how fast certain thing were, and how fast they were compared to each other. These things are the fundamental components of most programming languages: arrays, hashes, subroutines (by ref), sub's (by value). So, duh, Benchmark.

use Benchmark;

$a = shift;
$b = shift;
print "A = $a\tB = $b\n";
$hash{$_}  = $_ for (1..$b);
@array[$_] = $_ for (1..$b);

sub doSubR { ${$_[0]} = ${$_[0]}}
sub doSubV { return $_[0]       }

timethese ( $a, {
  '1 Nothing  ' => 'for (1..$b) { $trash = $_               };',
  '2 Mult     ' => 'for (1..$b) { $trash = $_ * $_          };',
  '3 Array    ' => 'for (1..$b) { $trash = $array[$_]       };',
  '4 Hash     ' => 'for (1..$b) { $trash = $hash{"$_"}      };',
  '5 Sub (Ref)' => 'for (1..$b) { doSubR(\$_); $trash = $_  };',
  '6 Sub (Val)' => 'for (1..$b) { $trash = doSubV($_)       };',
});
[download]

For A = 1,000 and B = 10,000 I get these results, which I find fairly satisfying for accuracy:

[lexicon]$ perl bench.pl 10 1000000
A = 10  B = 1000000
Benchmark: timing 10 iterations of 1 Nothing  , 2 Square   , 3 Array  
+  , 4 Hash     , 5 Sub (Ref), 6 Sub (Val)...
1 Nothing  : 35 wallclock secs (29.99 usr +  0.10 sys = 30.09 CPU)
2 Square   : 58 wallclock secs (48.79 usr +  0.23 sys = 49.02 CPU)
3 Array    : 63 wallclock secs (54.49 usr +  0.20 sys = 54.69 CPU)
4 Hash     : 184 wallclock secs (156.99 usr +  0.60 sys = 157.59 CPU)
5 Sub (Ref): 388 wallclock secs (327.87 usr +  1.30 sys = 329.17 CPU)
6 Sub (Val): 312 wallclock secs (266.54 usr +  1.02 sys = 267.56 CPU)
[download]

Here's my interpretation of things (the Relative column translates to the number of multiplication operations that could be performed in the time it takes to perform the Function):

Function  Time(S)   Relative
Mult:     19        01.00
Array:    24        01.25
Hash:     127       06.68
Sub (R):  299       15.70
Sub (V):  237       12.47
[download]

Question #1: Interpretation? My instinct is to subtract Nothing from the rest and use that as the actual base, thereby ignoring the overhead of Benchmark and the for loops.

Question #2: Those for loops: I want to get rid of cacheing in the hash and arrays. Is that a valid concern? Are the for loops a valid solution?

Question #3: Pass by reference is slower than Pass by value. Is that to be expected in this case? I thought bouncing things off the stack was supposed to be slow.

Question #4: What have I forgotten that completely invalidates my whole process?

-Lexicon

In reply to Fundamental Benchmarks by Lexicon

Are you posting in the right place? Check out Where do I post X? to know for sure.
Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
<code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
Want more info? How to link or How to display code and escape characters are good places to start.


go ahead... be a heretic
	PerlMonks