Cheers,
KM

Chapter 8 of the second ed. Camel has a bunch of things (and on different kinds of efficiency). Other n' that I don't know of a central repository. Perhaps an addition to the Tutorial page is in order ...

Philosophy can be made out of anything. Or less -- Jerry A. Fodor

What I would advise if you want to get your program running faster, is to use Devel::OpProf. This ties right into the perlfaq 3 question, "How do I make my Perl program run faster?" The advice given is to improve your algorithm. Okay, but how to do that? One of the best ways to compare different ways of doing things is to look at the resources they take up. Benchmark.pm will tell you how fast a code fragment runs, but that information isn't very portable. On a machine with lots of RAM, you might be developing programs that are very fast, but will generally lag on other people's machines. Devel::OpProf lets you identify these bottlenecks, even when they aren't slowing you down.

As an example, lets consider that we want to make a list of 1000 elements, with each element set to 1.

#!/usr/bin/perl -w
use strict;
use Devel::OpProf qw( profile print_stats zero_stats );
use Benchmark qw( timethese );

profile(1);

print "*** one() ***\n";
my @one = one();
print_stats();
zero_stats();

print "\n*** two() ***\n";
my @two = two();
print_stats();
zero_stats();

print "\n*** three() ***\n";
my @three = three();
print_stats();
zero_stats();

print "\n*** four() ***\n";
my @four = four();
print_stats();
zero_stats();

timethese( 0, {
    test_one   => 'one()',
    test_two   => 'two()',
    test_three => 'three()',
    test_four  => 'four()'
    } );

sub one {
    my @list;
    for ( my $i = 0; $i < 1000; $i++ ) {
    $list[$i] = 1;
    }
    return @list;
}

sub two {
    my @list;
    for ( 0..999 ) {
    $list[$_] = 1;
    }
    return @list;
}

sub three {
    my @list = map { 1 } (1..1000);
    return @list;
}

sub four {
    my @list;
    @list[0..999] = (1) x 1000;
    return @list;
}
[download]

*** one() ***
private variable         3002
constant item            2003
next statement           1009
private array            1004
numeric lt (<)           1001
logical and (&&)         1001
scalar assignment        1001
preincrement (++)        1000
iteration finalizer      1000
array element            1000
pushmark                 8
glob value               4
subroutine entry         3
conditional expression   1
list assignment          1
block entry              1
array dereference        1
loop entry               1
loop exit                1
return                   1
print                    1

*** two() ***
next statement           2009
private array            1004
constant item            1003
foreach loop iterator    1001
logical and (&&)         1001
array element            1000
scalar variable          1000
iteration finalizer      1000
scalar assignment        1000
pushmark                 9
glob value               5
subroutine entry         3
block entry              1
list assignment          1
conditional expression   1
array dereference        1
foreach loop entry       1
loop exit                1
return                   1
print                    1

*** three() ***
null operation           1002
constant item            1002
map iterator             1000
block                    1000
pushmark                 11
next statement           8
private array            4
glob value               4
subroutine entry         3
array dereference        2
list assignment          2
map                      1
block entry              1
conditional expression   1
return                   1
print                    1

*** four() ***
pushmark                 12
next statement           9
private array            5
glob value               4
constant item            4
subroutine entry         3
array dereference        2
list assignment          2
repeat (x)               1
conditional expression   1
null operation           1
array slice              1
block entry              1
return                   1
print                    1
Benchmark: running test_four, test_one, test_three, test_two, each for at least 3 CPU seconds...
  test_one:  3 wallclock secs ( 3.16 usr +  0.00 sys =  3.16 CPU) @ 274.68/s (n=868)
  test_two:  3 wallclock secs ( 3.25 usr +  0.00 sys =  3.25 CPU) @ 359.08/s (n=1167)
test_three:  3 wallclock secs ( 3.01 usr +  0.00 sys =  3.01 CPU) @ 408.64/s (n=1230)
 test_four:  3 wallclock secs ( 3.27 usr +  0.01 sys =  3.28 CPU) @ 501.83/s (n=1646)

So, while Benchmark could tell us which is faster, it doesn't really tell us why. Devel::OpProf shows us that it is because of the number of temporary variables, &&'s, assignments, etc.

(while not really relevant to the point of this post, I should point out that probably a better idea than the slice in #4 is to just my @list = (1) x 1000;, because it is more readable. But, you are welcome to profile them to see if there is an algorithmical difference... ;)

Paris Sinclair    |    4a75737420416e6f74686572
pariss@efn.org    |    205065726c204861636b6572
http://sinclairinternetwork.com

I was a little more interested in finding documentation (perldoc-style if nothing else), or a niche waiting to be filled by some useful documentation, on general optimization tips (e.g. use !/\S/ instead of /^\s*$/ and why). This information might still be useful to others though. Thanks.

Here's one rule you might like: It is almost always a mistake to use the non-greedy regex quantifiers like *? and +?.

Typical code looks like this:

    $churchill = qq{"If I were your husband," he said, "I should drink
+ it."};

    while ($churchill =~ /"(.*?)"/g) {
      print $1, "\n";
    }
[download]

The purpose of the (.*?) is to capture a quoted part of the string in $churchill. Beginners usually try (.*) which doesn't work, because it captures everything from the first quotation mark to the last, including the non-quoted part in between. So then they ask how to fix this and are advised to use (.*?) instead. This does work, but it's much slower than it needs to be. A faster solution is:

    while ($churchill =~ /"([^"]*)"/g) {
      print $1, "\n";
    }
[download]

This says that what you're interested in is a quote character, followed by a sequence of characters that are not quotes ([^"]) followed by another quote. The description of what you want is more exact, and it enables Perl to do the matching more efficiently.

So a good rule of thumb is to avoid .*? wherever possible and to use something like [^"]* instead when you can.

Oh, sorry. Maybe this will be more to your liking then:

perldoc Devel::OpProf

Paris Sinclair    |    4a75737420416e6f74686572
pariss@efn.org    |    205065726c204861636b6572
http://sinclairinternetwork.com

How would your book differ from MJD's Perl Advanced Techniques Handbook?

I don't think that "a compendium of elegant Perl idioms" will be a good description of the Perl Advanced Techniques Handbook. When people talk about 'idioms' they usually mean one- or two-line snippets like the one fastolfe mentioned that pre-extends a hash. But PATH isn't going to be about that at all.

The idea of PATH is that there are a lot of powerful programming techniques that are possible in Perl but not in other languages that Perl programmers are familiar with. Since Perl programmers haven't seen these techniques before, they don't know how to use them, what they are good for, or even that they exist, and they are letting a lot of the power and expressiveness of Perl go to waste. The techniques are not little things like pre-extending a hash. They are much bigger ideas that apply to the organization of entire programs, ideas on the scale of 'object-oriented programming'.

One example is the idea that in Perl you can write a function that manufactures other functions. Instead of writing a lot of similar functions in the source code, you instead write one function which, when called, generates the function that you actually want to use and returns it. By invoking this 'function factory' with different arguments, your program can manufacture as many functions as it needs to without your having to guess in advance what all the functions will need to do.

Perl's own sort operator is a limited example of this. If sort only sorted lists alphabetically, it would only bone ten-thousandth as useful as it is. But instead, it gets an argument, supplied by the programmer, which tells it how to compare two list elements. In effect, this extra argument transforms sort into a different function, which sorts lists in the way that the programmer specified; if you give it a different argument, you get a different kind of sorting function back out.

This is tremendously useful in the case of sort, but most Perl programmers don't realize that they can apply the same techniques to their own functions in many similar circumstances and make their own functions much more useful and general than they would have been otherwise.

Anyway, if you're interested, the correct URL is http://perl.plover.com/book/.

Mine wouldn't be a link to PerlMonks.org. ;)

Maybe it doesn't. I don't know -- I've never heard of this handbook of which you speak. I have heard a couple of his talks on the matter. So if he's making a compedium of them, that would be super.

$_="goto+F.print+chop;\n=yhpaj";F1:eval

Also, are you going to wait until Perl 6 to write this book?