Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

Perl Optimization

by bikeNomad (Priest)
on Jun 01, 2001 at 08:25 UTC ( [id://84842]=perlmeditation: print w/replies, xml ) Need Help??

I'd like to write a Meditation on Perl optimization and profiling. I don't see too much discussion here about it, and I feel it's something that might be of interest.

Given that I can edit this node later, I figure I'll first ask for some sample code to use as an example. I'd like someone to point me to or send me a Perl application that:

  • is something they wrote
  • is not a CGI or database app
  • has a moderate number of subroutines, rather than one big blob of code in main
  • has not yet been optimized
  • runs slowly enough to actually time
  • is not destructive to data
  • does not require more than 1Mb of disk space to run
My strategy will be to optimize the code step by step, detailing the process and code changes, and showing the resultant speed increase (or memory reduction)

Thanks!

Replies are listed 'Best First'.
Re: Perl Optimization
by gumpu (Friar) on Jun 01, 2001 at 14:52 UTC

    I can give you an example program that analyses C++ code. It has as input a tags file and a bunch of C++ code. And it generates a bunch of html files showing the class hierachy and the complexity off all classes. It does this in three steps:

    1. It parses the tagsfile to figure out where to find all the classes and operations in the code, and to build the class tree. This uses a fair amount of regular expressions and a couple of functions that recursively walks through a hashes and arrays.
    2. It scans the c++ source files to figure out how many lines of code there are per operation and class. The results are later used to sort the classes according to complexity.
    3. It transforms the results into html files.

    The code has not been optimized, and I think there probably plenty of stuff to optimize. It has about 20 subs. It was written when I just started programming in Perl, so it probably can also be used to show how to improve style and programming techniques.

    You would need to have a fair amount of C++ code to analyse and access to a ctags program. In addition, the C++ code should not use any namespaces.

    Have Fun

Re: Perl Optimization
by damian1301 (Curate) on Jun 02, 2001 at 07:59 UTC
    Just some slight, stupid optimizations to save the slightest bit of space.

    • substr before regex (if possible). transliteration before regex. Use regex as last resort.
    • use single quotes if no interpolation is needed (REALLY minor optimization).
    •  $foo ? $bar; is faster than $bar if $foo; which is faster than if($foo){$bar}.
    • require is always faster than use.
    • use Memoize if that slow function that is called a couple times needs speeding up.


    Tiptoeing up to a Perl hacker.
    Dave AKA damian

Re: Perl Optimization
by DrSax (Sexton) on Jun 01, 2001 at 17:34 UTC
    I will be very interested in the outcome of such a study. We are in the process right at this moment profiling some data loading code that takes about 1/2 hour to run on some very beefy hardware (Sun Sparc 4500, 2 gig memory, etc...).

    One of the things we are looking at is de-referencing. Assume you have a hash reference that is a complex data structure:
    my $href = { 'x' => { 'y' => { 'z' => [ $xref, $yref, $zref ], }, 'strange' => { ..... }, }, 'charm' => { .... }, };
    Now imagine that the example above is actually a reference to an enormous object, with grand-children, great-grand-children, and great-great-great-great-grandchildren. From what I can see, every time you de-reference some or all of the object, a copy is made.
    foreach my $obj (%{$href}) # This dereference ^ causes a deep copy to be made
    In the case where these are really big objects, as some of ours are, that would be rather rough, especially if we iterate through children, then grand-children, then deeper still.

    At this point, the savings gained by passing a reference to a function is lost due to the fact that a copy of the referent is made in the function. For instance:
    01: my $list = [1, 2, 3]; 02: doStuff($list); 03: 04: sub doStuff 05: { 06: my $listRef = shift; 07: foreach $listElement (@{$listRef}) 08: { 09: print "$listElement \n"; 10: } 11: }
    In line 7, a deep copy of the list is made. If we change the sub to take not a reference, but the whole list, is this equivalent, or is it different?
    Of course, if in the subroutine you de-reference the thing more than once, you could be incurring huge overhead. This is not at all an issue with a puny example like the above, but we have rather hierarchical data, with objects that contain child objects, with children, etc. All method calls are by reference, and most references are de-referenced all over the place.

    I can't send you our code (proprietary, you know) but I might be able to put together a better example to show you what I mean. Right now, one of our guys is looking in to this. I will publish the results of our study here.

    Brian - a.k.a. DrSax
      Copies are not being made where you think they are.
      foreach my $obj (%{$href})
      # This dereference ^ causes a deep copy to be made

      actually, this is more accurate:
      foreach my $obj (%{$href}) #here ^ is the copy
      If you check the ref()-iness of $obj, you'll find that there isn't any at all. The appropriate value is copied into $obj on each run of the loop. I'm not at all sure what it is you're trying to solve with this, but if you need access to the key/value pair without the big, horrendous copy, maybe something like this:
      foreach my $key (keys %{$href}) { #do what you need with $key and $href->{$key} }
      Hope you find this useful!

      ~Cybercosis

      nemo accipere quod non merere

        for and foreach make the argument an alias to (not a copy of) the values in the list.

        Even if you copy a reference, you still get a reference, so I don't understand what you are trying to say.

        my $href = { 'x' => { 'y' => { 'z' => [ $xref, $yref, $zref ], }, #'strange' => { ..... }, }, #'charm' => { .... }, }; for my $obj ( %$href ) { print "$obj: ",ref($obj),$/; my $copy= $obj; print "$copy: ",ref($copy),$/; } __END__ prints: x: x: HASH(0x1a65030): HASH HASH(0x1a65030): HASH

                - tye (but my friends call me "Tye")

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlmeditation [id://84842]
Approved by root
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others having an uproarious good time at the Monastery: (6)
As of 2024-04-19 12:45 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found