How does variable definition affect Perl performance ?

EdsterTech has asked for the wisdom of the Perl Monks concerning the following question:

I've been asked to add a function to a huge script that currently takes an age to run.

Note that I haven't been asked to look at performance (and a lot of the blame is on the large amount of data that it has to trawl through), but the Christmas period is usually a bit quiet and I think that a bit of fiddling might improve it, and am looking for confirmation (or warnings to the contrary!).

Firstly, the script initialises about 100 variables once in a while using the following style of declaration:

$totalamount = 0;
$companyname = "";
[download]

When I look at it, it looks wrong - I want to change them to:

$totalamount = undef;
$companyname = undef;
[download]

or:

$totalamount = $companyname = undef;
[download]

Is this likely to improve performance, or does "pre-loading/maintaining" the variable type (integer/string) have some kind of use that I've been ignorant of for the past n years ?

Secondly, the script tests whether string variables are populated by using the length function, which seems a pointless way of slowing the script - e.g.

if ((length($companyname)>0) && ($anotherstring eq 'DOIT')) { ... }
[download]

This is what I would have done:

if (($companyname) && ($anotherstring eq 'DOIT')) { ... }
[download]

Any comments appreciated.
Ed

Comment on How does variable definition affect Perl performance ? Select or Download Code

Replies are listed 'Best First'.
Re: How does variable definition affect Perl performance ? by ikegami (Patriarch) on Dec 04, 2009 at 23:09 UTC
I've never had to assign undef to a variable. Don't you use `my`? Worry about writing clear code first. If it runs too slow, then worry about optimisation. Start from the other end though. Don't start messing with every tiny thing. Use a profiler to see which what would have a big effect.	[reply] [d/l]
Re: How does variable definition affect Perl performance ? by gmargo (Hermit) on Dec 05, 2009 at 01:26 UTC
I can't speak authoritatively to any performance difference regarding initialization. However my engineering sense says that it is insignificant. I can however speak from many years of experience writing code and maintaining code written by others. And I say this: If you start messing with the code to change things to "do it the way you would have done it", even if you make "obvious", "cannot be wrong" changes, you will probably screw it up, no matter how careful you are. Remember, you said it was a "huge script", and it presumably works now. Then you'll be explaining to the boss why you are debugging code in areas you were not tasked to modify. And then you will ultimately toss your changes and go back and just make the required changes you were supposed to make. I can also state without fear of contradiction that you will hardly ever pick up a piece of code and not see something you would do differently. Even if it is code you wrote yourself only a few months ago. Now consider the example you provided, where you want to change `$companyname = "";` to `$companyname = undef;`. Now you have broken all the code that uses length(), so you have to fix that. But even your proposed fix is broken if `$companyname = "0";`, so now you have to test for `defined($companyname) && length($companyname)` which is longer than the original. So your "simple, obvious" change caused problems which have to be tested for and debugged. Bottom line: If it ain't broke, don't fix it. (Unless the current code doesn't use strict; then fix it.)	[reply] [d/l] [select]
Re: How does variable definition affect Perl performance ? by bv (Friar) on Dec 05, 2009 at 00:21 UTC
First let me say I totally agree with ikegami above regarding profiling. Also, check out When perl is not quite fast enough, a really good article. But don't change the variable declarations blindly! Some code might depend on a non-undef, false value like 0 or "" `@_=qw; Just another Perl hacker,; ;$_=q=print "@_"= and eval;` [download]	[reply] [d/l]
Re: How does variable definition affect Perl performance ? by Joost (Canon) on Dec 05, 2009 at 04:14 UTC
Short answer: no and no. First question: initializing with $var = undef. Absolutely first: why aren't you using my? Secondly: 0, "" and undef are distinct values that can interact differently with the rest of the code, Don't assume that you can use them interchangeably. Second question: length($string) == 0 vs if ($string): this is a good example of the difference mentioned above: 0, "0", undef and "" are all logically false in perl, but "0" and 0 are of length 1 while undef and "" are of length 0. As for performance; this kind of nitpicking is unlikely to give you a speedup even a couple of orders of magnitudes smaller than the time you need to spend to make the change and test that the code will still work correctly, even if your code is running continously for a year or two. Look at the overall algorithm and its performance characteristics and improve that instead. For instance, minimize disk I/O in favor of using more memory if you can: for any file that's read more than once you're probably better off reading it into an array or hash table and keeping it there. Even if you can't do that right now because of memory limits, it may be more efficient in terms of money and time to add a couple of Gb of RAM to the machine instead of messing about with the code. RAM is cheap and it'll only get cheaper, and so is CPU power; a couple of days worth of development time will buy you a top-of-the-line new machine. An hour or two development time will buy you a couple of Gb of RAM. "What should it profit a man, if he should win a flame war, yet lose his cool?"	[reply]
Re: How does variable definition affect Perl performance ? by cdarke (Prior) on Dec 05, 2009 at 16:34 UTC
If you want to improve your code then run Perl::Critic on it - you will get plenty of sugggestions. If you really want to know the effect of operations on variables then consider using Devel::Peek. But, as others have said, there are probably better things you should be spending your time on over the Christmas period. How about improving the testing for your modules? You do using Test::More, don't you? What about updating all that documentation you never got around to? Did you ever get around to learning POD?	[reply]
Re^2: How does variable definition affect Perl performance ? by EdsterTech (Initiate) on Dec 05, 2009 at 21:07 UTC
Documentation ?!?? What's that ? Seriously, you may have a point, and judging from others' comments, once you get a script of this size there are as many ways to write it as there are Perl authors. Thanks for all the responses - I'll play it safe and only do what I've been asked to do... ...although I may give Perl::Critic a whirl...	[reply]
Re: How does variable definition affect Perl performance ? by wazoox (Prior) on Dec 05, 2009 at 11:05 UTC
Aren't these variables global? As others said, initializing an empty variable should hardly matter; however setting to 0 a variable containing huge data can maybe take time.	[reply]
Re^2: How does variable definition affect Perl performance ? by ikegami (Patriarch) on Dec 05, 2009 at 17:15 UTC
setting to 0 a variable containing huge data can maybe take time. No. Why would assigning to a variable have to traverse the contained string? The length of the string contained doesn't matter. It's not even freed (which still wouldn't effect the time taken for long strings vs small strings). Or maybe you are talking about a scalar that contains the last reference to an object or a large structure? Destroying the structure can take time (e.g. if destructors need to be called), but I don't see the relevance. Destroying the structure will have to occur sooner or later. It's not going to speed up the OP's program by delaying when it occurs.	[reply]


No such thing as a small change
	PerlMonks