http://qs321.pair.com?node_id=977725

abcperl5 has asked for the wisdom of the Perl Monks concerning the following question:

Hi, are there any recommended modules or tools by which I can shrink the absolute byte size of my Perl script, while retaining all the original functionality? Things like:

* strip comments

* strip unnecessary whitespace

* rename $very_long_variable to $x1

My goal is solely to improve the execution time of my very large codebase which runs as CGI (reparsed with each request). I wrote my own utility that stripped comments (which take up about 30% of my files) and I saw a very noticeable improvement. I want to take it to the next level.

The obsfucators I've found so far are more directed towards protecting copyright and they use techniques that may slow everything down.

Replies are listed 'Best First'.
Re: suggested module/tools to automatically "tighten" my perl
by davido (Cardinal) on Jun 22, 2012 at 01:54 UTC

    The order of operations when it comes to speed optimization:

    1. Actually notice that there might be a performance issue.
    2. Profile to find where the issue is focused.
    3. Research the options with respect to algorithms that might improve upon the performance of the hotspots discovered in profiling.
    4. Benchmark the various algorithmic options.
    5. Implement the better algorithm.
    6. Test the implementation to ensure you haven't regressed in some way.
    7. If there's still an issue, repeat step 2.

    If there's no better algorithm, then it's time to trim cycles (paraphrasing Tom Duff).

    You mentioned CGI, and you are correct that many have discovered (probably through profiling) that the startup cost of CGI is high, particularly for short-running scripts (which CGI usually entails). This is one area where the profiling's been done again and again, there is a better algorithm, and the better algorithm is known. The better algorithm for that scenario is to eliminate the repeated start-ups. This can be done with something like mod-perl, or more modernly, through a persistent application framework that sits on top of a back-end like mod-perl, FastCGI, or some other persistent daemon. The modern frameworks such as Dancer, Catalyst, and Mojolicious attempt to do this (and much more).

    To summarize, if you have determined that startup time is your foe, the effectiveness of the micro-optimization of trimming cycles from startup pales in comparison to the algorithmic optimization of moving the startup cost out of the individual request cycle by creating a persistent process.


    Dave

      Thanks everyone for their suggestions; I'll look into it and report back.

      I would love to switch from CGI to a persistent setup. Everything runs on IIS on Windows 2003. Has anyone actually gotten FastCGI to work properly with Perl on Win2003/IIS? If so, I would love to hear it and get a pointer to any docs/downloads.

      I spent many hours investigating this, and it seemed that FastCGI didn't work on Win2003 (and I could never get it to work on Win2008 either). But I never posted to forum directly about it...

        Have you had a look at any of these (in particular the second link)?

        It seems like you've given this some thought -- done your due diligence. That's great. If the second link I provided helps, you should be able to, from there, get PSGI/Plack going, and that opens up a lot of possibilities for simplifying your life.


        Dave

Re: suggested module/tools to automatically "tighten" my perl
by ww (Archbishop) on Jun 21, 2012 at 23:30 UTC
    Noted that removing comments provided "noticeable improvement" but the other ways you mention in terms of tightening your code are UNLIKELY to provide similar gains.

    Before concerning yourself with whitespace and even $exceptionally-extraordinarily-long-and-verbose-nomenclature-for-each-and-every-possible-variable, try profiling your code. Some good reading? The Rules of Optimization Club and the other threads you'll find here with a Super Search for "nytprof" or "profile."

Re: suggested module/tools to automatically "tighten" my perl
by stevieb (Canon) on Jun 21, 2012 at 23:55 UTC

    This is only my personal opinion, so please take with a grain of salt. First, if 30% of your files are comments, you are doing something wrong. Unless you're just new to programming/Perl, the code should document itself, with only very brief comments sprinkled throughout to remind you and the next person who edits your code why you did something funky (or fancy, which you may not remember).

    Second, if you strip all whitespace, the code will become illegible. What happens if you need to edit something? Granted, you could keep the original and re-run your modification tool every time, but that is just going to be one extra step of potential breakage, and needless to say it is going to take more time as well.

    Third, I understand why you'd want to do this, but I'd advise you keep descriptive names for your variables. You'll thank yourself in the future.

    In closing, you might want to hand edit all the code. It may do two things: help you write better, more compact and readable code in the future, and help you fall into a more consistent, convenient writing style, as well as get you into a groove of writing shorter, but just as descriptive variable names (I'm not judging any of your code skills, because you didn't provide code ;)

    fwiw, I always thought perl stripped comments and eliminating whitespace by default, so are you talking about initial compile time that you want to minimize?

Re: suggested module/tools to automatically "tighten" my perl
by snoopy (Curate) on Jun 22, 2012 at 01:13 UTC
    Is lazy loading an option?

    I.E. can you replace global use and require statements with a lazy loader such as autouse or Class::Autouse.

    Alternatively, can you require modules just in time?

    # conditional use of require if ($heavy_lifting_needed) { require BigModule; ## BigModule->import(); # maybe BigModule->do_stuff(); }
    This could be of benefit:
    • if your code is organised into discrete modules
    • only some of them are being used in any particular hit
    • you're not importing symbols into your namespace
Re: suggested module/tools to automatically "tighten" my perl
by choroba (Cardinal) on Jun 21, 2012 at 23:10 UTC
    What about using perl -MO=Deparse to remove comments?

      Suggest Perl::Tidy using perltidy's -dac --mangle switches to remove all comments and pod.

Re: suggested module/tools to automatically "tighten" my perl
by chromatic (Archbishop) on Jun 22, 2012 at 02:40 UTC
    reparsed with each request

    Are you amenable to addressing this instead? Optimizing for compilation speed will likely require a lot of rewriting of code.

Re: suggested module/tools to automatically "tighten" my perl
by Anonymous Monk on Jun 22, 2012 at 03:24 UTC
Re: suggested module/tools to automatically "tighten" my perl
by Khen1950fx (Canon) on Jun 22, 2012 at 04:06 UTC
    Here's what I use to strip comments. Note: it also erases the shebang line.
    #!/usr/bin/perl use strict; use warnings; my $file = shift @ARGV; open STDOUT, '>', '/root/Desktop/squish.log'; use PPI; use Perl::Squish; my $Document = PPI::Document->new($file); $Document->prune( sub { my $Braces = $_[1]; $Braces->isa('PPI::Structure::List') or return ''; $Braces->children == 0 or return ''; my $Method = $Braces->sprevious_sibling or return ''; $Method->isa('PPI::Token::Word') or return ''; $Method->content !~ /:/ or return ''; my $Operator = $Method->sprevious_sibling or return ''; $Operator->isa('PPI::Token::Operator') or return ''; $Operator->content eq '->' or return ''; return 1; } ); my $Normal = $Document->normalized(2); print $Document;

      Here's what I use to strip comments. Note: it also erases the shebang line

      A copy/paste from Perl::Squish, clever

Re: suggested module/tools to automatically "tighten" my perl
by mr.nick (Chaplain) on Jun 22, 2012 at 20:02 UTC
    I don't actually recommend this, but the following produces runnable, tight(er) code:
    perl -MO=Deparse foo.pl| perl -nae 's/\n//g;s/\s+/ /g; print' > foo2. +pl
    At least it did on my test script... YMMV. But seriously, I don't recommend this. I just did it for shytes and grins.

    mr.nick ...