dextius has asked for the wisdom of the Perl Monks concerning the following question:
I'm doing some research on how I can optimize our Perl application. It's fairly large (100k lines). I've ran both dprof and NYTimes to try and fix some of the outliers, but we think that there are some things we're not seeing (based on the output).
If anyone can comment on the following, we'd greatly appreciate it.
First, some background: Our application is separated into many libraries (almost 1k) and within those libraries, many many methods (over 3k total).
1. We found this: http://www.perl.com/pub/a/2000/06/dougpatch.html ... (we realize this is very old, and things have probably changed, but I'd better ask)... We're worried that method call overhead isn't taken into account in any of our profiling runs. I couldn't find any resolution to this thread either. If anything, we'd gladly take a pragma to lock in inheritance to get the performance boost. We call a LOT of methods in our system, if we can make calling them faster, it would make a difference.
2. We're using "fields", since we desperately need compile time safety for attribute lookups, and I'm told that it's faster than using a hash directly, because they become array lookups. We're researching the move to Perl 5.10 as well, because of a purported speed boost as well as a decreased memory footprint. Lastly, we heard that there is a change to fields past 5.9, and 5.10 has a "new" way of doing this type of stuff altogether, but I haven't seen any benchmarks or documentation detailing the implementations and any rationale for using/not using them.
3. We use SWIG to talk to shared memory. Does anyone have any idea the difference in performance between hand rolling XS over SWIG for reading attributes off a C Structure? (or is there some even "better" way to do this kind of thing that I've never heard of?)
4. I read the section of perldoc concerning inline functions. I'm a little confused, you can't have an inline function if it has any arguments? I need to run some benchmarks of my own on this, but it's not something I see mentioned in any of my Perl books. (Is memoization of a method call even possible?)
Update:
The dprofpp -r produced some interesting results (1.5 hours of data collected).
Based on this, we definitely have some more work to do. Thanks to everyone with their insight on the issue.
If anyone can comment on the following, we'd greatly appreciate it.
First, some background: Our application is separated into many libraries (almost 1k) and within those libraries, many many methods (over 3k total).
1. We found this: http://www.perl.com/pub/a/2000/06/dougpatch.html ... (we realize this is very old, and things have probably changed, but I'd better ask)... We're worried that method call overhead isn't taken into account in any of our profiling runs. I couldn't find any resolution to this thread either. If anything, we'd gladly take a pragma to lock in inheritance to get the performance boost. We call a LOT of methods in our system, if we can make calling them faster, it would make a difference.
2. We're using "fields", since we desperately need compile time safety for attribute lookups, and I'm told that it's faster than using a hash directly, because they become array lookups. We're researching the move to Perl 5.10 as well, because of a purported speed boost as well as a decreased memory footprint. Lastly, we heard that there is a change to fields past 5.9, and 5.10 has a "new" way of doing this type of stuff altogether, but I haven't seen any benchmarks or documentation detailing the implementations and any rationale for using/not using them.
3. We use SWIG to talk to shared memory. Does anyone have any idea the difference in performance between hand rolling XS over SWIG for reading attributes off a C Structure? (or is there some even "better" way to do this kind of thing that I've never heard of?)
4. I read the section of perldoc concerning inline functions. I'm a little confused, you can't have an inline function if it has any arguments? I need to run some benchmarks of my own on this, but it's not something I see mentioned in any of my Perl books. (Is memoization of a method call even possible?)
Update:
The dprofpp -r produced some interesting results (1.5 hours of data collected).
- IO::Select was #1 with 62% of our cpu utilized (makes sense, we're an event based program that uses sockets to get information. It does bring up the question about whether EPOLL should be considered as an alternative).
- SWIG was #2 with roughly 3.5%. Ouch. I hope the XS really does make that big of an improvement
- fields::__ANON__ ate 1.3% of the time. I don't know if this is the constructor or the penalty we pay when accessing attributes (help?!)
Based on this, we definitely have some more work to do. Thanks to everyone with their insight on the issue.
|
---|
Replies are listed 'Best First'. | |
---|---|
Re: Optimizing a large project.
by salva (Canon) on Jun 12, 2008 at 17:45 UTC | |
by dextius (Monk) on Jun 12, 2008 at 17:57 UTC | |
by TGI (Parson) on Jun 12, 2008 at 18:32 UTC | |
Re: Optimizing a large project.
by perrin (Chancellor) on Jun 12, 2008 at 18:51 UTC | |
by dextius (Monk) on Jun 12, 2008 at 19:09 UTC | |
by perrin (Chancellor) on Jun 12, 2008 at 19:25 UTC | |
by dextius (Monk) on Jun 12, 2008 at 21:00 UTC | |
by perrin (Chancellor) on Jun 12, 2008 at 22:05 UTC | |
by salva (Canon) on Jun 13, 2008 at 08:15 UTC | |
by salva (Canon) on Jun 13, 2008 at 08:48 UTC | |
Re: Optimizing a large project.
by dragonchild (Archbishop) on Jun 12, 2008 at 19:44 UTC | |
by dextius (Monk) on Jun 12, 2008 at 21:05 UTC | |
by dragonchild (Archbishop) on Jun 12, 2008 at 21:24 UTC | |
by salva (Canon) on Jun 13, 2008 at 08:01 UTC | |
by dragonchild (Archbishop) on Jun 13, 2008 at 14:01 UTC | |
by dextius (Monk) on Jun 13, 2008 at 13:54 UTC | |
by mr_mischief (Monsignor) on Jun 17, 2008 at 20:14 UTC | |
Re: Optimizing a large project.
by Herkum (Parson) on Jun 12, 2008 at 17:46 UTC | |
by dextius (Monk) on Jun 12, 2008 at 17:56 UTC | |
by Herkum (Parson) on Jun 12, 2008 at 18:49 UTC | |
Re: Optimizing a large project.
by grinder (Bishop) on Jun 12, 2008 at 19:49 UTC | |
Re: Optimizing a large project.
by Zen (Deacon) on Jun 12, 2008 at 18:24 UTC | |
by dextius (Monk) on Jun 12, 2008 at 18:27 UTC | |
Re: Optimizing a large project.
by tsee (Curate) on Jun 13, 2008 at 14:14 UTC | |
Re: Optimizing a large project.
by BrowserUk (Patriarch) on Jun 12, 2008 at 22:24 UTC | |
by Gavin (Archbishop) on Jun 13, 2008 at 09:20 UTC | |
by dextius (Monk) on Jun 13, 2008 at 13:57 UTC | |
by BrowserUk (Patriarch) on Jun 13, 2008 at 21:35 UTC | |
by Gavin (Archbishop) on Jun 14, 2008 at 09:04 UTC |
Back to
Seekers of Perl Wisdom