Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw
 
PerlMonks  

Re: Managing multi-key hash tables....

by ivancho (Hermit)
on Apr 28, 2008 at 18:57 UTC ( [id://683348]=note: print w/replies, xml ) Need Help??


in reply to Managing multi-key hash tables....

Hash of hashes on 1.8 million entries might be a little heavy on the memory..
Additionally, you hit yourself for 50x the runtime, since you iterate through all the hash keys for every NetFanout in which you are interested.
What you need to ask yourself here is whether the hash table is required at all.. A hash table gives you fast random access to entries - you don't use that anywhere. All you need to do is iterate through it all. Plus, your $NetFanout is a small integer number - why hash it at all? - use it to index into an array directly - this way you don't even need to explicitly sort, you get that for free.
Instead of doing
$NetNameAndFanout = join (" ",$NetName,$NetFanout); $NetStatsHash{$NetNameAndFanout} = join (" ",$NetCap,$NetRes,$NetLengt +h,$FirstDriver);
try
use constant (p_name => 0, p_cap => 1, p_res => 2, p_length => 3, p_driver => 4); push @{$NetStats[ $NetFanout ] ||= []}, [ $NetName, $NetCap, $NetRes, +$NetLength, $FirstDriver, ];
Note that I use an extra technique to store things in small arrays to save on memory, but enumerate their positions to keep further code readable.
Now (with some other cleanup) your summary code becomes:
my $filter = sub { return 0 if ($UseDriverCell and $_[0]->[p_driver] !~ /$DriverPattern +/ ); return 0 if ($UseNetPattern and $_[0]->[p_name] !~ /$NetPattern/ ) +; # If we are here, then either we matched all requirements, or there +weren't any return 1; }; foreach my $i (1..$UpperFanoutLimitOfTable) { my $Count = 0; foreach my $net (grep {$filter->($_)} @{$NetStats[$i]}) { if ($DebugMode) { .. do debugging output .. } $TotalCap[$i] += $net->[p_cap]; $TotalRes[$i] += $net->[p_res]; $TotalLength[$i] += $net->[p_length]; $FanoutCount++; } if ($Count > 0) { $CapAvg[$i] = $TotalCap[$i] / $Count; $ResAvg[$i] = $TotalRes[$i] / $Count; $LengthAvg[$i] = $TotalLength[$i] / $Count; if ($DebugMode) { printf ("CapAvg[$i] = %0.6f\n",$CapAvg[$i]); printf ("ResAvg[$i] = %0.6f\n",$ResAvg[$i]); printf ("LengthAvg[$i] = %0.6f\n",$LengthAvg[$i]); } } else { print ("No nets match input criteria when fanout = $i. Please rev +iew command-line settings--they may be too restrictive!"); } }

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://683348]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others chanting in the Monastery: (3)
As of 2024-04-19 19:14 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found