Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses
 
PerlMonks  

Re: Data. Lots of Data.

by zeno (Friar)
on Feb 07, 2001 at 18:42 UTC ( [id://56961]=note: print w/replies, xml ) Need Help??


in reply to Data. Lots of Data.

You mentioned that you store most of the data in hashes. Why not try only loading data into the hash when it is asked for, using the hash as a cache? Then you get the speed of the hash without loading all of the data into the hash, when you may only need part of it.

An example cribbed more or less from page 609 of Programming Perl, 3d edition:

#!/usr/bin/perl -w use strict; use warnings; sub get_data { # here we simulate the expensive operation # of loading data into the hash my $idx = shift; print "retrieving data for $idx...\n"; return "value for $idx"; } my $result; my %cache; foreach (1,2,4,16,32,2,64,1,2) { # here we are asking for values from the cache # (hash), and if they aren't there, we get them # from the file. $result = $cache{$_} ||= get_data($_); print "$result\n"; }

The output from this program is:

retrieving data for 1... value for 1 retrieving data for 2... value for 2 retrieving data for 4... value for 4 retrieving data for 16... value for 16 retrieving data for 32... value for 32 value for 2 retrieving data for 64... value for 64 value for 1 value for 2
As you can see, after the first time we ask for an element, we no longer have to go back to the file to get the data. Neat, eh? I hope that helps you. -zeno

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://56961]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others perusing the Monastery: (9)
As of 2024-03-29 15:24 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found