http://qs321.pair.com?node_id=266924

Elgon has asked for the wisdom of the Perl Monks concerning the following question:

Fellow Monks,

I have a bit of query: I am trying to do some relatively straitforward file transformations as part of some testing we're doing on a project at work. The first stage of this involves reading in a big lookups file, circa 160,000 records ~3.4MB, and converting it into a hash.

The format of the file is basically records of twenty-one characters long, where the first thirteen characters are going to be the key for the next eight. I need to go through the file and read it into a hash for use as a lookup table. I am doing this thusly...

open (CONFIG, "<$config_file)" || die "Coulnd't open config file!"; my %lookup; while (<config>) { $lookup{substr($_, 0, 13)} = substr ($_, 13); }

This may seem hugely inefficient, however the overhead of setting this up is offset by the fact that we'll be processing some 700 files each of about a megabyte in length. This is being done on a BIG box needless to say and memory is not going to be a problem.

But it seems to be taking about half an hour to do the initial processing. Is there a faster way to do it?

Thanks in advance...

Elgon

Please, if this node offends you, re-read it. Think for a bit. I am almost certainly not trying to offend you. Remember - Please never take anything I do or say seriously.