in reply to Processing ~1 Trillion records
Why are you bothering (and paying the transport & memory costs) to query 5 fields for every record, when you know that 3 of those fields are always going to be '9', 'STR', & 'STR1' respectively?
Why build a 2-level hash:
$data{ $dat[0] }{ $dat[1] } = $tmarker;
when the first level is always '9'.
Ditto this:
$lines{ $dat[0] } = "";
And why are you doing this:
if( !defined $dat[3] ) { $dat[3] = 0; } if( !defined $dat[4] ) { $dat[4] = 0; }
When the query dictates that fields 3 & 4 can never be undefined.
What is the point of this:
$datachr{ $dat[1] }{ $dat[3] } = $dat[4];
when $dat[3] & $dat[4] are constants?
Ditto this:
$chr{ $dat[3] } = "";
With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
|
---|
In Section
Seekers of Perl Wisdom