Come for the quick hacks, stay for the epiphanies. | |
PerlMonks |
Re^3: Further on buffering huge text filesby Anonymous Monk |
on Mar 09, 2005 at 12:18 UTC ( [id://437875]=note: print w/replies, xml ) | Need Help?? |
Can you cache information? If files only grow with time, and your filters are constant, you could record for each line written to the file (or better, for each line read from the file) which filter(s) it matches. Using a bit-vector for each filter, and a single bit per line, you only need 125k of storage per filter for a million line file.
In Section
Seekers of Perl Wisdom
|
|