Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things

Re: Parse data into large number of output files.

by Roger (Parson)
on Sep 29, 2004 at 12:31 UTC ( #394944=note: print w/replies, xml ) Need Help??

in reply to Parse data into large number of output files.

I never had to worry about using up too much memory these days because my E2900 has 32Gb of RAM, so I would always build an enormous hash in memory. 100Mb doesn't sound like too much data at all, I would build a hash in memory with the sender id as the key, then the messages in the array:

my %log = ( 'user1' => [ 'msg1', 'msg2', ... ], 'user2' => ... );

Nowadays even a half decent machines should have plenty of memory. I'd say just read into memory and write out one user at a time.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://394944]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others taking refuge in the Monastery: (4)
As of 2020-11-26 02:07 GMT
Find Nodes?
    Voting Booth?

    No recent polls found