Beefy Boxes and Bandwidth Generously Provided by pair Networks
go ahead... be a heretic

Re^3: Reading huge file content

by dsheroh (Monsignor)
on Dec 06, 2007 at 15:23 UTC ( #655431=note: print w/replies, xml ) Need Help??

in reply to Re^2: Reading huge file content
in thread Reading huge file content

or better yet, try to just process a line at a time.

Indeed. Especially since even reading this much data (never mind processing it) will take considerable time. Reading one line, processing it, then reading the next would also allow you to store a counter on disk somewhere or displayed on the screen with the number of bytes successfully read and processed (obtained using tell) so that, if the process dies partway through, it can pick up from where it left off when restarted instead of having to go back to the beginning of the file and repeat what's already been done.

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://655431]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others romping around the Monastery: (5)
As of 2022-01-29 08:38 GMT
Find Nodes?
    Voting Booth?
    In 2022, my preferred method to securely store passwords is:

    Results (74 votes). Check out past polls.