Beefy Boxes and Bandwidth Generously Provided by pair Networks
Syntactic Confectionery Delight
 
PerlMonks  

Re: How to sort a large flat file 90mb with PERL-- various ways/tradeoffs/watchouts

by delirium (Chaplain)
on Jun 30, 2004 at 01:36 UTC ( [id://370663]=note: print w/replies, xml ) Need Help??


in reply to How to sort a large flat file 90mb with PERL-- various ways/tradeoffs/watchouts

If you aren't able to modify @INC or get modules installed, and have a lot of data to sort, perl may not be the best answer for you.

Do you have Unix on that AIX box? If so, I recommend sticking with the Unix sort program. A command like:

sort file.txt > file.sorted.txt

should do the trick.

Unix sort is handy with large files, breaking them down to many smaller files and reassembling them as it moves. It uses a good balance of cpu, ram, and drive space, in my experience. I used to use it to help sift through data on some syslog servers at my old job, and actually replaced a production perl script with a Unix grep/sort one-liner due to the performance boost (over my sloppy perl script, that is).

  • Comment on Re: How to sort a large flat file 90mb with PERL-- various ways/tradeoffs/watchouts
  • Download Code

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://370663]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others chilling in the Monastery: (5)
As of 2024-04-19 16:21 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found