Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

Re: Compare two large files and extract a matching row...

by mrguy123 (Hermit)
on Apr 17, 2012 at 12:57 UTC ( [id://965509]=note: print w/replies, xml ) Need Help??


in reply to Compare two large files and extract a matching row...

The problem here is that storing too much data in an array will give you a memory problem very fast (especially for large files)

Placing the data in a hash is a great solution (as mentioned above) but for large files that might still be a problem
What might help you out here, is that Perl is amazingly fast in reading files. I've seen it go through more than 40 GB of data in minutes. This means if your hash is to big you can split File1 into smaller hashes and then go over File2 a few times and it won't take you very long

Hope this helps
Mr Guy
  • Comment on Re: Compare two large files and extract a matching row...

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://965509]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others making s'mores by the fire in the courtyard of the Monastery: (5)
As of 2024-04-18 02:31 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found