Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses
 
PerlMonks  

Re: search a large text file

by chrestomanci (Priest)
on Feb 08, 2011 at 11:40 UTC ( [id://886931]=note: print w/replies, xml ) Need Help??


in reply to search a large text file

Considering the size of the file, nothing will be fast, however, unless you need to do lots of searches, there is no need to sort the file. You are only interested in the lines that match you search term.

If you don't have many searches to do, then the approach I would take would be to filter the file using unix tools to get the lines of interest. eg:

grep 'text2' <filename>

This will give you the lines containing the data you want in linear time. From there you easily turn it into a perl array.

If you need to do lots of searches, then you will need to consturct an index of some sort, or ingest the whole thing into a database, and let the database engine worry about the index.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://886931]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others contemplating the Monastery: (5)
As of 2024-04-24 09:33 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found