Beefy Boxes and Bandwidth Generously Provided by pair Networks
Welcome to the Monastery
 
PerlMonks  

Re: Searching text files

by runrig (Abbot)
on Sep 14, 2006 at 18:06 UTC ( [id://572962]=note: print w/replies, xml ) Need Help??


in reply to Searching text files

Is there anything preventing you from using a database? It would be faster than scanning large text files.

Replies are listed 'Best First'.
Re^2: Searching text files
by SteveS832001 (Sexton) on Sep 14, 2006 at 18:09 UTC
    No, There is not, I was just Trying to find the best way to do this since my way sucks
      If you are just looking for exact matches, then a non-SQL database like DB_File (the tied-hash interface) is probably adequate; if not, then a SQL database using 1 table, 1 column, and an index. Then of course you have to develop processes to update the database and keep in sync with the text files.
      There is also File::SortedSeek which I have no experience with, but might be "good enough" if the files are sorted, and would save you from keeping the database in sync with the text files.
      It appears that you are installing and running this on each desktop. This is highly inefficient a slient server model would be a lot better as this would decrease the workload on the desktop machine.
      The simplest way of doing this is using a database and you get the performance benifits if server caching index.
      Your current code appears to do a straight string match similar to a like. Indexes are not as good as handling these kinds of searches as the whole string is not matched. If you are matching a complete sting an index is very fast.
      If you need fast lookups on partial numbers you might look at either reverse indexes that sort based on the digits in reverse order or preprocessing the numbers and extracting sequences of digits and then indexing them.

      UnderMine

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://572962]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others sharing their wisdom with the Monastery: (3)
As of 2024-04-25 21:56 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found