Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical

Re^3: Matching lines in 2+ GB logfiles.

by dave_the_m (Monsignor)
on May 01, 2008 at 17:55 UTC ( #683979=note: print w/replies, xml ) Need Help??

in reply to Re^2: Matching lines in 2+ GB logfiles.
in thread Matching lines in 2+ GB logfiles.

I am basically looking for something faster than grep
You're unlikely find anything much faster than grep - it's a program a written in C and optimised to scan though a text file printing out lines that match. You may also be running into IO limits. For example, you could try
time wc -l bigfile
Which will effectively give you a lower bound (just read the the contents of the file, and find the \n's). If the grep isn't a whole lot slower than that, then there's probably no way to speed it up.


Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://683979]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others musing on the Monastery: (2)
As of 2022-01-27 04:56 GMT
Find Nodes?
    Voting Booth?
    In 2022, my preferred method to securely store passwords is:

    Results (70 votes). Check out past polls.