Beefy Boxes and Bandwidth Generously Provided by pair Networks
XP is just a number

Re: Dealing with large logfiles

by young perlhopper (Scribe)
on Jul 20, 2000 at 05:29 UTC ( #23323=note: print w/replies, xml ) Need Help??

in reply to Dealing with large logfiles

In a project like this I really recommend going with a SQL database of some type. (MySQL and Postgres are both viable free alternatives see this column for a good overview of each's strengths and weakness's. Also, see this for a good intro to some of the things you need to be aware of like data modeling and so forth.

If you want to do cross referencing, SQL offers a powerful methodology for that type of thing. SQL in general makes it very easy to do very complex tasks with very few words. I would recommend SQL not merely for performance issues but for the degree to which it will help you deal with some of the complexity it sounds like you want to work into this project.

Good Luck,

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://23323]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others musing on the Monastery: (5)
As of 2023-09-22 09:36 GMT
Find Nodes?
    Voting Booth?

    No recent polls found