Beefy Boxes and Bandwidth Generously Provided by pair Networks
laziness, impatience, and hubris
 
PerlMonks  

Re: Optimizing Memory consumption

by brig (Scribe)
on Nov 10, 2006 at 00:02 UTC ( [id://583261]=note: print w/replies, xml ) Need Help??


in reply to Optimizing Memory consumption

This isn't answering the question you asked; Mostly because I'm not convinced you are asking the right question.

My Rule of Thumb is that anytime there is a significant change to requirements, there needs to be a significant review of the architecture. I would say that a quadrupling of your memory requirements and the fact that you are using a 200MB text file confirms this.

There isn't really enough info about your situation to come to any realistic conclusion because you have already decided to optimize (fair enough). However, when I see a monstrous text file, I always wonder if the use of a Relational DB and clever queries would generate a more efficient solution. You mention that "the data to be stored has increased by (a?) large (margin?)" this is often a point at which data abstraction needs to be reviewed.

Finally, please forgive me if I am off base.

(update: recomposed 1 sentence.)

Love,
Brig

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://583261]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others sharing their wisdom with the Monastery: (3)
As of 2024-04-19 01:41 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found