Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

Re^3: Advice for optimizing lookup speed in gigantic hashes

by BrowserUk (Patriarch)
on Aug 23, 2011 at 15:25 UTC ( [id://921934]=note: print w/replies, xml ) Need Help??


in reply to Re^2: Advice for optimizing lookup speed in gigantic hashes
in thread Advice for optimizing lookup speed in gigantic hashes

the virtual-memory thrashing that occurs by trying to keep millions of words “in memory” at the same time.

Do you actually read the posts you reply to, or just divine their content through your lower lumbar regions? Because that's what you are talking out of right now.

He isn't storing millions of words “in memory”. He clearly states that his dictionary contains 100,000 words. Mine contains 178,000 words:

@words = do{ local( @ARGV, $/) = 'words.txt'; <> };; print scalar @words;; 178691 undef @words{ @words };; print total_size( \%words );; 14 224 469 ## Spaces added for clarity!

Mine occupies just 14MB (his less than 10MB). My 3 yr old decidedly unsmart £30 cellphone has 10 times that amount. In case that hasn't sunk in, let me make this clear:

None of the millions of words from the huge file is ever stored in the hash!

They are read from the file a line at a time, split into a list of words, looked up in the hash (NOT STORED), and then discarded.

So will you please, please, please stop trotting out your "virtual memory is disk" missive at every inopportune moment. Read what you are responding to. (Follow your own advice: read it twice, and then once more). Think about it for a while. AND THEN SHUT THE F*** UP. Because it is getting boring trying to keep correcting you over and over and over.

You remind me for all the world of my dear ol' Nan. Loved her to bits, but towards then end she was getting a little bit fixated.

How are you Nan?

During the war we had to make ends meet, so we stained our legs with used coffee grounds, and drew lines on the back of our legs with eye-liner, so we didn't have to buy stocking.

No, no, no. Nan. Nan. Sit down Nan, you need your rest. You don't need to demonstrate. Really no. We believe you.

So, how did you get on at the doctors Nan?

During the war we ....


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.

Replies are listed 'Best First'.
Re^4: Advice for optimizing lookup speed in gigantic hashes
by CountZero (Bishop) on Aug 23, 2011 at 18:22 UTC
    The only good answer to this posting is a humble "I stand corrected". :)

    CountZero

    A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://921934]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others browsing the Monastery: (3)
As of 2024-04-20 04:29 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found