http://qs321.pair.com?node_id=938688


in reply to Re: "Just use a hash": An overworked mantra?
in thread "Just use a hash": An overworked mantra?

An array to hold 1000 integers requires 32k:

@a = 1 .. 999;; print total_size \@a;; 32144

A hash to hold 1000 keys & integer values requires 100k:

$h{ $_ } = $_ for 1.. 999;; print total_size \%h;; 109055

So, on a machine with less memory than some musical birthday cards, the hash or array one-liners will perform this task efficiently, and scale linearly for files containing at least 18,446,744,073,709,551,616 lines.

To put that into perspective, it represents a single data file containing 16 Exabytes. Or approximately a million times more data than Google's entire storage capacity. If the OP can afford the amount of disk required to hold the file, it seems very unlikely that he'll have any trouble affording the 100k of ram.

In the meantime, it would take a computer running at 10Ghz and able to perform 1 comparison per clock cycle, 3,741 years to sort that file, assuming no other time costs including IO or memory.

So do the world a favour, and apply a little, the merest modicum, of thought to the problem at hand, before trotting out your olde worlde compooter wisdoms. Regurgitating received knowledge, long since superseded, as a substitute for actually thinking about the problem, does no one any good.


With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.