Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?
 
PerlMonks  

Re^2: Sorting data that don't fit in memory (DBM::Deep)

by tye (Sage)
on Oct 10, 2007 at 16:05 UTC ( #644011=note: print w/replies, xml ) Need Help??


in reply to Re: Sorting data that don't fit in memory
in thread Sorting data that don't fit in memory

So, does the act of putting a hash into DBM::Deep cause the keys to be sorted (that would make sense for an efficient on-disk format and would be a very welcome feature to me) or are you suggesting that using perl's sort on an array from DBM::Deep would efficiently swap parts in and out of memory such that the sorting would work reasonably fast and also not require that much memory? (update: I guess the latter would also require storing the sorted results into a DBM::Deep array, perhaps the same one)

- tye        

  • Comment on Re^2: Sorting data that don't fit in memory (DBM::Deep)

Replies are listed 'Best First'.
Re^3: Sorting data that don't fit in memory (DBM::Deep)
by dragonchild (Archbishop) on Oct 10, 2007 at 17:10 UTC
    The hashkeys are not stored sorted, no. But, in doing a sort (and I could be wrong), Perl doesn't necessarily pull all the data into memory. If it does, then I would certainly accept help in getting Perl to do the sort in the way you're describing.

    My criteria for good software:
    1. Does it work?
    2. Can someone else come in, make a change, and be reasonably certain no bugs were introduced?

      Then you might want to test your suggestion of using sort on an enormous DBM::Deep'd array. I'm not convinced that it will be easy to actually avoid using huge amounts of memory or that it will be reasonably fast. More detailed instructions on how to accomplish the former might be helpful and you might find improvements to make while looking at the latter.

      - tye        

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://644011]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others exploiting the Monastery: (4)
As of 2020-11-01 01:35 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    My favourite web site is:












    Results (291 votes). Check out past polls.

    Notices?