Re^2: Sorting data that don't fit in memory (DBM::Deep)

in reply to Re: Sorting data that don't fit in memory
in thread Sorting data that don't fit in memory

So, does the act of putting a hash into DBM::Deep cause the keys to be sorted (that would make sense for an efficient on-disk format and would be a very welcome feature to me) or are you suggesting that using perl's sort on an array from DBM::Deep would efficiently swap parts in and out of memory such that the sorting would work reasonably fast and also not require that much memory? (update: I guess the latter would also require storing the sorted results into a DBM::Deep array, perhaps the same one)

- tye

Comment on Re^2: Sorting data that don't fit in memory (DBM::Deep)

Replies are listed 'Best First'.
Re^3: Sorting data that don't fit in memory (DBM::Deep) by dragonchild (Archbishop) on Oct 10, 2007 at 17:10 UTC
The hashkeys are not stored sorted, no. But, in doing a sort (and I could be wrong), Perl doesn't necessarily pull all the data into memory. If it does, then I would certainly accept help in getting Perl to do the sort in the way you're describing. My criteria for good software: Does it work? Can someone else come in, make a change, and be reasonably certain no bugs were introduced?	[reply]
Re^4: Sorting data that don't fit in memory (DBM::Deep) by tye (Sage) on Oct 10, 2007 at 17:24 UTC
Then you might want to test your suggestion of using sort on an enormous DBM::Deep'd array. I'm not convinced that it will be easy to actually avoid using huge amounts of memory or that it will be reasonably fast. More detailed instructions on how to accomplish the former might be helpful and you might find improvements to make while looking at the latter. - tye	[reply]

In Section Seekers of Perl Wisdom