http://qs321.pair.com?node_id=1000852


in reply to Re^3: Processing ~1 Trillion records
in thread Processing ~1 Trillion records

Is this like using Redis to store/retrieve the key-value?

Replies are listed 'Best First'.
Re^5: Processing ~1 Trillion records
by Jenda (Abbot) on Oct 25, 2012 at 13:37 UTC

    Yes. You can use Redis itself, seems it does have a Perl binding. The whole point is to make sure the process fits in memory and the data that had to be moved to the disk is accessed/updated efficiently.

    Jenda
    Enoch was right!
    Enjoy the last years of Rome.