in reply to Fast seeking in a large array of hashes to generate a report.

Perhaps you could change the way you build your Dataobject. Instead of one array of hashes, you could build four hashes of (arrays of) hashes, each with a different field as the key. So, you could for example build a HoAoH with surname as key:

%Persons_by_surname = ( 'Schwartz' => [ { ..person 1 }, { ..person 2}, ], 'Wall' => [ {..person 1..} ] );

And the same for the other fields (except perhaps for age, where there will be a limited number of discrete values). Instead of looping over an array, this changes your algorithm to a hash lookup, which should be faster.

To facilitate your later lookup phase, you could put these data hashes in a wrapper hash like this:

my %data= ( 'surname' => \%Persons_by_surname, 'firstname' => \%Persons_by_firstname, # ... and so on );

Doing this, you'll have an initial hit on performance when building the extra hashes, but you should have faster lookups in the end. You'll have to benchmark your scripts to see whether this is worth it.


Replies are listed 'Best First'.
Re^2: Fast seeking in a large array of hashes to generate a report.
by jbrugger (Parson) on Jun 23, 2005 at 08:54 UTC
    Correct, but this only works for the "eq" operator, not for a regexp (i tried).
    update i'll try it, i did not know about Tie::Hash::Regex. i'll let you know the result.

    "We all agree on the necessity of compromise. We just can't agree on when it's necessary to compromise." - Larry Wall.

      Have you looked at Tie::Hash::Regex -- that allows you to look for hash keys with a regular expression. I have no idea as to the performance impact of this though, so you'll have to benchmark it.