Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister
 
PerlMonks  

Re^2: Hash Search is VERY slow

by AnomalousMonk (Archbishop)
on Sep 29, 2021 at 19:13 UTC ( [id://11137125]=note: print w/replies, xml ) Need Help??


in reply to Re: Hash Search is VERY slow
in thread Hash Search is VERY slow

I think you'll find that it was choroba's advice to process the file on a line-by-line (CSV-record-by-record in this case) basis that did the trick. :)


Give a man a fish:  <%-{-{-{-<

Replies are listed 'Best First'.
Re^3: Hash Search is VERY slow
by NERDVANA (Deacon) on Sep 29, 2021 at 22:37 UTC
    I’m going to bet it was bliako’s observation that the array was getting cloned every time an element was added. That’s where the N^2 behavior came from.
      Oh I missed that.

      => Re: Hash Search is VERY slow

      Brilliant!

      > that the array was getting cloned

      I think it's not so much the copying alone but the allocation of a new array each time.

      Because the arrays are growing, old released memory often can't be reused, leading to a fragmentation and increasing waste of space.

      Fun! :)

      Tho I might be wrong, the reserved space for an array grows by doubling IIRC, this should be easier to reuse ...

      Cheers Rolf
      (addicted to the Perl Programming Language :)
      Wikisyntax for the Monastery

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://11137125]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others goofing around in the Monastery: (6)
As of 2024-03-28 20:19 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found