Beefy Boxes and Bandwidth Generously Provided by pair Networks
XP is just a number
 
PerlMonks  

Re: Hashing Memory Usage

by skyknight (Hermit)
on Jul 16, 2006 at 18:09 UTC ( [id://561583]=note: print w/replies, xml ) Need Help??


in reply to Hashing Memory Usage

It sounds like you might want something called a "database". On another random note, maybe the Perl garbage collector never runs in the case where it gets to 1GB. For the one that stays at 53MB, do you cycle through 1GB worth of data but perhaps manage to reuse memory? Sometimes garbage collectors are sloppy when they aren't pressed up against the wall.

Replies are listed 'Best First'.
Re^2: Hashing Memory Usage
by awkmonk (Monk) on Jul 17, 2006 at 13:11 UTC
    Ah a red herring - it's not hashes at all! I'm changing values in a large hash as I loop through files. The problem occurs when I come to create the key for the hash. The following code works properly on 5.6 but grows on 5.8. Delete the substitution line, and hey presto, no more leak.
    use strict; my $ret = `ps v $$`; print "$ret\n"; for ( 1 .. 1000 ){ my $key = ':PERSON-NUM(1:*):NI-NUM(1:*):'; for my $bb ( "PERSON-NUM", "NI-NUM" ){ if ( $key =~ /:$bb *\( *([0-9]+) *: *([0-9\*]+) *\) *:/ ){ $key =~ s/:$bb *\( *[0-9]+ *: *[0-9\*]+ *\) *:/:$bb=1:/; } } } $ret = `ps v $$`; print "$ret\n";
    Now I'm confused!

    'I think the problem lies in the fact that your data doesn't fit my program'.

      I ran your code with the loop iterator set 1e6, and see no memory growth under either 5.8.6 or 5.8.8.

      I seem to recall that 5.8.2 (the version shown in your OP), was a particularly buggy, short-lived release. I strongly suggest you try upgrading to something newer. My memory tells me that 5.8.3 was a pretty good build as is 5.8.6 which I still use as my default install. I encountered a few problems with 5.8.8 but I think they may have been limited to the AS-Win distribution I use.


      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://561583]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others cooling their heels in the Monastery: (5)
As of 2024-04-25 08:51 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found