Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw
 
PerlMonks  

Re: Re: Tracking Memory Leaks

by Hrunting (Pilgrim)
on Aug 15, 2001 at 01:24 UTC ( #104886=note: print w/replies, xml ) Need Help??


in reply to Re: Tracking Memory Leaks
in thread Tracking Memory Leaks

Heh, well, we do all those things, but they should be controlled.

  1. Yes, in various places, but they should all be limited in one way or another (either they go out of scope at various points or they only have a limited number of elements in them).
  2. We do have one module that uses circular references, but using Weakref, that shouldn't be a problem.
  3. Yes, scoping on the project is very tight.
  4. This is the thing that I'm worried about. We're constantly loading new classes, dymanically created from database information. Now, theoretically, we have a finite number of classes, so the memory usage from loading these things should be finite itself (and that limit should be hit rather quickly). However, I'm not positive this is the case. Anyway know much about the finer points of this?
  5. No. Thanks for the help.

Replies are listed 'Best First'.
Re: Re: Re: Tracking Memory Leaks
by dragonchild (Archbishop) on Aug 15, 2001 at 02:19 UTC
    To reply to a few points:
    1. Be very careful. Simply because something is out of scope does not mean that it's been garbage-collected. Directly from the Camel (3rd ed., p.223):

      Lexical variables don't get automatically garbage collected just because their scope has exited; they wait to get recycled until they're no longer used, which is muych more important. To create private variables ...

      I would suspect that this is a minor problem, but probably not the whole issue.

    2. As for the constantly loaded classes ... I'm going to guess that you're creating the classes using some sort of AUTOLOAD(), eval, or combination of the two. If that's the case, these classes are probably taking quite a chunk of memory. Now, by doing it this way, you're taking away any possible compile-time optimizations that the compiler could do. In addition, I think that once you use AUTOLOAD() once, the compiler will allocate additional memory every time, just as if you used $` or $' in regexps.

      I think that this is the culprit. I would suspect that it's not that you're loading 10^12 classes, but that each class is taking a bigger chunk than you realize.

    ------
    /me wants to be the brightest bulb in the chandelier!

    Vote paco for President!

      How does the interpreter determine whether a lexical is no longer "used"? You have to explicitly set it to undef? That seems doesn't seem like too much a "feature". So my memory usage won't really top out until all my functions have been called and all my variables used, is that correct?

      I didn't know that about AUTOLOAD(). What do you mean by 'allocate additional memory every time'? Does that mean that every time I effectively call AUTOLOAD() (In my case, once for each undeclared function as I use AUTOLOAD() to then declare the function), the interpreter allocates a chunk of memory, or just that everytime I load a module with an AUTOLOAD() in it, it will allocate a larger chunk of memory than it would for a regular module? I know I miss out on compile-time optimizations for the system, but do those optimizations involve the re-use of resources, as opposed to the allocation of resources (which I know they involve)? I would expect that resource re-use would be a function of the running system, not the compile-time optimizations. If perl allocates memory for eval'd statements and AUTOLOAD()ed subroutines and then doesn't re-use it, that sounds like a pretty serious issue.

        I don't know what exactly dragonchild was referring to, except that if you create methods or classes on the fly, that's going to consume memory. No more so than if you'd put the same definitions in a .pm file and use'd it, though. I don't think AUTOLOAD or eval("string") themselves leak memory. And I don't know what "compile time optimizations" he was referring to; eval is as much compile time as what you get when you load your main program or use/require'd modules.
Re: Re: Re: Tracking Memory Leaks
by bikeNomad (Priest) on Aug 15, 2001 at 20:47 UTC
    We're constantly loading new classes, dymanically created from database information. Now, theoretically, we have a finite number of classes, so the memory usage from loading these things should be finite itself (and that limit should be hit rather quickly). However, I'm not positive this is the case. Anyway know much about the finer points of this?

    Are you cleaning up the symbol tables after you're done using these new classes, or are they being re-used? We found in Class::Prototyped that each new class takes up around 1.5-2K of memory; more (of course) with methods. You can clean up the symbol table when you're done with code like this (assumes $package does not contain '::'):

    no strict 'refs'; foreach my $key ( keys %{"$package\::"} ) { delete ${"$package\::"}{$key}; } # this only works because we're not a multi-level package: delete( $main::{"$package\::"} );

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://104886]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chilling in the Monastery: (3)
As of 2020-08-11 04:36 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Which rocket would you take to Mars?










    Results (58 votes). Check out past polls.

    Notices?