http://qs321.pair.com?node_id=103470

MeowChow has asked for the wisdom of the Perl Monks concerning the following question:

What strategies or techniques have you used to locate memory leaks in your long-lived perl scripts?
   MeowChow                                   
               s aamecha.s a..a\u$&owag.print

Replies are listed 'Best First'.
(tye)Re: Looking for Leaks
by tye (Sage) on Aug 10, 2001 at 01:45 UTC

    I put something in to log calls to destructors. I also "stub out" parts to see if the leak goes away.

    FYI, a fairly recently found bug is that Perl creates a circular reference when it creates a closure, so creating closures leaks memory. I think this has been fixed but that no released version of Perl contains the fix yet.

            - tye (but my friends call me "Tye")
Re: Looking for Leaks
by marvell (Pilgrim) on Aug 09, 2001 at 20:16 UTC

    Care to drop us an example of a memory leak?

    --
    Brother Marvell

    ¤

      The canonical examples have already been given, but for a practical example, try something like:
        
      sub make_a_leak { my $tree = HTML::TreeBuilder->new(); $tree->parse($some_big_html_file); return; }
      Without a call to $tree->delete, the above code will snarf tons of memory, because of all the circular references contained within the TreeBuilder parse-tree.
         MeowChow                                   
                     s aamecha.s a..a\u$&owag.print
        Maybe I should RTFM, but I've not had a problem with memory leaks in Perl, so I'm going to claim ignorance. Does anyone know which modules are known for causing problems like this? My mind immediately jumps to binding columns with DBI. Would wrapping calls in an eval help prevent a memory leak?
Re: Looking for Leaks
by dragonchild (Archbishop) on Aug 09, 2001 at 20:11 UTC
    I don't have any examples of techniques, and that's cause I'm curious as to how to generate memory leaks in Perl...

    ------
    /me wants to be the brightest bulb in the chandelier!

    Vote paco for President!

      Memory leak version 0.1 :)

      my $var; $var = \$var;

      -- Hofmator

      Since Perl still uses a reference count based garbage collector, creating memory leaks is quite easy. You just need to construct something with a reference count above 0 for all elements :

      use strict; my @a; a[0] = \a;

      In this (contrived) case, the reference count of a can't sink below 1, since a itself holds a reference to a.

      Of course, in this trivial example, it's easy to find the memory leak, but if you have a single linked list, it's easy to create a circular structure that will never be reaped.

      One way to overcome this would be to use another style of garbage collection, like a mark-and-sweep collector, that starts with a single "good" pointer, and recursively traces all memory reachable from there, and deallocates all memory unreachable from there. This obviously will create some interesting situations for when destructors will be called, but that's another story then ;-)

      I can't seem to find the post, but I remember somebody saying that it was possible to cause a memory leak by doing some strange stuff with references; if you can keep the reference count from being decremented to 0, the reference will never get destroyed even when it goes out of scope and viola; a memory leak.
Re: Looking for Leaks
by oakbox (Chaplain) on Aug 09, 2001 at 22:46 UTC
    I must have missed something, MeowChow;

    Are you looking for a way to monitor your system within the script to make sure it isn't gobbling up too much memory?

    Or are you looking for a tool that will help you identify where the leak is occuring within the script?

    oakbox

      I'm looking for the latter.
         MeowChow                                   
                     s aamecha.s a..a\u$&owag.print
        Well, isn't there a way/tool to, from the main section of the script, find out which memory locations aren't accessible? Everything should be accessible (except for closures) and all of that can be pointed out. You can then read through the output, mark off which you know to be closures, and then you have your answer...

        ------
        /me wants to be the brightest bulb in the chandelier!

        Vote paco for President!

Re: Looking for Leaks
by LD2 (Curate) on Aug 10, 2001 at 05:15 UTC