http://qs321.pair.com?node_id=642679


in reply to understanding devel::leak

Hi nonk ,
This theme seems to be quite common , memory gets eaten up and is not released .
The issue is eroding my own confidence in proceeding to develop a perl based tool which I would like to be able to control the release of MEM , when not needed via the best possible code .
On my NT PC / with Perl running under ActiveState , the following code eats up 92,608 KB MEM . After the undef the MEM usage drops only to 90,572 KB MEM . An awefull lot of Residual MEM is not returned to the O/S until the program exits .
I know that I am pushing an anonymous array into X , but it really concerns me that undef is not reclaiming the MEM . Can my code be modified to allow the MEM to be reclaimed to the O/S . I have not found a way
Nova <code> #!perl { my $i=0 ; my @X ; while ( $i < 500000 ) { push @X , $i , $i , $i , $i ; if ($i%10000==0) { printf "Index %-7d %-7d \n\n" , $i , $X$i2 ;} $i++ ; } ; printf "Check MEM usage then Return to undef X" ; <> ; undef @X ; }; printf "Enter Return to quit program" ; <> ; <\code>

Replies are listed 'Best First'.
Re^2: understanding devel::leak
by almut (Canon) on Oct 04, 2007 at 16:18 UTC

    The problem you are having is not a 'true' memory leak, i.e. the memory in fact is being freed. It just isn't being returned to the OS, but rather to Perl's own memory pool. Those two things are often confused. The difference is that if you'd run your while loop another time, it would not allocate another 90MB, but rather reuse the memory freed upon undef-ing the @X.  If it were a memory leak in the sense that the rest of the thread is talking about, it'd eat up more and more memory, without you having a chance to reuse it (without exiting the interpreter).

[reply]
[d/l]
[select]
      Thank you for your responses and I apologize for the format problem in my previous post .
      From the point of view of the O/S , I can see that , once I have done work via my script which requires MEM , a large chunk of that MEM may no longer be available to the O/S for concurrently running apps .
      My perl app is running on WIN32 / ActiveState Perl 5.8.8 . Can I set any variables which would apply the -DPURIFY after Perl has already been compiled , or the binaries intalled .
      Further , while the script is running is there a variable which can be examined which I can use to monitor MEM usage .
      Kind regards, Nova .
[reply]
        Can I set any variables which would apply the -DPURIFY after Perl has already been compiled

        Unfortunately not, it's a compile-time option only. You can read a bit more about this option and its primary use in perlhack, section External Tools for Debugging Perl.   Also, I suspect you'd incur a noticable decrease in performance with -DPURIFY (even when not running under Purify, valgrind, etc.); in other words, it's probably a matter of trading speed for memory control... (I haven't done any benchmarking, though.)

        is there a variable which can be examined which I can use to monitor MEM usage

        That would definitely be a nifty feature, but I don't think such a variable exists.   So, in addition to the respective system tools (like top on Unix — not sure what the best tool on Windows is), you might try the module GTop (based on libgtop from GNOME), but I don't know how well - if at all - that'd work on (native) Windows. Grepping through libgtop's configure file produces a couple of hits in connection with Cygwin and MinGW, but that's about all I can tell...

[reply]
[d/l]
[select]
Re^2: understanding devel::leak
by diotalevi (Canon) on Oct 04, 2007 at 20:19 UTC

    I recently learned you can compile your perl with the parameter -DPURIFY which disables perl's own memory pool. This means whenever you release memory it is visible outside the process. Normally freed memory is just available to perl to do new things with. Perl's own pool may also free parts of itself but that's a separate issue from your structures as visible from within your program. This is just an effect of using a language with its own garbage collection.

    ⠤⠤ ⠙⠊⠕⠞⠁⠇⠑⠧⠊

[reply]
      Thank you Monks for the pointers.

      After several weeks of bashing my head against this problem I have at last made a breakthrough, of sorts.

      Along my voyage of discovery I've tried:
      Devel::leak - no use as I've no idea how to recompile on win32 / activestate. I downloaded the source and read the readme but it just left me even more confused.
      Devel::Leak::Object - Don't recall why I gave up on that one.
      Devel::Cycle - My data structure is pretty complex so it was entirely plausible that I'd looped somewhere. I thought I wasn't using find_cycle right but it turns out there was nothing there to find.
      Devel::Monitor - plain doesn't work on win32 / activestate.
      Devel::Peek - I sort of understood what this was telling me but not entirely.
      Win32::Perfmon - doesn't work for any of the counters that have instances.

      In the end all I could do was step through with the degugger looking for patterns in where perls private bytes increased. Not easy due to the way perl grabs memory in 4k chunks. In the end I got down to one suspect package function. I wrote a fake version of this that returns realistic data. Using this I have proved that there are absolutely no memory leaks in my code. I'd be quite please with this if only it did not render 6 weeks effort pretty worthless.

      The module causing the problem is Win32::OLE:OPC and the function is SyncRead.

      I'll have a brief look inside the package but I think I reached my limit. It's entirely possible that the problem isn't in that package or even in any of it's perl dependancies but elsewhere in OLE or COM.

      Bit depressing really.
      nonk

[reply]

In Section Seekers of Perl Wisdom