Memory leak on definition of anonymous code?

octothorpe has asked for the wisdom of the Perl Monks concerning the following question:

I've identified what seems to be a memory leak in Perl and verified it on 5.8.0 on a Windows machine and 5.005_03 on a UNIX server. It seems to occur when an anonymous subroutine is defined with code that would cause recursion. The simplified example below demonstrates the problem. Try running it on your system with Task Manager or `top` running to see the memory usage climb. On my system it gobbles up about 100MB per second for the first 3 seconds. Note that &$sub_ref() is never actually called: the recursion is defined but never executed.

use strict;             # leak occurs whether strict is used or not

while(1) {
  &leak();
}

sub leak {
  my $sub_ref;
  $sub_ref = sub { &$sub_ref(); };
  return 0;
}
[download]

To provide context, a more realistic usage might involve computing a factorial.

sub leak {
  my $factorial_sub_ref;

  $factorial_sub_ref = sub {
    my ($number) = @_;
    if ($number < 2) { return 1; }
    return($number + &$factorial_sub_ref($number - 1));
  };

  return 0;
}
[download]

Additional context: this problem originally occured for me in a recursive descent parser, which had a whole lot of tiny functions I wanted to obscure from UltraEdit's function list. Is this a Perl bug? If so, where can I find more information about its status?

Comment on Memory leak on definition of anonymous code? Select or Download Code

Replies are listed 'Best First'.
Re: Memory leak on definition of anonymous code? by Aristotle (Chancellor) on Oct 06, 2003 at 19:38 UTC
That's rather old news. You have just generated a cyclic reference: the variable references the closure and the closure references the variable. Perl cannot garbage collect such data structures because it uses a simple reference counter to keep track of referenced memory; in the case of cyclic structures all members reference each other, so the counter never reaches zero even if there aren't any references pointing to the structure from the outside. WeakRef may be of interest. Makeshifts last the longest.	[reply]
Re: Re: Memory leak on definition of anonymous code? by tilly (Archbishop) on Oct 07, 2003 at 06:52 UTC
For the curious, here is the leaky code fixed with WeakRef: `use strict; use WeakRef; while(1) { &leak(); } sub leak { my $sub_ref; $sub_ref = sub { &$sub_ref(); }; weaken($sub_ref); # <- This fixes the leak return 0; }` [download]	[reply] [d/l]
Re: Memory leak on definition of anonymous code? by dragonchild (Archbishop) on Oct 06, 2003 at 20:22 UTC
Not only is a factorial calculator not a good candidate for recursion, Perl (currently) doesn't do tail-recursion optimization, which would fix your memory leak. I'm also trying to figure out how your two examples are similar. The first defines new anonymous subs, but the second only defines one. (The additional subs are defined because you're calling the function definer over and over.) Try the following: `use strict; # leak occurs whether strict is used or not my $fact = build_sub(); while(1) { print &$fact(5), $/; } sub build_sub { my $factorial_sub_ref; $factorial_sub_ref = sub { my ($number) = @_; if ($number < 2) { return 1; } return($number * &$factorial_sub_ref($number - 1)); }; return $factorial_sub_ref; }` [download] Also, factorials are multiplied, not added. ------ We are the carpenters and bricklayers of the Information Age. The idea is a little like C++ templates, except not quite so brain-meltingly complicated. -- TheDamian, Exegesis 6 Please remember that I'm crufty and crochety. All opinions are purely mine and all code is untested, unless otherwise specified.	[reply] [d/l]
Re: Memory leak on definition of anonymous code? by waswas-fng (Curate) on Oct 06, 2003 at 19:36 UTC
welcome to the wonderful world of recursion. Check out This (off site). -Waswas	[reply]
Re: Memory leak on definition of anonymous code? by jonadab (Parson) on Oct 06, 2003 at 20:16 UTC
Yes, we're still waiting to find out whether Perl6 will get real garbage collection that would fix this, or have the same reference counting that Perl5 has. `$;=sub{$/};@;=map{my($a,$b)=($_,$;);$;=sub{$a.$b->()}} split//,".rekcah lreP rehtona tsuJ";$\=$ ;->();print$/` [download]	[reply] [d/l]
Re: Re: Memory leak on definition of anonymous code? by sandfly (Beadle) on Oct 06, 2003 at 20:46 UTC
I believe the issue has been decided - Perl 6 will have real GC.	[reply]
Re: Memory leak on definition of anonymous code? by jonadab (Parson) on Oct 06, 2003 at 20:49 UTC
I believe the issue has been decided - Perl 6 will have real GC. Really? Cooool. `$;=sub{$/};@;=map{my($a,$b)=($_,$;);$;=sub{$a.$b->()}} split//,".rekcah lreP rehtona tsuJ";$\=$ ;->();print$/` [download]	[reply] [d/l]
Re^2: Memory leak on definition of anonymous code? by Aristotle (Chancellor) on Oct 07, 2003 at 06:57 UTC
Yes it will, courtesy of the Parrot VM it will run on, which (AFAIK) uses a mark and sweep type GC. Makeshifts last the longest.	[reply]
Re: Mark-and-sweep in Parrot by jonadab (Parson) on Oct 07, 2003 at 12:57 UTC
`[`Thinks about this answer for a brief moment.] So, if the GC is built into Parrot... does that mean Ponie will get it too? `$;=sub{$/};@;=map{my($a,$b)=($_,$;);$;=sub{$a.$b->()}} split//,".rekcah lreP rehtona tsuJ";$\=$ ;->();print$/` [download]	[reply] [d/l] [select]
Re: Re: Memory leak on definition of anonymous code? by Elian (Parson) on Oct 07, 2003 at 15:04 UTC
No we're not. The decision's been made for years... No refcounting for perl 6.	[reply]


Keep It Simple, Stupid
	PerlMonks