Beefy Boxes and Bandwidth Generously Provided by pair Networks
go ahead... be a heretic
 
PerlMonks  

How to avoid closures and memory leaks.

by runnerup (Novice)
on Oct 18, 2014 at 11:04 UTC ( [id://1104224]=perlquestion: print w/replies, xml ) Need Help??

runnerup has asked for the wisdom of the Perl Monks concerning the following question:

I have single file code that has lots of
my $vars, my %vars, my @vars or use vars qw($vars, $vars, @vars)
at the top.

Then I am using Anyevent event library to run code and the code uses the variables in functions like
sub work {
stuff that uses $var or %vars or @vars
}

or

http_request (
...,
sub { //callback
stuff that uses $var or %vars or @vars
});

How would I avoid memory leaks due to closures in such cases?
reference) http://www.perlmonks.org/?node_id=268891
  • Comment on How to avoid closures and memory leaks.

Replies are listed 'Best First'.
Re: How to avoid closures and memory leaks.
by Corion (Patriarch) on Oct 18, 2014 at 12:18 UTC

    Without more concrete examples, this has to remain somewhat abstract:

    With "real" global variables, you won't get memory leaks, as these variables are truely global and basically referred to by their name instead of a counted reference anyway. The great aspect of global variables is that they are accessible from everywhere, so you can clear them out or change them once it becomes necessary in your program. So using our is unproblematic in any case.

    With "lexical global" variables (or "file scope lexical" variables), that is, variables declared using my but near the top of your file, these variables will be shared between all callback instances that use them. This can still lead to some memory leaking, but as these variables are, by almost all intents, global too, it's just a little bit worse than using a real global variable.

    With lexical variables declared "just above" your callback and then used in your callback, things become complicated:

    The first case is if your callback simply closes over a lexical variable, like in the below code. Then, $message will basically be only used by $callback and thus only stay alive as long as $callback stays alive.

    sub handle_foo { my $message= 'Example'; my $callback= sub { print $message; }; ... };

    The problematic part comes when a subroutine closes over a variable that in turn holds a reference to the subroutine. This will basically prevent the reference counting garbage collection from reclaiming the memory for the variable, the closure and all variables that hang off the closure until the end of the process:

    sub fetch_item { my( $item, $on_success )= @_; my $fetch; $fetch= sub { http_get( $item, success => $on_success, error => sub { if( can_retry( $item ) ) { goto &$fetch; }; }, }; };

    In the above case, the callback subroutine stored in $fetch closes over $fetch itself and thus the closure created with each call to fetch_item will ever be automatically reclaimed until the end of the process. The pattern proposed in AnyEvent is to manually undefine $fetch at the end of every relevant callback:

    sub fetch_item { my( $item, $on_success )= @_; my $fetch; $fetch= sub { http_get( $item, success => sub { undef $fetch; # we are done goto &$on_success; }; error => sub { if( can_retry( $item ) ) { goto &$fetch; } else { undef $fetch; # we are done retrying }; }, }; };
      Thanks,
      What happens if
      use vars qw(%list ....)

      %list being a hash of sockets like
      %list{listName}->socketObject;

      Then I use the sockets in subroutines like
      sub sendToList {
      send stuff to all $list{listName}->socketObjects;
      ...
      }

      Would that be okay with no memory leaks, or would it hold on to the socketObjects even after delete $list{listName} ?
      Would it be the same for
      http_request (
      ...
      ,sub {
      send stuff to all $list{listName}->socketObjects;
      });

      Closure stuff seems difficult for me to grasp :(
      Again thanks for your reply.

        This is less an issue of closures and more an issue of you not understanding the difference between global variables (use vars) and lexical variables (my). See Coping With Scoping for a good explanation of these things.

        Basically, what you declare with use vars are global variable names, and these have no issues with closures, because anyone can get at a global variable as long as they know the name.

Re: How to avoid closures and memory leaks.
by ikegami (Patriarch) on Oct 19, 2014 at 05:51 UTC

    What memory leak? There's no memory leak in what you posted, and there's no mention of memory leaks in the document you mention as the basis for this.

    This is how you get a memory leak from a closure:

    my $foo; $foo = sub { ... $foo->() ... };

    The sub reference $foo, and $foo is a reference to the sub. But you don't have that.

Re: How to avoid closures and memory leaks.
by LanX (Saint) on Oct 18, 2014 at 11:12 UTC
      Will this avoid closure?

      my $test = "a";

      sub test {
      my $temp = $test;
      }

      Will $temp avoid closure?
        well if you meant $temp instead of $test ... YES!

        my $temp = "a"; sub test { my $temp = $test; }

        please read the link provided about Coping with Scoping and please use <code> tags!

        update

        Basically does a my variable (aka lexical) belong to the surrounding '{ BLOCK }'.

        A sub's (or an if's) body is a { BLOCK } and the file itself (i.e w/o surrounding curlies) is practically the top-level block.

        Cheers Rolf

        (addicted to the Perl Programming Language and ☆☆☆☆ :)

Re: How to avoid closures and memory leaks.
by salva (Canon) on Oct 20, 2014 at 10:57 UTC
    Use curry::weak from curry or Method::WeakCallback.

    Those help when your memory leaks are due to subs wrapping method calls as for instance:

    $self->{watcher} = AE::io $fh, 0, sub { $self->_read(@_) };

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://1104224]
Approved by vinoth.ree
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others chanting in the Monastery: (3)
As of 2024-04-25 23:34 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found