Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

ithreads memory leak

by DNAb (Novice)
on Apr 08, 2015 at 03:38 UTC ( [id://1122762]=perlquestion: print w/replies, xml ) Need Help??

DNAb has asked for the wisdom of the Perl Monks concerning the following question:

I'm puzzled by something that seems simple, but yet is yielding strange results. I'm using threading as part of a decent sized program, the easiest way to describe the purpose of this code is it's part of an alarm system.

What I'm running into is a memory leak when using ithreads, sleep, and detach. This doesn't seem to happen if I use join to wait for the thread to finish execution, but I can't do this because the main loop has to keep running.

Here is a short snippet of code that exhibits the issue, the modules are in there for two reasons, one being it shows the leak better, two being these are what are used in the production code.

use warnings; use IO::Handle; use threads; use strict; use DBI; use Email::MIME; use Email::Sender::Simple qw(sendmail); while (<STDIN>) { chomp(my $msg = $_); print "Received: $msg\n"; if ($msg =~ /start/) { my $thread = threads->create(\&thread); } } sub thread { print "Thread started!\n"; sleep 5; threads->detach(); }

I'm using PERL 5.20.2, compiled using perlbrew. I'm using top and ps to watch memory usage on a Linux server. Type "start" a few times and the memory usage balloons about 6MB each time. At random points the ballooning stops, and PERL will reclaim seemingly about 1MB of memory, then start ballooning again.

What am I missing here?

Replies are listed 'Best First'.
Re: ithreads memory leak
by BrowserUk (Patriarch) on Apr 08, 2015 at 04:27 UTC
    the memory usage balloons about 6MB each time. At random points the ballooning stops, and PERL will reclaim seemingly about 1MB of memory, then start ballooning again.

    What happens if you recast the code as follows?:

    use warnings; use IO::Handle; use threads stack_size => 64*1024; use strict; use DBI; use Email::MIME; use Email::Sender::Simple qw(sendmail); while (<STDIN>) { chomp(my $msg = $_); print "Received: $msg\n"; if ($msg =~ /start/) { threads->create(\&thread)->detach; } } sub thread { print "Thread started!\n"; sleep 5; }

    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority". I'm with torvalds on this
    In the absence of evidence, opinion is indistinguishable from prejudice. Agile (and TDD) debunked

      Same effect unfortunately, to confirm my code now looks like:

      #!/home/alerting/perl5/perlbrew/perls/perl-5.20.2/bin/perl use warnings; use IO::Handle; use threads stack_size => 64*1024; use strict; use DBI; use Email::MIME; use Email::Sender::Simple qw(sendmail); while (<STDIN>) { chomp(my $msg = $_); print "Received: $msg\n"; if ($msg =~ /start/) { threads->create(\&thread)->detach; } } sub thread { print "Thread started!\n"; sleep 5; }

      Memory usage still balloons, stops, then starts up again. Very puzzling, at least to me.

        I don't believe you are seeing a leak. I have to express it this way because I don't have *nix to test on; but running your code on my system, I see memory growth, but not a leak.

        To confirm my findings, tweak your code so:

        #!/home/alerting/perl5/perlbrew/perls/perl-5.20.2/bin/perl use warnings; use IO::Handle; use threads stack_size => 64*1024; use strict; use DBI; use Email::MIME; use Email::Sender::Simple qw(sendmail); while (<STDIN>) { threads->create(\&thread)->detach; sleep 5; } sub thread { print "Thread started!\n"; sleep 4; }

        Essentially what that does is limit your program to using only child thread at a time. (Not useful in the real world, but good for testing.).

        Run that and hold the enter key down while monitoring the memory usage and you should see an initial jump from circa 7MB to around 13MB, and then the memory usage should stay steady no matter how long you hold the key down for. Come back and let me know if that is true; and then I'll try and explain what you are seeing.


        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority". I'm with torvalds on this
        In the absence of evidence, opinion is indistinguishable from prejudice. Agile (and TDD) debunked
A reply falls below the community's threshold of quality. You may see it by logging in.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://1122762]
Approved by BrowserUk
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others meditating upon the Monastery: (4)
As of 2024-03-29 10:10 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found