|
Re: Strange memory leak using just threads
by Corion (Patriarch) on Sep 19, 2010 at 20:17 UTC
|
This might be the same that happens in Perl bug #77854, as detailed on Google Groups. Maybe 5.10 has a bug there, or it depends on libpthreads. dave_the_m commented that he can't reproduce the problem on 5.10 or 5.12, so it might be interesting to see whether you can reproduce the problem on 5.12 (or bleadperl) too.
| [reply] |
|
|
My libpthreads seems to be 2.7 (or, at least, the libc6 package it is contained in is 2.7. Specifically, 2.7-18lenny4).
I'll give 5.12 a shot and see what happens.
| [reply] |
|
|
I downloaded perl 5.12.2, and I'm still experiencing the memory leak. This is using the threads 1.75 that came with the distribution, not the 1.77 I installed for 5.10.
Here are the options I used with configure:
-des -Dprefix=$HOME/perl512 -Dusethreads -Duselargefiles -Dccflags=-
+DDEBIAN -Dcccdlflags=-fPIC -Darchname=x86_64-linux-gnu -Doptimize=-O2
+ -Duseshrplib -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Ud_ua
+larm -Uusesfio -Uusenm -DDEBUGGING=-g
| [reply]
[d/l] |
Re: Strange memory leak using just threads
by BrowserUk (Patriarch) on Sep 19, 2010 at 20:47 UTC
|
Which version of threads?
I don't see any leak using perl 5.010001 and threads: 1.76.
| [reply] |
|
|
My thread version seems to be 1.67. I upgraded to 1.77, and the problem still occurs.
I did add print $threads::VERSION,"\n"; so I was sure it was using the correct module.
I also tried it on another system with 5.8.8 installed and threads 1.07, and got the same result.
| [reply]
[d/l] |
|
|
This would appear to be a platform specific thing. Not particularly helpful, but it might allow you to look for a solution in the right place; eg. outside of Perl/threads.
Perhaps there is a later version of the pthreads libraries for your system? Or known bugs within the version you have?
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
| [reply] |
|
|
Re: Strange memory leak using just threads
by zentara (Archbishop) on Sep 20, 2010 at 10:58 UTC
|
As far as I know, this sort of memory leak is very common in Perl when using threads. The only solution that I know I can count on is to reuse threads, and avoid the create-destroy cycle which leaks memory. See Reusable threads demo I see the small leak also on my 64 bit slackware with the latest Perl 5.12.2. From what I've been able to figure out, the leak comes from Perl's memory management which holds onto memory because there are ref counts there. See does threads (still) memleak? and Memory and garbage collection and Memory leaks and reference counting within Perl. for examples of previous discussion..... its all been talked about before.
| [reply] |
|
|
#! /usr/bin/perl
use strict;
use warnings;
use threads;
print "threads: ", $threads::VERSION;
print "perl: ", $];
my @child;
my @list = 1 .. 100;
while (1) {
my @child = map threads->create("test","$_"), @list;
$_->join for @child;
}
sub test {
my ($item) = @_;
}
And the memory usage is rock steady. And that has been true for a long time.
So, what does that tell you?
Your knowledge is out of date. Your platform--be it the pthreads or memory management or whatever--has an underlying problem that cannot be directly addressed by, nor attributed to, Perl. So stop doing that.
BTW: Your "simple reusable threads demo", is way overcomplicated. And always was.
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
| [reply]
[d/l] |
|
|
And the memory usage is rock steady. And that has been true for a long time.
So, what does that tell you?
Your knowledge is out of date. Your platform--be it the pthreads or memory management or whatever--has an underlying problem that cannot be directly addressed by, nor attributed to, Perl. So stop doing that.
So what does that tell me? First, his script gained memory and your's didn't. I observe the same behavior. So a script written 1 way in Perl may leak, and written another may not. Furthermore, these 2 sample thread scripts are the ultimate in simplicity. Leakage problems become more pronounced when you start adding objects to threaded programs.
I have the latest Slackware, and before I believe my slackware c libs are faulty, I have to assume Perl's behavior is somehow causing it. Since this memory leak problem keeps coming up, all I can do is to mention that thread reuse does prevent the problem. But you are more of a thread expert than me, so if you insist that perl threads do not have memory problems, and it is the underlying c library's fault, I call that a cop out. If threads have no problems with ref counts and memleaks, why did you post threads: spawn early to avoid the crush.? Or is that information from you already outdated?
| [reply] |
|
|
|
|
|
|
|
| [reply] |
|
|
|
|
|
