Open, While and memory management

matthew has asked for the wisdom of the Perl Monks concerning the following question:

Hi all,
I was recently writing a log parser and discoverd a large memory leak in my code. When I found the problem I was rather surprised. Hopefully someone can explain the details to me.

Here's where the leak was:

while (my $line = <FILE>) { ... }
[download]

I fixed it with the following:

while (<FILE>) { my $line = $_; ... }
[download]

Apparently in the first example a new block of memory was being allocated to each $line, but was never reclaimed. I was unable to determine whether perl ever reclaimed the memory, reclaimed after the end of the loop, or at the end ot the process. According to top (on FreeBSD) the memory was never reclaimed, but that could just be a peculiarity of how top reports memory usage.

I always thought the above two lines were functionally and lexically identical. Can anyone explain why they behave differently?

Thanks,
-M

Comment on Open, While and memory management Select or Download Code

Replies are listed 'Best First'.
(tye)Re: Open, While and memory management by tye (Sage) on Oct 17, 2001 at 01:02 UTC
I recall this coming up a long time ago here. The problem is that you end up creating a new $line lexical each time through the loop and these don't get free()d until you exit the loop. So the leak is temporary. Based on other parts of the thread, this behavior may be fixed in newer versions of Perl. - tye (but my friends call me "Tye")	[reply]
Re: Open, While and memory management by nardo (Friar) on Oct 16, 2001 at 23:31 UTC
I can not recreate this. When testing with a 1 million line logfile both perl5.005_03 and perl5.6.0 use 4k more memory for the first example than the second. When you try removing code from within the while loop do you still experience the leak? If not, you may try adding code until the leak is back then taking a closer look at that.	[reply]
Re: Re: Open, While and memory management by matthew (Acolyte) on Oct 16, 2001 at 23:38 UTC
It occurred on FreeBSD 3.2 with perl 5.005_03. The leak still occurred with all code within the loop removed.	[reply]
Re: Open, While and memory management by Aristotle (Chancellor) on Oct 17, 2001 at 07:05 UTC
They are not exactly equivalent, in that the latter will read into $_, destroying any previous content. You would need to prepend the "local $_;" mantra to make them indistinguishable from calling code's point of view (though their precise behavior will still differ of course). Anyway, on your question, I seem to recall word being that my-variables used directly in the loop construct (ie while(my $var) and for my $var) are scoped into the loop's block's context, but allocated in the surrounding block's frame. In conjunction with them apparently not being deallocated at the loop's end but rather at the surrounding block's end in older 5.5.x versions of Perl, that would explain why your code "leaks".	[reply]


Don't ask to ask, just ask
	PerlMonks