http://qs321.pair.com?node_id=957527

bloonix has asked for the wisdom of the Perl Monks concerning the following question:

Hi Monks, I'm trying to understand the garbage collector. Please have a look at my code example:

#!/usr/bin/perl
use strict;
use warnings;

package Foo;

sub new {
    my $self = bless { }, shift;
    for (1..2_000_000) {
        $self->{x} .= "x" x 1000;
    }
    return $self;
}

package Bar;

sub new {
    my $self = bless { }, shift;
    for (1..2_000_000) {
        $self->{$_} = "x" x 1000;
    }
    return $self;
}

package main;

print "### Test Foo\n";
my $foo = Foo->new;

print "Object created\n";
print "Size: ", qx{ps -p $$ -o rss=};

$foo = undef;
sleep 3;

print "Object destroyed\n";
print "Size: ", qx{ps -p $$ -o rss=};

print "### Test Bar\n";
my $bar = Bar->new;

print "Object created\n";
print "Size: ", qx{ps -p $$ -o rss=};

$bar = undef;
sleep 3;

print "Object destroyed\n";
print "Size: ", qx{ps -p $$ -o rss=};

[download]
Output: 
### Test Foo
Object created
Size: 1955472
Object destroyed
Size:  2348
### Test Bar
Object created
Size: 2647552
Object destroyed
Size: 2631164

The Foo example is that what I expect. But I don't understand why the memory is not releases back to the operating system in the Bar example.

Could someone bring some light into my darkness?

Cheers, Jonny

Replies are listed 'Best First'.
Re: Memory is not released back to operating system
by educated_foo (Vicar) on Mar 02, 2012 at 18:33 UTC
    Sorry, that's just the way your malloc() works. Some mallocs use mmap() for large allocations, to at least give the process a chance to return memory to the OS, but most hold onto memory until the process exits.
[reply]
      I thought perl specifically tries to hold onto memory so it can reuse it again instead of having to allocate new memory?

      Elda Taluta; Sarks Sark; Ark Arks
      My deviantART gallery
[reply]

        No, it defers to the implementation of the C runtime library malloc.

        Though I wouldn't recommend it unless you are desperate, it is possible to change the "release memory to the OS" behaviour by building Perl using a custom malloc.

        This has been discussed here many times before, see for example: Not able to release memory and How to return unused memory to OS?.

        From my testing across a number of OSes, I would expect only Linux and Windows to return "large" blocks of memory back to the OS. I would be interested to learn what operating system the OP is running on.

[reply]
Re: Memory is not released back to operating system
by zentara (Archbishop) on Mar 02, 2012 at 19:27 UTC
    To be absolutely sure your memory gets released back when you want, fork the code off. See Re: undefining hashes to free memory for more info on non-forked problems with memory. Otherwise, the Perl interpreter will greedily hang on to the memory, figuring if you used it once, you probably will want to use it again.
    I'm not really a human, but I play one on earth.
    Old Perl Programmer Haiku ................... flash japh
[reply]

      Forking... that is exactly my problem.

      My daemon process needs a lot of memory (~2-8GB) for different data. Every time the daemon forks some new processes to distribute its work, each child process has the same size, but each child process needs only a lower part (~20MB) of the data. I thought it could be possible at first to destroy the object and release the memory back to the OS and then send the lower part over a pipe to the child process. The idea was really bad.

      > fork the code off

      I could fork the memory off.

      I think I have no other chance as to fork a process that holds the data and distribute the workload to other child processes. The main daemon just managing the forking machine without the huge amount of data.

      Thanks for the inspiration!

[reply]

        The usual implementation is to have a small controller section of code that forks off the workload onto its child processes. Try to keep you memory footprint of the controller as small as possible. Both the controller section and the worker section are often included in the same binary or script, so that if you have one, you have everything you need to run.

        If you are writing a true daemon, have it die and restart periodically. This will get rid of unused memory, open files and sockets, and zombie children that were never harvested. This is usually done by having an entry in the appropriate /etc/rc file.

[reply]
[d/l]
Re: Memory is not released back to operating system
by JavaFan (Canon) on Mar 02, 2012 at 20:33 UTC
    Note that the layout of the used memory will be quite different. Your Foo object will use a single piece of 2 Gb of memory - while the Bar object will use 2 million pieces of about 1 kb. That will come from different pools, both on Perl, as on the OS level. And will be treated differently.
[reply]

Back to Seekers of Perl Wisdom