in reply to Memory is not released back to operating system

To be absolutely sure your memory gets released back when you want, fork the code off. See Re: undefining hashes to free memory for more info on non-forked problems with memory. Otherwise, the Perl interpreter will greedily hang on to the memory, figuring if you used it once, you probably will want to use it again.

I'm not really a human, but I play one on earth.
Old Perl Programmer Haiku ................... flash japh
  • Comment on Re: Memory is not released back to operating system

Replies are listed 'Best First'.
Re^2: Memory is not released back to operating system
by bloonix (Monk) on Mar 03, 2012 at 00:22 UTC

    Forking... that is exactly my problem.

    My daemon process needs a lot of memory (~2-8GB) for different data. Every time the daemon forks some new processes to distribute its work, each child process has the same size, but each child process needs only a lower part (~20MB) of the data. I thought it could be possible at first to destroy the object and release the memory back to the OS and then send the lower part over a pipe to the child process. The idea was really bad.

    > fork the code off

    I could fork the memory off.

    I think I have no other chance as to fork a process that holds the data and distribute the workload to other child processes. The main daemon just managing the forking machine without the huge amount of data.

    Thanks for the inspiration!

      The usual implementation is to have a small controller section of code that forks off the workload onto its child processes. Try to keep you memory footprint of the controller as small as possible. Both the controller section and the worker section are often included in the same binary or script, so that if you have one, you have everything you need to run.

      If you are writing a true daemon, have it die and restart periodically. This will get rid of unused memory, open files and sockets, and zombie children that were never harvested. This is usually done by having an entry in the appropriate /etc/rc file.