Beefy Boxes and Bandwidth Generously Provided by pair Networks
Syntactic Confectionery Delight
 
PerlMonks  

Analyzing Apache memory usage

by Azhrarn (Friar)
on Jul 18, 2008 at 15:09 UTC ( [id://698643]=sourcecode: print w/replies, xml ) Need Help??
Category: Utility Scripts
Author/Contact Info Azhrarn
Description: After a day of trying to figure out why one of my web servers was locking up, I found that it was using a bit too much memory. But I had no idea how much, and Linux memory reporting is a bit arcane at best. Especially with something like apache + mod_perl/php using shared memory pools. So after some analysis, I came up with the included script.

It has the following output:


Total memory available: 3.21G
Total used by apache2 (451 instances): 3.80G
Total used by other processes: 0.12G

Average memory used per apache2 process: 8.63M
Recommended number of processes based on Average: 381
Needed memory for 500 processes based on Average: 4.21G

Max memory used for apache2 process: 17.61M
Recommended number of processes based on Max: 186
Needed memory for 500 processes based on Max: 8.60G

Mean plus two Standard Deviations (bulk of usage under max): 11.28M
Recommended number of processes based on Mean + 2*Stdev: 291
Needed memory for 500 processes based on Mean + 2*Stdev: 5.51G


At the time I ran it, I had 450 instances out of a configured max of 700 active. Come to find out I only had the ram to support about 380 of those at best. :(

Hopefully some other people find this useful, as it's pretty hard to get a straight answer on "How many processes can I support?" Or even quickly analyze how much memory a threaded process with shared memory is using. Note I don't bother to include the root apache process, or the negligible amount of base shared memory.

This can also be applied to other arbitrary processes by changing the name it scans for. It is also dependant on the "top" utility, as included in Debian. I'm not sure how it will act under other variations of top, although it should be fairly robust at pulling out the column names.
#!/usr/bin/perl
# Copyright Erik Jacobson - erik@underhanded.org

use warnings; use strict;
use Statistics::Descriptive;

my $webuser = 'www-data'; # The user your apache children run as.  I i
+gnore the root process.
my $command = 'apache2'; # The name of your processes as they appear t
+o the "top" command.

my @top = `top -bn1`;
my %header;
my $mem = Statistics::Descriptive::Full->new();
my $othermem = Statistics::Descriptive::Full->new();
my $totalmem;

foreach (@top)
{
        s/^\s+|\s+$//sg;
        my @line = split(/\s+/, $_);

        if(/mem:\s+(\S+)\s+total/i)
        {
                $totalmem = convtok($1);
        }

        if(defined $line[3] && exists $header{res} && $line[0] =~ /^\d
++$/)
        {
                next unless (defined $line[$header{user}] && defined $
+line[$header{res}]
                        && defined $line[$header{shr}] && defined $lin
+e[$header{command}]);

                if(($line[$header{user}] eq $webuser) && ($line[$heade
+r{command}] eq $command))
                {
                        $mem->add_data(convtok($line[$header{res}]) - 
+convtok($line[$header{shr}]));
                } else {
                        $othermem->add_data(convtok($line[$header{res}
+]) - convtok($line[$header{shr}]));
                }
        } elsif (!exists $header{res})
        {
                my %tempheader;
                for (0 .. $#line)
                {
                        $tempheader{lc($line[$_])} = $_;
                }

                if(defined $tempheader{user} && defined $tempheader{re
+s}
                        && defined $tempheader{shr} && defined $temphe
+ader{command})
                {
                        %header = %tempheader;
                }
        }
}

printf "Total memory available: %.2fG\n", ($totalmem / 1024 / 1024);
printf "Total used by $command (%d instances): %.2fG\n", $mem->count()
+, ($mem->sum() / 1024 / 1024);
printf "Total used by other processes: %.2fG\n\n", ($othermem->sum() /
+ 1024 / 1024);

$totalmem -= $othermem->sum();

printf "Average memory used per $command process: %.2fM\n", ($mem->mea
+n() / 1024);
printf "Recommended number of processes based on Average: %d\n", ($tot
+almem / $mem->mean());
printf "Needed memory for 500 processes based on Average: %.2fG\n\n", 
+($mem->mean() / 1024 / 1024 * 500);

printf "Max memory used for $command process: %.2fM\n", ($mem->max() /
+ 1024);
printf "Recommended number of processes based on Max: %d\n", ($totalme
+m / $mem->max());
printf "Needed memory for 500 processes based on Max: %.2fG\n\n", ($me
+m->max() / 1024 / 1024 * 500);

printf "Mean plus two Standard Deviations (bulk of usage under max): %
+.2fM\n", (($mem->mean() + $mem->standard_deviation() * 2) / 1024);
printf "Recommended number of processes based on Mean + 2*Stdev: %d\n"
+, ($totalmem / ($mem->mean() + $mem->standard_deviation() * 2));
printf "Needed memory for 500 processes based on Mean + 2*Stdev: %.2fG
+\n", (($mem->mean() + $mem->standard_deviation() * 2) / 1024 / 1024 *
+ 500);

sub convtok
{
        my $num = shift;
        if($num =~ /^\s*([0-9.]+)k\s*$/i)
        {
                $num = $1;
        } elsif ($num =~ /^\s*([0-9.]+)m\s*$/i)
        {
                $num = $1 * 1024;
        } elsif ($num =~ /^\s*([0-9.]+)g\s*$/i)
        {
                $num = $1 * 1024 * 1024;
        }

        return $num;
}
Replies are listed 'Best First'.
Re: Analyzing Apache memory usage
by zentara (Archbishop) on Jul 18, 2008 at 15:31 UTC
    I couldn't tell by your code whether it updates constantly, but you may be interested in linux memory leak monitor. If you have an xserver running on the apache machine, it will open a little Tk window in the bottom right corner, and update it every second. That way you can watch it over the day, just with a glance.

    If you need to run commandline, you could run your script under POE or even Roll your own Event-loop where you can setup a Glib timer. (Of course, you can also put it in a simple loop where you run it, then sleep 5) ;-)

    But feel free to modify my MeM script, and add all your info into the Tk Label.


    I'm not really a human, but I play one on earth CandyGram for Mongo
Re: Analyzing Apache memory usage
by Anonymous Monk on Sep 22, 2020 at 09:36 UTC
    The script is a little old but still works. However, on boxes with very large amounts of RAM (100G+), top doesn't give an answer in K anymore and the script breaks. I took out the if statement in the main loop that grabs the memory and replaced it with:
    my $free= `free|fgrep Mem`; if($free=~/Mem\s*:\s+(\S+)\s+/i) { $totalmem = convtok($1); } else { die("Can't find memory info\n"); }
    just above the foreach and that worked for our 680GB ram monster.
Re: Analyzing Apache memory usage
by Anonymous Monk on Oct 29, 2011 at 20:25 UTC
    Thank-you. Looks like our Apache is loading far too much:

    ./apache.memory.analyse.pl Total memory available: 0.49G Total used by apache2 (3 instances): 0.73G Total used by other processes: 0.44G Average memory used per apache2 process: 248.45M Recommended number of processes based on Average: 0 Needed memory for 500 processes based on Average: 121.31G Max memory used for apache2 process: 258.87M Recommended number of processes based on Max: 0 Needed memory for 500 processes based on Max: 126.40G Mean plus two Standard Deviations (bulk of usage under max): 270.93M Recommended number of processes based on Mean + 2*Stdev: 0 Needed memory for 500 processes based on Mean + 2*Stdev: 132.29G
Re: Analyzing Apache memory usage
by Anonymous Monk on Oct 29, 2011 at 19:55 UTC
    Useful script. People who stumble upon this .pl, but do not know perl, will copy and paste this, end up with line breaks in the wrong place, and then never use the script ;)
      They should press on the Download button to download a clean copy of the script.
Re: Analyzing Apache memory usage
by Anonymous Monk on Oct 29, 2011 at 20:10 UTC
    # ./apache.memory.analyse.pl Total memory available: 0.49G Total used by apache2 (0 instances): 0.00G Total used by other processes: 0.00G Average memory used per apache2 process: 0.00M Illegal division by zero at ./apache.memory.analyse.pl line 61.
      This is a copy and paste error. Use the Download button to download a clean copy of the script.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: sourcecode [id://698643]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others studying the Monastery: (5)
As of 2024-04-24 00:30 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found