Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling
 
PerlMonks  

Re: Looking for Leaks in all the wrong places

by Matts (Deacon)
on Jun 09, 2002 at 20:21 UTC ( #172956=note: print w/replies, xml ) Need Help??


in reply to Looking for Leaks in all the wrong places

This is a tough one. Finding memory leaks in Perl code is *really* hard. Unfortunately Devel::Leak (and Apache::Leak which is based on the former) are next to no use whatsoever, because they tend to tell you only that you have a leak, not what caused it.

You may find some wisdom in the mod_perl guide section on Debugging. But I wouldn't hold out too much hope there.

My personal solution to detecting memory leaks in mod_perl scripts (and which I used with good success in finding memory leaks in AxKit, is to use what I guess is a simple step through system, with a slight difference.

First of all it's important to note how AxKit is structured (and try and apply it to your code if you can).

In AxKit, the entire request is wrapped in an eval{} block. At the end of the eval block if I detect an error I can handle it appropriately, depending on what kind of error the user has and what configuration is set. But don't worry too much about that - the important thing is the eval{} block. Even if you don't write the error handling bit, you can debug this just by using that eval{} block and ignoring that your script might return something invalid for now.

Now in order to debug my memory leaks, I first find a URI that causes the leak on every request. Then I set apache running in single thread mode (/path/to/apache/bin/httpd -X). Then I hit the page with apachebench (/path/to/apache/bin/ab). I check the process is leaking by watching it in another window running "top". Following so far? Good.

Now, I add a simple: die "Leak test"; to the start of my eval{} block (just inside the opening curly bracket). Stop/start apache (still in single thread mode). And hit it again with apachebench. Your request should not leak here. If it does, you know the problem is in your eval handling code or before you got to your die().

Now move your "die" further into your code, following the path you know this request takes. Use fairly coarse steps at first, and when you get into the leak, step back in single lines.

I guarantee you that eventually you will discover where your leak is.

Now of course fixing it, that's another matter... The things that are the worst culprits for memory leaks in perl code are:

  • Closures - this is a particular concern for mod_perl since the closure will sometimes remain active for the lifetime of the process. Fixing closure related memory leaks requires an understanding of what a closure is and what it's for and how and why it holds onto an instance of your variable.
  • Circular references - these nasty little buggers mean your variables never get freed. There are techniques for avoiding them, and almost always they are unnecessary.
  • Perl bugs - yes there are such things. Like for example in 5.00503, the following code (which really is a think-o on my part for writing it in the first place as it makes no real sense) leaked like a seive:
    my @foo = get_list() if $condition;
    (or it was something like it - I don't fully remember the exact details now, they are lost in the anals of axkit history)
  • XS modules - sometimes an author of an XS module will forget to free some memory, or mess up the refcount of a variable. It's very easy to do - I should know ;-)

I hope that's some help. I feel deep sympathy for you as finding memory leaks is no fun at all. Good luck!

Replies are listed 'Best First'.
Re: Re: Looking for Leaks in all the wrong places
by Elian (Parson) on Jun 10, 2002 at 01:44 UTC
    It should be noted that closures fall into the perl bugs section. They leak no matter what you do. Perl through 5.6.1 has some leaks that there's just not much you can do about, unfortunately, short of avoiding the conditions that cause them.

    5.8.0 should be much better in this regard, though it's not quite out yet.

Re: Re: Looking for Leaks in all the wrong places
by Stegalex (Chaplain) on Jun 09, 2002 at 21:01 UTC
    Thanks Matts! And hey, I really enjoy using all your XML stuff too.

    The process for identifying memory leaks looks like a real pain in the butt. After thinking about it for a while, I would be perfectly happy to just automatically reboot the server after CPU / Memory are depleted to a certain point.

    So my new question is: What combination of CPU utilization / memory paucity should I use as the trigger to automatically reboot?

    My system is a single processor Linux box with 750M of memory running Apache and Oracle with mod_perl.

    ~~~~~~~~~~~~~~~
    I like chicken.
      The process for identifying memory leaks looks like a real pain in the butt. After thinking about it for a while, I would be perfectly happy to just automatically reboot the server after CPU / Memory are depleted to a certain point.
      this is a really bad attitude to take towards your code. not only has matts given you great code which you compliment, but he's given you an outline for advanced debugging techniques he used in the same code you praise.

      instead of taking the opportunity to learn more about bugs that might be in your code (and software development in general) from someone who took the time to provide an detailed answer your question, you've decided to throw away his proven advice.

      if you feel his advice is over your head, and you would like to learn but don't think you're capable yet, say so. i'm sure other suggestions can be offered, including advice on your follow-up question. as it stands, your dismissive tone leaves me unsympathetic to your current situation.

      why would anyone here want to provide you with help in the future if you openly throw away such good advice?

      i certainly wouldn't.

      ~Particle *accelerates*

      A reply falls below the community's threshold of quality. You may see it by logging in.
      You should *never* need to reboot the whole server - instead set MaxRequestsPerChild low (axkit.org has it at 100), and install something like Apache::SizeLimit to make sure you don't get one particular process going totally out of control.
      I agree with particle on this one. Who is to say that tomorrow you won't write another script with exactly the same problem your current one has? In that case, your server resources will become depleted twice as fast without an end in sight. Also, why be satisfied with code that you know is broken?

      thor

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://172956]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others cooling their heels in the Monastery: (3)
As of 2023-02-06 05:21 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    I prefer not to run the latest version of Perl because:







    Results (33 votes). Check out past polls.

    Notices?