comment on

It is possible I could try running it in Win-XP but it would take quite a bit of setup.

Hm. It's possible that the code could be adapted for use on Linux, but I do not have that expertise.

The basic idea is that I have a module (Devel::MemWatch) that hooks the DB::DB() interface and pushes the current caller information (package/file/line no) onto a Thread::Queue as each line executes. It has a setable threshhold value above which it discards old information as it pushes new, effectively turning the queue into a circular buffer of the last N lines executed.

It also starts a background thread that wakes up every setable N millseconds and queries the process memory size from the system. If the memory expands beyond a setable limit, it dumps the circular buffer to stderr; doubles the memory limit, and goes back to monitoring.

The idea is that you get a dump of the last N lines that executed when the memory expanded beyond your preset limit. (And again every time it doubles again). Thus, you can quickly get an idea of what code was executing when the memory started to snowball.

The (unbelievably crude but functional) Win32 code looks like this:

#! perl -slw
use strict;
package Devel::MemWatch;
use threads;
use Thread::Queue;
use Win32::API::Prototype;

our %OPTS = (
    LINES => 100,
    THRESHHOLD => 100 *1024,
    FREQUENCY => 2000,
);

sub import {
    for ( @_[ 1 .. $#_ ] ) {
        my( $key, $value ) = split '=';
        $OPTS{ $key } = $value || '1';
    }
    warn "@{[ %OPTS ]}\n";
}

ApiLink( 'Kernel32', q[
    HANDLE GetCurrentProcess( void )
] ) or die $^E;

ApiLink( 'Kernel32', q[
    BOOL GetProcessHandleCount( DWORD h, LPDWORD c)]
) or die $^E;

ApiLink( 'PSAPI', q[
    BOOL GetProcessMemoryInfo(
        HANDLE Process, LPVOID p, DWORD cb
    )
] ) or die $^E;

sub getProcessMemoryInfo {
    my $hProcess = GetCurrentProcess( [] );
    my $buf = pack 'L10', 40, (0) x 9;
    my $size = 40;

    if( GetProcessMemoryInfo( $hProcess, $buf, $size ) ){
#        warn __LINE__;
        my( @MemStats ) = unpack( "L10", $buf );
        my $memusage = int( $MemStats[3] / 1024 );
        my $peak_memusage = int( $MemStats[2] / 1024 );
        my $vmsize = int( $MemStats[8] / 1024 );
        return wantarray ? ( $memusage, $peak_memusage, $vmsize ) : $m
+emusage;
    }
    else {
        die "GPMI: $^E";
    }
}

my $Q;

BEGIN{ $Q = new Thread::Queue }

sub DB::DB {
#    warn caller;
    my( $package, $file, $line ) = caller;
    scalar $Q->dequeue if $Q->pending > $OPTS{ LINES };
    $Q->enqueue( "$package $file $line" );
    return;
}

sub _dump {
    warn $Q->dequeue() . "\n" while $Q->pending;
}

async {
    warn "Watchthread started\n";
    while( Win32::Sleep( $OPTS{ FREQUENCY } ) ) {
        my $mem = getProcessMemoryInfo();
#        warn "Watchthread awoke:$mem\n";
        if( $mem > $OPTS{ THRESHHOLD } ) {
            warn "Memsize: $mem\n";
            $OPTS{ THRESHHOLD } *= 2;
            _dump()
        }
    }
}->detach;

1;
[download]

Usage is:

perl -d:MemWatch=LINES=100,THRESHHOLD=100*1024,FREQUENCY=2000 yourscri
+pt.pl
[download]

Where

LINES: no. of caller traces held in the circular buffer.
THRESHHOLD is in Kbytes. (default 100MB)
FREQUENCY (at which the watchthread awakes) is in milliseconds. (Default:2 seconds)

The output looks like:

C:\test>perl -d:MemWatch=FREQUENCY=5000 junk9.pl
FREQUENCY 5000 LINES 100 THRESHHOLD 102400
Watchthread started
Memsize: 103156
main junk9.pl 8
main junk9.pl 9
main junk9.pl 9
main junk9.pl 8
main junk9.pl 9
main junk9.pl 9
...
[download]

Maybe you can adapt it to your needs?

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

"Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."

In reply to Re^3: Getting a memory dump by BrowserUk
in thread Getting a memory dump by awy

Are you posting in the right place? Check out Where do I post X? to know for sure.
Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
<code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
Want more info? How to link or How to display code and escape characters are good places to start.


Do you know where your variables are?
	PerlMonks