Valgrind, Perl, and banishing leaks

creamygoodness has asked for the wisdom of the Perl Monks concerning the following question:

Greets,

Before each release of KinoSearch, which is an XS distro, I run the test suite under Valgrind, logging the results. Eyeballing the log, I can verify that at least the ERROR SUMMARY for each test is 0.

Checking for leaks is tougher. I have a debugging perl compiled, and I set the PERL_DESTRUCT_LEVEL environment variable to 2, as per the instructions in perlhack. However, perl still seems to have some leaks no matter what.

Here's plain perl:

$ valgrind --leak-check=full perl -e1
==5030== Memcheck, a memory error detector.
==5030== Copyright (C) 2002-2005, and GNU GPL'd, by Julian Seward et a
+l.
==5030== Using LibVEX rev 1471, a library for dynamic binary translati
+on.
==5030== Copyright (C) 2004-2005, and GNU GPL'd, by OpenWorks LLP.
==5030== Using valgrind-3.1.0, a dynamic binary instrumentation framew
+ork.
==5030== Copyright (C) 2000-2005, and GNU GPL'd, by Julian Seward et a
+l.
==5030== For more details, rerun with: -v
==5030== 
==5030== 
==5030== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 21 from 
+1)
==5030== malloc/free: in use at exit: 46,178 bytes in 549 blocks.
==5030== malloc/free: 654 allocs, 105 frees, 53,220 bytes allocated.
==5030== For counts of detected errors, rerun with: -v
==5030== searching for pointers to 549 not-freed blocks.
==5030== checked 325,632 bytes.
==5030== 
==5030== 1,211 (104 direct, 1,107 indirect) bytes in 1 blocks are defi
+nitely lost in loss record 1 of 3
==5030==    at 0x401A619: malloc (vg_replace_malloc.c:149)
==5030==    by 0x8098B14: Perl_safesysmalloc (in /usr/local/bin/perl)
==5030==    by 0x809AA63: Perl_my_setenv (in /usr/local/bin/perl)
==5030==    by 0x8060011: perl_parse (in /usr/local/bin/perl)
==5030==    by 0x805E417: main (in /usr/local/bin/perl)
==5030== 
==5030== LEAK SUMMARY:
==5030==    definitely lost: 104 bytes in 1 blocks.
==5030==    indirectly lost: 1,107 bytes in 24 blocks.
==5030==      possibly lost: 0 bytes in 0 blocks.
==5030==    still reachable: 44,967 bytes in 524 blocks.
==5030==         suppressed: 0 bytes in 0 blocks.
==5030== Reachable blocks (those to which a pointer was found) are not
+ shown.
==5030== To see them, rerun with: --show-reachable=yes
$
[download]

... and here's the debugging perl:

$ valgrind --leak-check=full /usr/local/debugperl/bin/perl -e1
==5028== Memcheck, a memory error detector.
==5028== Copyright (C) 2002-2005, and GNU GPL'd, by Julian Seward et a
+l.
==5028== Using LibVEX rev 1471, a library for dynamic binary translati
+on.
==5028== Copyright (C) 2004-2005, and GNU GPL'd, by OpenWorks LLP.
==5028== Using valgrind-3.1.0, a dynamic binary instrumentation framew
+ork.
==5028== Copyright (C) 2000-2005, and GNU GPL'd, by Julian Seward et a
+l.
==5028== For more details, rerun with: -v
==5028== 
==5028== 
==5028== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 21 from 
+1)
==5028== malloc/free: in use at exit: 127 bytes in 16 blocks.
==5028== malloc/free: 644 allocs, 628 frees, 74,906 bytes allocated.
==5028== For counts of detected errors, rerun with: -v
==5028== searching for pointers to 16 not-freed blocks.
==5028== checked 288,268 bytes.
==5028== 
==5028== 127 bytes in 16 blocks are definitely lost in loss record 1 o
+f 1
==5028==    at 0x401A619: malloc (vg_replace_malloc.c:149)
==5028==    by 0x80B1EFE: Perl_safesysmalloc (util.c:78)
==5028==    by 0x80B3343: Perl_savepvn (util.c:789)
==5028==    by 0x8068DD9: Perl_gv_fetchpv (gv.c:754)
==5028==    by 0x8065045: S_init_main_stash (perl.c:3510)
==5028==    by 0x8061650: S_parse_body (perl.c:1657)
==5028==    by 0x8061465: perl_parse (perl.c:1598)
==5028==    by 0x805E9E8: main (perlmain.c:97)
==5028== 
==5028== LEAK SUMMARY:
==5028==    definitely lost: 127 bytes in 16 blocks.
==5028==      possibly lost: 0 bytes in 0 blocks.
==5028==    still reachable: 0 bytes in 0 blocks.
==5028==         suppressed: 0 bytes in 0 blocks.
==5028== Reachable blocks (those to which a pointer was found) are not
+ shown.
==5028== To see them, rerun with: --show-reachable=yes
$
[download]

Looks like symbol table stuff, and it's almost certainly harmless, but it would be nice if it were not there. If all those numbers were zero, then when any of my tests trigger a non-zero I would know that's a red flag. As it is, I have to look for patterns. Big stuff is easy, but small stuff might sneak by.

Anybody else do this kind of debuggery? Do you see the same things?

--
Marvin Humphrey
Rectangular Research ― http://www.rectangular.com

Comment on Valgrind, Perl, and banishing leaks Select or Download Code

Replies are listed 'Best First'.
Re: Valgrind, Perl, and banishing leaks by eyepopslikeamosquito (Archbishop) on May 19, 2006 at 10:01 UTC
In addition to setting `PERL_DESTRUCT_LEVEL=2` you should also set `PERL_DEBUG=1` and `PERL_VALGRIND=1`. Last year, I asked P5P about a valgrind memory leak I was seeing but noone responded. In my test case: `PERL_DEBUG=1 PERL_VALGRIND=1 PERL_DESTRUCT_LEVEL=2 valgrind --leak-check=full perl -e 'sub k {1}'` [download] produced no memory leak report from valgrind, while: `PERL_DEBUG=1 PERL_VALGRIND=1 PERL_DESTRUCT_LEVEL=2 valgrind --leak-check=full perl -e 'sub k () {1}'` [download] produced: ==4943== Memcheck, a memory error detector for x86-linux. ==4943== Copyright (C) 2002-2005, and GNU GPL'd, by Julian Seward et a +l. ==4943== Using valgrind-2.4.0, a program supervision framework for x86 +-linux. ==4943== Copyright (C) 2000-2005, and GNU GPL'd, by Julian Seward et a +l. ==4943== For more details, rerun with: -v ==4943== ==4943== ==4943== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 17 from +1) ==4943== malloc/free: in use at exit: 3 bytes in 1 blocks. ==4943== malloc/free: 775 allocs, 774 frees, 84102 bytes allocated. ==4943== For counts of detected errors, rerun with: -v ==4943== searching for pointers to 1 not-freed blocks. ==4943== checked 282624 bytes. ==4943== ==4943== 3 bytes in 1 blocks are definitely lost in loss record 1 of 1 ==4943== at 0x1B909222: malloc (vg_replace_malloc.c:130) ==4943== by 0x80C698C: Perl_safesysmalloc (util.c:78) ==4943== by 0x80C8152: Perl_savepv (util.c:774) ==4943== by 0x80A5AE6: Perl_newCONSTSUB (op.c:4554) ==4943== by 0x80A4994: Perl_newATTRSUB (op.c:4305) ==4943== by 0x8097207: Perl_yyparse (perly.y:364) ==4943== by 0x806433E: S_parse_body (perl.c:1844) ==4943== by 0x8062FBA: perl_parse (perl.c:1308) ==4943== by 0x805F658: main (perlmain.c:96) ==4943== ==4943== LEAK SUMMARY: ==4943== definitely lost: 3 bytes in 1 blocks. ==4943== possibly lost: 0 bytes in 0 blocks. ==4943== still reachable: 0 bytes in 0 blocks. ==4943== suppressed: 0 bytes in 0 blocks. ==4943== Reachable blocks (those to which a pointer was found) are not + shown. ==4943== To see them, rerun with: --show-reachable=yes [download] BTW, I'm eager to improve my skills in this area, so if anyone knows some useful links, please let me know. Update: At the time I wrote this, I didn't realise that functions containing an empty prototype are candidates for inlining (see "Constant Functions" section in perlsub for more details).	[reply] [d/l] [select]
Re^2: Valgrind, Perl, and banishing leaks by creamygoodness (Curate) on May 19, 2006 at 15:42 UTC
Setting `PERL_VALGRIND=1` and `PERL_DEBUG=1` didn't change my results. My debugperl is built from the 5.8.8 source; grepping both that and bleadperl's source for `VALGRIND` didn't turn up any change in behavior; it looks like the only thing `PERL_VALGRIND` does is turn on some tests. A similar grep for `PERL_DEBUG` was produced more voluminous output which was harder to scour; nothing that would change memory management jumped out at me, though maybe I missed something. In contrast, `PERL_DESTRUCT_LEVEL` affects `perl_destruct` in perl.c rather dramatically. Perhaps the system that this perl is built on (RedHat 9) has something to do with our divergent results. BTW, I'm eager to improve my skills in this area, so if anyone knows some useful links, please let me know. If you're still following p5p, you may have seen this, but there was a thread recently on running Perl's test suite under Valgrind -- Marvin Humphrey Rectangular Research ― http://www.rectangular.com	[reply] [d/l] [select]


XP is just a number
	PerlMonks