Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

FFI::Platypus: Replace malloc with GC_MALLOC?

by karlgoethebier (Abbot)
on Dec 11, 2022 at 18:28 UTC ( [id://11148746]=perlquestion: print w/replies, xml ) Need Help??

karlgoethebier has asked for the wisdom of the Perl Monks concerning the following question:

Hi all, please consider this example from FFI::Platypus:

#include <string.h> #include <stdlib.h> const char * string_reverse(const char *input) { static char *output = NULL; int i, len; if(output != NULL) free(output); if(input == NULL) return NULL; len = strlen(input); output = malloc(len+1); for(i=0; input[i]; i++) output[len-i-1] = input[i]; output[len] = '\0'; return output; }
use FFI::Platypus 2.00; my $ffi = FFI::Platypus->new( api => 2, lib => './string_reverse.so', ); $ffi->attach( string_reverse => ['string'] => 'string' ); print string_reverse("\nHello world"); string_reverse(undef);

Note the second call of string_reverse. From ibidem:

"At the end of the program we call reverse_string with undef, which gets translated to C as NULL. This allows it to free the output buffer so that the memory will not leak."

A smart aleck might jump to the conclusion to replace malloc with GC_MALLOC as described here.

Is this recommended or will it cause problems in the future not yet imaginable now?

Thanks for any hint and best regards, Karl

P.S.: See also Boehm–Demers–Weiser Garbage Collector

Update: Thank you all for the kind and very informative responses. I now consider this a research project for the future. So far I don't even know if the mentioned library compiles on the Mac. I will report back.

«The Crux of the Biscuit is the Apostrophe»

Replies are listed 'Best First'.
Re: FFI::Platypus: Replace malloc with GC_MALLOC?
by eyepopslikeamosquito (Archbishop) on Dec 12, 2022 at 01:48 UTC

    What I enjoy about Perl and C++ (and don't enjoy about Java ;-) is Deterministic Destructors. Of course, a drawback to Deterministic Destructors that Java-like tracing GCs solve is the dreaded Circular Reference problem.

    I've not answered your question because I don't understand it (though I added it my Deterministic Destructor References :). Perhaps you could give us some background about the "why" behind your question. What specific problem are you trying to solve?

      «…"why"…»

      The background is curiosity and boredom - and I'm just trying to improve my poor C knowledge.

      It seems quite unusual to me that a function has to be called twice to behave correctly - that is, to free the memory in this case.

      Regards, Karl

      «The Crux of the Biscuit is the Apostrophe»

        I'm just trying to improve my poor C knowledge

        Be careful what you wish for. :) I'm not sure studying large and complex C code bases is a good way to learn C. I started by reading The C Programming Language, aka K&R, from cover to cover.

        Having slaved over many huge C code bases over the years, I can assure you they all rely heavily on static and dynamic code analysis tools to keep the code clean (e.g. Coverity, Valgrind, AddressSanitizer and many more). P5P is no different, as indicated by the EXTERNAL TOOLS FOR DEBUGGING PERL section at perlhack.

        So, if you want to become a serious C coder, you must master these tools.

        I'm just trying to improve my poor C knowledge.

        You picked something that's probably a bit too advanced for that. It involves not just C, but knowledge of FFI and of Perl internals as well, both of which are complex systems.

        Is [replacing malloc with GC_MALLOC] recommended or will it cause problems in the future not yet imaginable now?

        The answer to the question depends on whether FFI or Perl can free the returned buffer.

        I think the const in the return type tells FFI to copy the string and avoid freeing it. If so, then yes, you could use a different allocator.

        I'm guessing that because the example would have problem as-is if FFI or Perl could free the buffer.

Re: FFI::Platypus: Replace malloc with GC_MALLOC?
by ikegami (Patriarch) on Dec 13, 2022 at 14:03 UTC

    The issue is that you don't want to forget to free the buffer.

    Another approach is to not use FFI directly, but hide the glue in a module, where you could use an END block to ensure it's freed on exit.

    package My::StringUtils; use strict; use warnings; use Export qw( import ); use FFI::Platypus 2.00; our @EXPORT_OK = qw( string_reverse ); my $ffi = FFI::Platypus->new( api => 2, lib => './string_reverse.so', ); $ffi->attach( string_reverse => [ 'string' ] => 'string' ); END { string_reverse( undef ); } 1
    use strict; use warnings; use feature qw( say ); use My::StringUtils qw( string_reverse ); say string_reverse( "Hello world" );

    (Fine, you don't need a module to use END, but using a module means you don't forget to use END.)

      you could use an END block to ensure it's freed on exit

      I don't see the point of this. AIUI, on exit, perl's garbage collector will free that memory anyway.

      Also, as I mentioned above, I'm not convinced that string_reverse(undef) does not assign additional memory after it frees the previously allocated memory.
      I am convinced that, with Inline::C, it will assign more memory because XS does not equate the undef argument to NULL.
      Is there any reason to believe that it would be different under FFI::Platypus ?

      Cheers,
      Rob

        I don't see the point of this. AIUI, on exit, perl's garbage collector will free that memory anyway.

        Because the OP asked how to do it.

        And there's plenty of reason to want to learn how to perform cleanup.

        As for the specific case, it's a question of debate whether you should free memory on exit. But it does have benefits. One major benefit is that allows us to use valgrind or similar to find meaningful leaks.

        Finally, you're incorrect assuming that the termination of a Perl interpreter only occurs when the program is unloaded (i.e. when process exits, execs, etc). Each threads of a process has its own interpreter, for example. A process embedding Perl could load and unload the interpreter without exiting.

        Is there any reason to believe that it would be different under FFI::Platypus?

        There's no reason to believe it would be the same.

        And the OP specifically quoted someone saying it was different (that it results in a NULL argument).

        If there's a bug, fine. Point it out, fix it, whatever. But it has nothing to do with my answer. The question is about ensuring that cleanup is performed, and that's what I was addressing. The exact interface and implementation doesn't matter, so that's a separate topic you've already addressed and didn't need to bring up here.

      This looks good - very good . I also don't know why I didn't come up with the END block idea myself to hide that ugly second call. Regards, Karl

      «The Crux of the Biscuit is the Apostrophe»

Re: FFI::Platypus: Replace malloc with GC_MALLOC?
by syphilis (Archbishop) on Dec 12, 2022 at 02:31 UTC
    string_reverse(undef);

    Does that actually free() the memory ?
    I would be wanting to see proof that the if(output != NULL){} block is being entered:
    if(output != NULL) { free(output); printf("YES - output was free()ed\n"); }
    I guess that the $ffi->attach() line might somehow provide the necessary magic.

    Note: I've just installed FFI::Platypus (Windows 11, perl-5.36.0) - it required some force, and installed a shudderingly disturbing amount of Alien::* (and other) shit.
    But I couldn't quickly get anywhere with the demo that you supplied. Does that demo comprise 2 separate files ? If so, how are they linked ?

    I've no experience with GC_MALLOC, and hadn't even heard of it.
    I'll read, with interest, the thoughts of any who might have used this.

    In the Inline::C/XS environment, if I want to avoid having to explicitly free/Safefree memory, then I'll create objects blessed into a specific package and have perl automagically free/Safefree the objects as perl sees appropriate.

    Cheers,
    Rob
      "… I would be wanting to see proof."

      I guess I should probably do the obviousthing and just try it out.

      Regards, Karl

      «The Crux of the Biscuit is the Apostrophe»

        I guess I should probably do the obvious thing and just try it out

        That could be informative ;-)
        I've had a bit of a fiddle with it in Inline::C, and it seems that it probably almost works as you want.
        The print string_reverse("\nHello World"); works exactly as expected and does not call free().
        And the 2nd line of perl code (string_reverse(undef);) definitely calls free().

        The only problem is that having called free(), Inline::C/XS evaluates the condition input == NULL as FALSE - and it therefore allocates more memory, which in turn would require a third call to be cleared ... and so on.
        With Inline::C/XS, the argument to the second call needs to be something string_reverse() will evaluate as NULL - and undef does not fit that bill.
        Maybe, with FFI::Platypus, undef will be evaluated as NULL ... you could verify that it's being evaluated as desired, by using Devel::Peek and changing that second line to Dump(reverse_string(undef);.

        If that results in a Dump() that looks something like:
        SV = NULL(0x0) at 0x22cfb41ffb0 REFCNT = 1 FLAGS = (PADTMP)
        then the undef was evaluated as NULL and all is probably good.
        But for me, with Inline::C, that Dump() produces:
        SV = PV(0x22cfb55afc8) at 0x22cfb43c068 REFCNT = 1 FLAGS = (PADTMP,POK,pPOK) PV = 0x22cfd20c3d8 ""\0 CUR = 0 LEN = 10
        which indicates that more memory was allocated by that second call.

        I remember that this problem with undef not being equated with NULL has come up here before ... I haven't done the search for it :-(
        UPDATE: The thread I was thinking of is this one.

        I can't recall whether there exists a perl arg that will be equated to NULL, or whether you have to hack the perl typemap to achieve it.
        Anyway ... FAIRK, FFI::Platypus might handle undef differently.

        Cheers,
        Rob
        use strict; use warnings; use Devel::Peek; use Inline C =><<'EOC'; #include <string.h> #include <stdlib.h> const char * string_reverse(const char *input) { static char *output = NULL; int i, len; if(output != NULL) { free(output); printf("YES - output free()ed\n"); } if(input == NULL) return NULL; len = strlen(input); output = malloc(len+1); for(i=0; input[i]; i++) output[len-i-1] = input[i]; output[len] = '\0'; return output; } const char * nullstr() { return NULL; } EOC print string_reverse("\nHello World"); Dump(string_reverse(undef)); Dump(nullstr()); __END__ Outputs: dlroW olleH Use of uninitialized value in subroutine entry at try.pl line 42. YES - output free()ed SV = PV(0x22b16cbafc8) at 0x22b16bdcba8 REFCNT = 1 FLAGS = (PADTMP,POK,pPOK) PV = 0x22b19158598 ""\0 CUR = 0 LEN = 10 SV = NULL(0x0) at 0x22b16bbfed0 REFCNT = 1 FLAGS = (PADTMP)

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://11148746]
Approved by marto
Front-paged by haukex
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others exploiting the Monastery: (3)
As of 2024-04-20 02:06 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found