Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical
 
PerlMonks  

Problem on perl callbacks from XS

by cosimo (Hermit)
on May 24, 2008 at 21:44 UTC ( [id://688332]=perlquestion: print w/replies, xml ) Need Help??

cosimo has asked for the wisdom of the Perl Monks concerning the following question:

Hi monks, I've been fighting with this problem for a while, and despite my best efforts, I'm still stuck. Maybe very near to the solution, but still stuck. I believe that the following simple case is the root of the problem. Feel free to correct/direct me.

The code

GV *to_string = gv_fetchmethod (SvSTASH (sv), "toString"); if (to_string) { dSP; SV *res_sv; /* result scalar */ ENTER; SAVETMPS; /* Declare function arguments */ PUSHMARK (SP); XPUSHs (sv_bless (sv_2mortal (newRV_inc (sv)), SvSTASH (sv))); PUTBACK; /* Call in scalar context (G_SCALAR) */ call_sv ((SV *)GvCV (to_string), G_SCALAR); SPAGAIN; /* Fetch returned SV from the stack */ res_sv = POPs; PUTBACK; /* Append returned SvPVX to our SV */ append_sv(res_sv, result); FREETMPS; LEAVE; }
This code is partly taken from JSON::XS. I'm trying to write an XS function that takes an array or hashref in sv and, with some custom rules, serializes the content into a result scalar as one big string with everything.

When an hashref is really a blessed object, I want its toString() method to be called to dump the hashref content to my result SV. Calling the object's toString() method is the task for the code I shown you.

The problem

I think the problem lies here: what happens when, eventually, an object's toString() method code, for whatever reason, happens to contain a call to my XS function? (that could, in turn, call other toString() methods for other kind of objects...).

Does this break my stack manipulation? I think so, because I see segfaults and other bad things when toString() completes and the the code tries to return the resulting SV to the caller.

I feel I'm missing something about the stack manipulation in this case, but I really don't know how to proceed. Thank you for any help you can provide.

UPDATE: 2008-05-25 14:53 CET

The solution

I believe I have solved the problem. It was a real bitch of a hellish problem, trust me. Seems I've been bitten by old XS conventions about returning values to Perl code.

As it turned out, the problem was not in the snippet I showed you, but rather in this one, that called the function above, and incorrectly returned the resulting scalar to the caller. Real names changed to protect the innocents :)

void func (SV *scalar) PPCODE: SV *result = newSV(INIT_SIZE); sv_setpv(result, ""); SvPOK_only(result); serialize(result, scalar); XPUSHs(result);

The fixed and working version follows:

SV* func (SV *scalar) CODE: SV *result = newSV(INIT_SIZE); sv_setpv(result, ""); SvPOK_only(result); serialize(result, scalar); RETVAL = result; OUTPUT: RETVAL

Replies are listed 'Best First'.
Re: Problem on perl callbacks from XS
by almut (Canon) on May 24, 2008 at 23:22 UTC

    I don't see a principal problem with the Perl stack and your function getting called again before it has completed, so I suppose the problem lies elsewhere (it's hard to tell though when only seeing a snippet of code taken out of its context).

    One potential problem could be that you aren't caring about the return value of call_sv() (which is the number of items returned by the Perl routine). This is dangerous, because if the routine - for whatever reason - leaves a different number of items on the stack than you're expecting/handling (you're handling exactly one item with your POPs), the stack will be messed up...   (not necessarily your problem here, but still something to be aware of).

      Yes, in fact, the complete code contains this:

      int count; ... count = call_sv(..., G_SCALAR); assert(count==1);

      which, AFAIK, should be "safe".

Re: Problem on perl callbacks from XS
by syphilis (Archbishop) on May 25, 2008 at 00:09 UTC
    but I really don't know how to proceed

    One thing that may help is to write a self-contained Inline::C script that demonstrates the problem. In the writing of that script, you may even find the solution yourself. If not, at least at the end of the exercise, you have some code that you can present for others to download and run.

    Obviously, the more minimalistic the script, the better.

    Cheers,
    Rob
Re: Problem on perl callbacks from XS
by tachyon-II (Chaplain) on May 25, 2008 at 06:46 UTC

    When an hashref is really a blessed object, I want its toString() method to be called to dump the hashref content to my result SV. Calling the object's toString() method is the task for the code I shown you.

    But why do you need to go to XS and back to do that? I appreciate this is probably a simplified example but if the thing you want to serialise already has a toString method to serialise it in Perl AND you want to return this in your results SV why do that in a round about way in XS?

    sub serialise { my $ref = shift; my $str = undef; eval{ $str = $ref->toString }; return $str if defined $str; $str = XS_serialise($ref); return $str; }

      Yes, indeed. This should be the approach. However, (there's always one :), I want the serialization function to handle a big data structure, which can contain these blessed objects rather deep inside it.

      Since the main purpose of all this mess is to speed up things, I thought I'd want to pass a reference to this "big structure" and then the XS function would do the heavy work.

      Doing as you say, and I've tried that before, probably means that I must move all the data structure traversal outside of XS. Example:

      my $big_struct = [ {a=>1}, {b=>2}, [1, 2, 3, 4, 5], ['blah', 'blah', ['bluh', $blessed_object]], {d=>3, e=>[1,2,3]} ];

      If I want to correctly handle $blessed_object from XS, I must call its toString() (assume for now it has one) with call_sv().
      Doing this from Perl code, means I must shift all the structure traversal from XS to Perl... losing a share of the speed gain...

        Here is an example. It's done in Inline but is just straight vanilla XS. The stack pointer behaves itself over the separate calls.

        package Serialize; use Data::Dump; sub new { bless {foo=>'bar'}, shift } sub toString { return Data::Dump::dump($_[0]) } my $obj = new Serialize; my $str = "some string"; print serialize($obj), $/; print serialize($str), $/; use Inline Config => FORCE_BUILD => 1, CLEAN_AFTER_BUILD => 0; use Inline C => <<'END_OF_C_CODE'; void serialize(SV* sv_data) { int count; dXSARGS; sp = mark; printf("\nEntry SP is %d\n", sp); if (sv_isobject(sv_data)) { // this is an object so presume a toString method PUSHMARK(sp); XPUSHs(sv_data); // whack it back on the stack PUTBACK; count = perl_call_pv("toString", G_SCALAR); if ( count != 1 ) croak("Expected 1 value got %d\n", count); XPUSHs(sv_2mortal(newSVsv(POPs))); } else { // just return SV with something appended for this example sv_catpv(sv_data, " - serialized\0"); XPUSHs(sv_2mortal(newSVsv(sv_data))); } PUTBACK; printf("Exit SP is %d\n", sp); XSRETURN(1); } END_OF_C_CODE __DATA__ bless({ foo => "bar" }, "Serialize") some string - serialized Entry SP is 32203548 Exit SP is 32203552 Entry SP is 32203548 Exit SP is 32203552

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://688332]
Approved by moritz
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others making s'mores by the fire in the courtyard of the Monastery: (4)
As of 2024-03-28 18:25 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found