Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?
 
PerlMonks  

Re^4: XS: EXTEND/mPUSHi

by BrowserUk (Patriarch)
on Sep 26, 2011 at 23:43 UTC ( [id://927978]=note: print w/replies, xml ) Need Help??


in reply to Re^3: XS: EXTEND/mPUSHi
in thread XS: EXTEND/mPUSHi

these macros can be found in Inline/C.pm

The problem is not in finding their definitions, it is understanding what they do, when they must be used and when not etc.

Their definitions are all in terms of other (XS) macros, and unwinding those is much harder. And once you have unwound them to the actual code that gets executed, it is generally a horrible mess of nested macro expansions calling a bunch of undocumented apis -- often repetitively -- and manipulating another bunch of undocumented global variables.

Trying to work out what calls are actually need when, rather than just cargo-culting someone else who cargo-culted someone else who ... is very difficult. Weirdness like the completely unused & redundant PUTBACK; return; sequence generated by the Inline::C wrapper functions just compounds matters.

I think these macros are mostly useful for beginners in that they provide a mantra that gets most jobs done - and despite their verbosity, are easier to remember than the corresponding list of XS symbols.

Do you really find verbosity more memorable than conciseness? I don't. I realise that you inherited those definitions, but I think that it is a fallacy to believe they are more memorable than the XS_equivalents.

With Ike's help above, I've reduced my fast random generator to:

void rnd64( int n ) { dXSARGS; static unsigned __int64 y = 88172645463325252i64; EXTEND( SP, n ); while( n-- ) { y ^= y << 13; y ^= y >> 7; y ^= y << 17; mPUSHu( y ); } PUTBACK; return; }

Which appears to work well. It doesn't appear to leak any memory after producing over 12 billion 64-bit rands (in batches of 1 million) in an hour , but that still leaves me with the question of whether I should be resetting the stack to account for the single input parameter, or whether the EXTEND() takes care for that for me?

And it is this aspect of using Inline::C or XS that leaves me cold. The reason for dipping into the quagmire is to achieve speed unobtainable in Perl, but working out what bits of the templated examples are actually required and when is (it seems) only possible by suck-it-and-see.

I also wonder if I shouldn't be passing in a reference to the an array and populating it directly, rather than assigning the returned stack to it?

Equally, rather than creating a whole new bunch of SVs to hold my rands in the target array, couldn't I just modify their IVs in place? Would that be more efficient?


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.

Replies are listed 'Best First'.
Re^5: XS: EXTEND/mPUSHi
by syphilis (Archbishop) on Sep 27, 2011 at 00:39 UTC
    but that still leaves me with the question of whether I should be resetting the stack to account for the single input parameter, or whether the EXTEND() takes care for that for me

    No - as it stands, the first value that gets returned should be the supplied argument - at least that's what I'm finding. If you want to avoid that, then you need to reset the stack pointer or take some alternative measure.
    Ike questions the validity of doing sp = mark though it's something that has been in the Inline test suite for the last few years, and has not produced any problems with the cpan-testers. Is there a better way of resetting the stack pointer ?
    I think that, in this instance you could also just sp--; (update: or, more generally, sp -= items) before you EXTEND.

    I can't actually find any code of mine that resets the stack pointer - I usually take a different approach such as just assigning to ST(0), ST(1), etc.

    I also wonder if I shouldn't be passing in a reference to the an array and populating it directly, rather than assigning the returned stack to it?

    I've benchmarked that for some huge arrays in the past ... and not detected any advantage in passing by reference (with Inline::C).

    Cheers,
    Rob
      I've benchmarked that for some huge arrays in the past ... and not detected any advantage in passing by reference (with Inline::C).

      I'm guessing that you were populating the passed-by-reference Av with new SVs?

      This (rnd64i( int n, SV *avref ) (i for inplace--terrible name)), assumes a pre-populated (not just pre-sized) array of n scalars into which it then sets the values to be returned. The Perl code contrasts ARGV[0] calls for 1 million values each time, with the same for rand64() which returns them on the stack and assigns them to an array.

      #! perl -slw use strict; use Inline C => Config => BUILD_NOISY => 1; use Inline C => <<'END_C', NAME => 'monkeys', CLEAN_AFTER_BUILD => 0; void rnd64( int n ) { dXSARGS; static unsigned __int64 y = 88172645463325252i64; SP = MARK; EXTEND( SP, n ); while( n-- ) { y ^= y << 13; y ^= y >> 7; y ^= y << 17; mPUSHu( y ); } PUTBACK; return; } void rnd64i( int n, AV* av ) { dXSARGS; SV **ary = AvARRAY( av ); static unsigned __int64 y = 88172645463325252i64; SP = MARK; while( n-- ) { y ^= y << 13; y ^= y >> 7; y ^= y << 17; sv_setuv( ary[ n ], y ); } PUTBACK; return; } END_C use Data::Dump qw[ pp ]; use Devel::Peek; use Time::HiRes qw[ time ]; my $start = time; my @rands = (1) x 1e6; for( 1 .. $ARGV[0] ) { @rands = rnd64( 1e6 ); } my $stop = time; printf "stack->array assign: Rate: %.9f\n", ( $stop - $start ) / ( 1 +e6 * $ARGV[ 0 ] ); $start = time; my @rands2 = (1) x 1e6; for( 1 .. $ARGV[0] ) { @rands = rnd64i( 1e6, \@rands2 ); } $stop = time; printf "Modify array in-place: Rate: %.9f\n", ( $stop - $start ) / ( 1 +e6 * $ARGV[ 0 ] ); exit;

      The result is rnd64i() is up to 100 times faster:

      C:\test>Monkeys 1 stack->array assign: Rate: 0.000011359 Modify array in-place: Rate: 0.000000121 C:\test>Monkeys 10 stack->array assign: Rate: 0.000001353 Modify array in-place: Rate: 0.000000037 C:\test>Monkeys 100 stack->array assign: Rate: 0.000000365 Modify array in-place: Rate: 0.000000030

      That's a saving worth having.


      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
        I'm guessing that you were populating the passed-by-reference Av with new SVs?

        Quite possibly - looking at those figures I must have been doing *something* dumb :-)

        Cheers,
        Rob

      xsubpp currently generates SP -= items;.

      That change to the stack pointer is lost, since it's done in the real XS function (the wrapper), which doesn't doesn't "publish" the change using PUTBACK before calling the wrapped function.

      I'd make Inline::C call PUTBACK before calling the wrapper function and make Inline_Stack_Reset a no-op.

      In two places:

      PPCODE: + PUTBACK; temp = PL_markstack_ptr++;

      Update: I thought there would be backwards compatibility issues, but there aren't. ST(0) and such will still get the args. Adjusted.

        I'd make Inline::C call PUTBACK before calling the wrapper function and make Inline_Stack_Reset a no-op

        Thanks - something to think about.
        Of course, while there's not actually anything that's broken, I'm inclined to leave things as they are.

        Cheers,
        Rob
Re^5: XS: EXTEND/mPUSHi
by ikegami (Patriarch) on Sep 27, 2011 at 01:08 UTC

    It doesn't appear to leak any memory

    Indeed, there's no leak.

    Perl's stack isn't refcounted, so that means you can't decrement the refcount of a scalar you place on the stack even if you don't keep a reference to it.

    What you do is place the scalar on a list of variables whose refcount needs to be decremented when they are removed from the stack. Variables on this list are called mortals.

    (This way of doing things is a source of many bugs, but that's the box you have to play in.)

    The "m" in "mPUSHi" stands for mortal. The created scalar will be made mortal.

    mPUSHi(4);
    boils down to
    *(++SP) = sv_2mortal(newSViv(4));

    but that still leaves me with the question of whether I should be resetting the stack to account for the single input parameter

    Check the first value being returned. It's always your argument (n).

    or whether the EXTEND() takes care for that for me?

    As documented EXTEND will make sure there is n spaces available. That's it. It doesn't remove anything already there.

    In your case, you end up with at least n+1 spaces on the the stack total, one of which is used for the argument.

    Weirdness like the completely unused & redundant PUTBACK; return; sequence generated by the Inline::C wrapper functions

    It's actually added by the XS to C converter, xsubpp. Inline::C doesn't put there. In fact, Inline::C specifically doesn't want it, but doesn't have a way of telling xsubpp not to call it.

    Inline::C requires that you call PUTBACK (explicitly or via XSRETURN) because it can't do it for you because it doesn't have access to your sub's SP to "put it back".

    Real XS doesn't require that you call PUTBACK because xsubpp actually creates the sub, so it has access to the sub's SP.

    PS — C doesn't require the use of return; when the return type is void.

      Inline::C requires that you call PUTBACK (explicitly or via XSRETURN) because it can't do it for you because it doesn't have access to your sub's SP to "put it back".

      Hm. The wrapper sub knows (or can find out) how many values were passed on the stack. For fixed-length argument lists (no ellipses), it could clean up the stack before calling the I::C sub. Indeed, it already seems to (attempt to) do that:

      XS(XS_main_rnd64); /* prototype to pass -Wmissing-prototypes */ XS(XS_main_rnd64) { #ifdef dVAR dVAR; dXSARGS; #else dXSARGS; #endif if (items != 1) croak_xs_usage(cv, "n"); PERL_UNUSED_VAR(ax); /* -Wall */ SP -= items;

      Which is the source of my confusion regarding the need to do this (again) within the sub.

      It (the wrapper sub) could also take a mark on the stack prior to invoking the I:C sub: (which it also seems to do:

      temp = PL_markstack_ptr++;
      ),

      and then use that to determine how many (if any) values have been pushed onto the stack during the call, thereby removing the need for the I::C programmer to do this manually.

      For all the world it looks like the wrapper sub were originally intended to automate this part of the process, but then something changed. Maybe the author changed his mind or couldn't get it to work. Or some other programmer went in there to correct some bug and made changes that were incompatible with the original vision. (This all pre-dates syphilis' involvement.).

      The closer you start to look at this code, the more oddities and (possibly) missed opportunities you see.


      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.

        The wrapper sub knows (or can find out) how many values were passed on the stack. For fixed-length argument lists (no ellipses), it could clean up the stack before calling the I::C sub.

        I concur. I came to the same conclusion.

        and then use that to determine how many (if any) values have been pushed onto the stack during the call, thereby removing the need for the I::C programmer to do this manually.

        I'm guessing "do this" means "calling PUTBACK". If so, sweet. I hadn't looked that far.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://927978]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others surveying the Monastery: (3)
As of 2024-03-29 02:22 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found