Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things
 
PerlMonks  

Perl crash during perl_clone

by perlmonk1729 (Acolyte)
on Oct 27, 2010 at 08:33 UTC ( #867652=perlquestion: print w/replies, xml ) Need Help??

perlmonk1729 has asked for the wisdom of the Perl Monks concerning the following question:

Hello Monks,

My post on perl5-porters didnt get any reponses. So, hoping someone here knows about this.

I use SWIG to extend a C API and I also embed perl interperter to handle callbacks from the C API into Perl subroutines. The API is quite large and the .so and .pm files are 10MB, 1MB respectively.

I get either (a) a crash down one of the functions called from perl_clone or (b) an assert ("Not a CODE reference @ line xxx") pointing to "random" locations in the perl script...usually, just around the time when the first callback is about to be called into perl. Small timing differences seem to produce/hide these problem, but I'm not able to figure out the cause of the problem. Note I'm not able to reproduce it with a sample/test script :-(

My impl worked most of the time on an ubuntu, perl 5.10.x and older desktop HW configuration, except I got these errors occasionally. But, the problems have become acute on another fedora, perl 5.10 on laptop configuration. Browsing through the archives and online hasnt yielded any clue so far.

Please help!

Thx

#0 0x008ad4d5 in Perl_sv_free () from /usr/lib/perl5/CORE/libperl.so #1 0x0083fba2 in Perl_gv_fetchmeth () from /usr/lib/perl5/CORE/libper +l.so #2 0x0084061d in Perl_gv_fetchmethod_autoload () from /usr/lib/perl5/CORE/libperl.so #3 0x008b790a in ?? () from /usr/lib/perl5/CORE/libperl.so #4 0x008a1be2 in ?? () from /usr/lib/perl5/CORE/libperl.so #5 0x008b24b1 in perl_clone () from /usr/lib/perl5/CORE/libperl.so #0 0x0089689c in Perl_av_store () from /usr/lib/perl5/CORE/libperl.so #1 0x00897252 in Perl_av_push () from /usr/lib/perl5/CORE/libperl.so #2 0x0088f4d7 in ?? () from /usr/lib/perl5/CORE/libperl.so #3 0x0088fb9f in Perl_mro_get_linear_isa () from /usr/lib/perl5/CORE/libperl.so #4 0x0083fc06 in Perl_gv_fetchmeth () from /usr/lib/perl5/CORE/libper +l.so #5 0x0084061d in Perl_gv_fetchmethod_autoload () from /usr/lib/perl5/CORE/libperl.so #6 0x008b790a in ?? () from /usr/lib/perl5/CORE/libperl.so #7 0x008a1be2 in ?? () from /usr/lib/perl5/CORE/libperl.so #8 0x008b24b1 in perl_clone () from /usr/lib/perl5/CORE/libperl.so #0 0x008a2928 in Perl_sv_dup () from /usr/lib/perl5/CORE/libperl.so #1 0x008a3f45 in Perl_mg_dup () from /usr/lib/perl5/CORE/libperl.so #2 0x008a32cd in Perl_sv_dup () from /usr/lib/perl5/CORE/libperl.so #3 0x008a41ea in Perl_gp_dup () from /usr/lib/perl5/CORE/libperl.so #4 0x008a3533 in Perl_sv_dup () from /usr/lib/perl5/CORE/libperl.so #5 0x00894f17 in Perl_he_dup () from /usr/lib/perl5/CORE/libperl.so #6 0x008a30dc in Perl_sv_dup () from /usr/lib/perl5/CORE/libperl.so #7 0x008a3515 in Perl_sv_dup () from /usr/lib/perl5/CORE/libperl.so #8 0x008b2966 in perl_clone () from /usr/lib/perl5/CORE/libperl.so

Replies are listed 'Best First'.
Re: Perl crash during perl_clone
by BrowserUk (Patriarch) on Oct 27, 2010 at 09:51 UTC
    My post on perl5-porters didnt get any reponses.

    If you supplied a similarly minimal amount of information there as here, that is not surprising. It will be very difficult to reach any definitive conclusions based up such scant information.

    As I said in a previous thread, if you don't post representative, working code to demonstrate your problem, then anyone attempting to help you has to try and re-create your scenario from your very brief description. And that really is asking far too much of volunteers.

    I notice that you actually did post some code in a previous thread, but you did so as replies to yourself, which means that no one else ever received a notification of those posts.

    Ie. It is probable that none of the monks involved in that thread, including cdarke who actually attempted to reproduce your failing scenario, almut nor I were even aware that you had posted it. I certainly wasn't.


    Attempting to read between the lines of the information you have provided here (and in your related previous threads), you are attempting to call a coderef (callback)--in an interpreter/thread that you spawned yourself from within C--that was passed into your C code from another interpreter/thread spawned by Perl.

    With the best will in the world, that is never going to work reliably. I'd be amazed if it ever worked at all. Though from your description it appears it has and does--so color me amazed.

    The only way I can see it working sometimes and not others, is if your code is dependant upon which thread happens to be running when the callback is made. That is to say, if at the instant the asynchronous event (socket read completion?) that triggers the callback, the same thread that passed the callback coderef to your C code happens to be running, then it will likely work. If however, the other thread happens to be running, then it doesn't.

    If this is the case--which should be fairly easy to verify by having your C/XS code log the current thread id: a) when it is passed the callback coderef; b) just before it invokes that callback. And if it is the case, the resolution would be to ensure that when your C-code creates it's own interpreter/thread, it follows the exact same steps as used by threads::create(). Of course, in reality, it would almost certainly be easier to allow Perl to that for you.

    But then comes the question of why you feel the need to create a new interpreter in the first place? And for anyone to be able to understand that, you will have to describe the overall functionality and methodology of your complete application in far more detail than you have to date.

    But...there is a simpler route. Take your 10MB of C code out of the equation. Once you have a reliable demonstration of passing in a Perl coderef to C code, which is subsequently called back at a later time--possibly triggered by a timer--in a multi-threaded environment, it should then be pretty easy to port that back to your real code.

    Perhaps based on the code you posted in that other thread, create a minimal, runnable example of the callback scenario you require. But do it in a single post--as a reply to someone other than yourself.

    Preferably do it as a single file using Inline::C to make it easy for others to download and run. I for one wouldn't know what to do with all the separate files you posted in that other thread, in order to get to the point where I could actually run the code.

    If you cannot see how to do it as a single file using Inline::C, then at least wrap the multiple files up into a proper CPAN -style distribution and make it available somewhere, so that others can use CPAN to build and test it.


    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
      Hi BrowserUk:

      Yes, I meant to put a link to the old post but missed it. Thx for finding it yourself! That code is still quite valid. I also did not realize that others did not get notification of my replies.

      Reg "you are attempting to call a coderef (callback)--in an interpreter/thread that you spawned yourself from within C--that was passed into your C code from another interpreter/thread spawned by Perl."

      I think you got the gist of my approach, except that the the coderef was passed into C code from Perl's "default" interpreter (the instance created automatically when I run perl.exe) & NOT from another interpreter/thread spawned by Perl (not sure the difference matters).

      I found this approach from perlembed, other posts and online references. It seems others had it working similarly.

      I'll try to package the sample code as downloadable. However, CPAN packaging is new to me so need to research it.

      When this problem occurs, there are only two "threads" of execution..perl.exe's interpreter and the 2nd interpreter/thread I creaetd. Anyway, I'm open to trying any hunches..so, will try this as well.

      I wondered if the crash could be due to perl's "default" interpreter changing data structures while the cloning (perl_clone) is still executing. I was planning to explore 'blocking' the default interpreter until the perl_clone is complete.

      Appreciate your time/comments very much!

        except that the the coderef was passed into C code from Perl's "default" interpreter (the instance created automatically when I run perl.exe) & NOT from another interpreter/thread spawned by Perl (not sure the difference matters).

        What you've described is exactly what I was trying and failing to describe. Two interpreters, one created by perl, one by your C code. And passing a code reference between those two. Perl will not allow you to attempt to pass a coderef between interpreters:

        perl -Mthreads -Mthreads::shared -Mstrict -wle" my $c:shared; async{ ## This next line triggers the "Invalid value for shared scala +r" $c = sub{ print 'Hi'}; sleep 100 }->detach;; sleep 3; $c->(); sleep 100" Thread 1 terminated abnormally: Invalid value for shared scalar at -e +line 1. Can't use an undefined value as a subroutine reference at -e line 1.

        But, by passing the coderef via C, you are by-passing this protection. Hence, as I surmised, if the callback gets triggered at a point in teh application when the "wrong" interpreter has the cpu, then it fails. Whereas if the originating inpterpreter is in control, it works,

        The solution I'm afraid is to re-think your methodology. If you can describe why you are starting a second interpreter, we might be able to see a solution.


        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://867652]
Approved by marto
Front-paged by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others studying the Monastery: (4)
As of 2022-10-07 08:31 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    My preferred way to holiday/vacation is:











    Results (29 votes). Check out past polls.

    Notices?