Beefy Boxes and Bandwidth Generously Provided by pair Networks
Welcome to the Monastery
 
PerlMonks  

XS from XS?

by BrowserUk (Patriarch)
on Apr 10, 2010 at 13:10 UTC ( [id://833995]=perlquestion: print w/replies, xml ) Need Help??

BrowserUk has asked for the wisdom of the Perl Monks concerning the following question:

Can one XS library easily make use of the functions within another XS library? If so, how? Are there any examples of this?


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.

Replies are listed 'Best First'.
Re: XS from XS?
by creamygoodness (Curate) on Apr 10, 2010 at 14:40 UTC

    I know of two ways to do it.

    First, you can go through the Perl interface using the callback API from perlcall. That would definitely not qualify as "easy", though.

    Alternatively, you can get the source library to export its C functions somehow. There are numerous ways to accomplish this, some of which are discussed in Dynaloader/XS: sharing C symbols across shared objects. Cross-platform exporting is difficult. I ultimately adopted the technique from Time::HiRes described by salva.

    Here's code from Snowball.xs which makes 4 symbols from the Snowball stemming library available (sb_stemmer_list, etc):

    That works for a small list of functions. It would be cumbersome and impractical for a large export list.

      If I understand that right--and I'm not certain I am--you are importing the Perl callable functions in one XS module, into the namespace of another XS module?

      If so, that's not quite what I'm after--though I understand that my question was unclear.

      My thought--albeit perhaps not fully thought through--is to have an XS module that is callable/usable from Perl in the normal way. But to also allow other XS modules to make use of the internal C functions (that sit behind the Perl-callable XS interface), directly at runtime. That is, without #includeing the sources, or linking against the objects or a lib. There would be two or three entrypoints to import at most.

      The notion was to remove the compile-time dependancy upon the headers or a lib for potential XS users of the module. They could then decide at runtime whether it is available and use it. Or fallback to other mechanisms. Ie. If the module is installed locally, load it and "obtain the required entrypoints".

      Hm. I know I can do this on Windows. LoadModule()/GetProcAddr() are all I need. But I would need a portable solution, and I don't know enough about other platforms to know if there are equivalents.


      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.

        What about creating a nice set of C wrapper functions for the OS dependant LoadLibrary/GetProcAddr/FreeLibrary calls? Makefile.PL / Build.PL could decide which one to use, or panic if the OS is unknown / not implemented.

        BTW: Linux seems to use dladdr, dlclose, dlerror, dlopen, dlsym, dlvsym, the first four are POSIX.1-2001, the other two are GNU extensions, according to the man page. So, if you use the Win32 API for Windows as a special case, and simply try to use dlopen and friends on other platforms, you could get the job done for most Unix platforms. Other platforms should fail and complain loudly during make.

        Alexander

        --
        Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)

        I believe that second technique I described (the Time::HiRes approach) does exactly what you want. In the exporting module, you store function pointer address values within a global Perl hash as integer scalar values. In the importing module, you recover those function pointer address values from the integer scalars and cast them to function pointers of the appropriate type.

        If I understand that right--and I'm not certain I am--you are importing the Perl callable functions in one XS module, into the namespace of another XS module?

        He's saving the address of functions callable from C in the current Perl package.

      This is a situation where "X macros" (https://en.wikipedia.org/wiki/X_Macro) really help, as used by PDL. It used to have highly redundant, multiple lists of the PDL API functions in several files. Now it uses the "X macro" technique, which for this application would look like (untested - see https://github.com/PDLPorters/pdl/blob/master/Basic/Core/pdlcore.h for defines and use, and https://github.com/PDLPorters/pdl/blob/master/Basic/Core/Core.xs for use, of PDL_CORE_LIST):
      /* in C part of .xs, else in a .h */ #define SNOWBALL_FUNCS(X) \ X(list, 39) \ X(new, 38) \ X(delete, 41) \ X(stem, 39) \ X(length, 41) MODULE = Lingua::Stem::Snowball PACKAGE = Lingua::Stem::Snowball PROTOTYPES: disable BOOT: { #ifdef I_HAVE_C99 /* not a real symbol, don't use this */ #define X(symname, no, ...) \ SV *sb_stemmer_ ## symname ## _sv = newSViv(PTR2IV(sb_stemmer_ # +# symname)); \ hv_store(PL_modglobal, "Lingua::Stem::Snowball::sb_stemmer_" #symn +ame, no, sb_stemmer_ ## symname ## _sv, 0); SNOWBALL_FUNCS(X) #undef X #else #define X(symname, no, ...) \ SV *sb_stemmer_ ## symname ## _sv = newSViv(PTR2IV(sb_stemmer_ # +# symname)); SNOWBALL_FUNCS(X) #undef X #define X(symname, no, ...) \ hv_store(PL_modglobal, "Lingua::Stem::Snowball::sb_stemmer_" #symn +ame, no, sb_stemmer_ ## symname ## _sv, 0); SNOWBALL_FUNCS(X) #undef X #endif }
      The biggest benefit is maintainability. You'll see the PDL example also captures the C functions' return types and argument types, so adding to the API is a matter of adding lines to the big macro calling X.

      Edited to add how to use this if you're not using C99, where you can declare things anywhere.

Re: XS from XS?
by Anonymous Monk on Apr 11, 2010 at 01:06 UTC
    Does this help? ExtUtils::Depends - Easily build XS extensions that depend on XS extensions
      Does this help? ExtUtils::Depends

      No, not exactly. Actually, not even slightly. That requires that in order to make use of the module, that it be available at install/compile time. That pretty much belies the reasoning for having dynamic linking.

      What I'm pursuing is a cross-platform way of expressing these two operations:

      1. Is there an (unqualified) .dll/so/.other of this name accessible?
      2. If so, does it export these entrypoints?

      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://833995]
Approved by marto
Front-paged by moritz
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others surveying the Monastery: (3)
As of 2024-04-19 22:01 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found