http://qs321.pair.com?node_id=593298

Hi mongers,

I'm planning to build a Perl Extension and my first insight was to learning XS, but reading "Advanced Perl Programming", where Sriram recomend to use SWIG I'm getting confusing to choose the "best" choice.

Sriram, just recommend XS because is bundled with Perl, and say that swig is a lot cleanner and less internals-oriented than XS.

I'd like know if someone can advice-me what is the best wrapper ?

Thanks

Solli Moreira Honorio
Sao Paulo - Brazil

  • Comment on $perl_extension ? require SWIG : require XS;

Replies are listed 'Best First'.
Re: $perl_extension ? require SWIG : require XS;
by sfink (Deacon) on Jan 07, 2007 at 08:31 UTC
    I have used all three.

    Inline::C will immediately work for simple things, and you'll be very gratified and encouraged -- enough so that you'll quickly expand to use it for all kinds of things and wrap your whole UI. Then you'll run into some small problem. Probably something to do with reference counts. You've made it so far already, it can't be that hard to just fix up the one little problem... soon you'll be learning large portions of XS, in a completely ass-backwards order, and just before you realize that you've completely lost track of what led you down that hellish path, you'll go insane.

    XS will require a big investment of time and documentation-reading up front, which will do you no good whatsoever, since everything you try will break for bizarre and subtle reasons. You'll think you understand it, and get more and more confident as you read the documentation and feel everything falling neatly into place. Until you actually try to use it, when nothing will seem to work at all like what you read, even though you can't find anything that directly contradicts what you read. Looking at the generated C code will make you even more confused.

    At some point, you'll realize that the only way to make any headway is to copy an existing XS interface and mutate it into what you want. This will mostly work, and you'll gradually make progress. Gradually but not steadily -- you will frequently encounter major roadblocks that will leave smatters of blood and brain pulp on the nearest brick wall. It's a bit like crossing a freeway with a chihuahua chewing on your ankle, getting flattened by several cars and an SUV on the way across. Still, everything you learn will be useful; you'll just have to gradually build up your knowledge in stages.

    SWIG is bliss. It effortlessly wraps everything you want (and far more, since it traces through a bunch of headers you didn't really intend). It presents the API to you in nice clean Perl.

    But it's slow. And 20% of the time, it doesn't work at all. Another 20% of the time, it almost works but does something blindingly idiotic. And when it doesn't work, it'll be like dissecting a frog in high school science class to figure out what's going wrong. You know -- those special high school science class frogs, that have groxbitner where the textbook says they should have lungs, sporkinits instead of intestines, and a blob of yerfikation in place of a heart. In other words, there are layers and layers of nonsensical internals specifically designed to make your brain dribble out your ears.

    All clear now?

Re: $perl_extension ? require SWIG : require XS;
by jettero (Monsignor) on Jan 06, 2007 at 13:33 UTC

    In my opinion, swig is for people who just want to get something working, have little interest in distribution, and don't have any interest in learning perl XS. If you went through the modules on CPAN looking at the ones that import C into perl, you'd find that nearly all of them use XS.

    I wouldn't call XS a wrapper either, it's actually perl's interface to call C functions natively. I could be semantically mistaken, but I think that's pretty close. XS can be tricky to learn, but I think it's worth it. I have loved every XS project I've completed so far.

    -Paul

Re: $perl_extension ? require SWIG : require XS;
by Util (Priest) on Jan 06, 2007 at 18:33 UTC
    I think Sriram's advice in A.P.P.1 should be considered very out-of-date.
    1. 1997: Advanced Perl Programming (First Edition) by Sriram Srinivasan
      • Pages on SWIG: 5, XS: 4, Both: 8, Inline: 0.
      • SWIG underwent a major rewrite between 1999 and 2001.
    2. 2002: Extending and Embedding Perl by Tim Jenness and Simon Cozens
      • Pages on SWIG: 6, XS: 200+, Inline: 10.
    3. 2005: Advanced Perl Programming (Second Edition) By Simon Cozens
      • Complete rewrite, with little overlap from the First Edition.
      • One whole chapter on Inline.
    From A.P.P.2 - "Packaging Inline Modules":
    In the past I'd always seen Inline::C as useful for prototyping, or a simple glue layer between C and Perl for quick hacks, and would discourage people from using it for the "serious" business of creating CPAN modules.
    However, Brian "Ingy" Ingerson has worked hard on these issues and there are now two equally suitable ways to write fully functional Perl modules in Inline, without bothering with XS.

      APP2 isn't quite up-to-date here:

      We now have InlineX::C2XS, InlineX::CPP2XS and InlineX::XS. The first two modules convert the code from Inline::C or Inline::CPP respectively into "stand-alone" XS. InlineX::XS is a wrapper for Inline::C which can auto-convert the inlined C (or rather the inlined XS) into "stand-alone" XS code during make dist time.

      This means that you can write a module using Inline::C and use that for a rapid development and benefit from the close Perl+C association that makes Inline::C attractive. Then, when you're ready to upload to CPAN, you just make dist and get an ordinary CPAN distribution that does not rely on Inline::C. Sounds whacky? Probably. Have a look at the documentation for a better explanation of the concept.

      Cheers,
      Steffen

Re: $perl_extension ? require SWIG : require XS;
by syphilis (Archbishop) on Jan 06, 2007 at 13:43 UTC
    I know a little bit about XS - just enough to be dangerous (ie "a little knowledge is a dangerous thing"). I couldn't help at all with SWIG (though "swigging grog" is what I've been doing for most of the evening :-)

    One advantage of XS over SWIG seems to be that help and advice is much more readily available for XS than it is for SWIG - simply because more people are using XS.

    Another advantage of taking the XS route, imho, is that Inline::C (which should perhaps have instead been called "Inline::XS") is available - which makes the learning curve so much gentler.

    Cheers,
    Rob
Re: $perl_extension ? require SWIG : require XS;
by almut (Canon) on Jan 06, 2007 at 19:36 UTC

    A few years ago I did a somewhat larger project, which aimed at providing a Perl scripting interface to some PDM (Product Data Management) system, which came with only a C/C++ API.

    I got hooked on the 'saving implementation time' thing, and started doing it with SWIG. Problem was I wanted to implement a convenient, more perl-style UI than the original API did provide. In other words, the conversion wrappers had to do a bit more work than mapping simple data types. Well, to make it short, I wasn't able to figure out how to do this with SWIG, though I tried hard.

    Ultimately, I ended up redoing the whole thing in XS. It wasn't that much more difficult, and the great advantage was that I essentially could do anything I wanted... So, if you should decide to go with SWIG (after having read the other responses), my advice would be to begin with wrapping the most complex function interface first (maybe after having acquainted yourself with a couple of preliminary easier exercises). This will minimize wasting time. (I had made the error to start with the simple things -- so I essentially did the project twice: 90% in SWIG, then once again in XS...)

    Lastly, and somewhat less seriously: using XS would likely boost your ego with that warm and fuzzy feeling of having done it the way most 'real hackers' would do ;)

      You should have done the Perlish API stuff in wrappers written in Perl that called the SWIG / XS / Inline::C wrappers. Minimize the amount of code you write in SWIG / XS / Inline::C and you'll be happier in the long run (as will your users). That's why I advocate using Inline::C (it encourages simple, C-friendly interfaces which encourages writing the complex interface in Perl).

      Lastly, and somewhat less seriously: using XS would likely boost your ego with that warm and fuzzy feeling of having done it the way most 'real hackers' would do ;)

      Ugh. The "cool" factor of writing ugly, fragile code.

      - tye        

        It depends. I'm working on a project that uses a lot of C & C++ code for convenience and speed, (but especially speed) and I would say, minimize the amount of interface code period.

        Interface code is overhead. Extending the API in C to match exactly what you need and then wrapping the new API in XS will probably give you the "best" results. If you're going for speed you need as few cross-language calls as possible.

        As an example, Audio::LADSPA wouldn't be able to generate sound in realtime if you needed a method call for each plugin and frame. That's why it takes buffers of floats and loops through them in C. Looping through half a million scalars a second in perl tends to slow your program down, but C will go through half a million floats in no time. That means it's probably better to write some "higher level" code that does the looping for you, so you only need 1 perl call for a batch of X floats.

        Another thing I've noticed is that SWIG's OO interface code is horribly slow (especially when getting/setting properties). That's usually only an issue when the API doesn't really match what you're trying to do, but still it's something to be mindful of.

        Cheers,
        Joost.

        You should have done the Perlish API stuff in wrappers written in Perl that called the SWIG / XS / Inline::C wrappers.

        As Joost said, it depends.   In the project I was referring to, SWIG simply didn't provide enough flexibilty to get the (sometimes weird and dynamic) data structures converted into anything which would have remotely made sense in Perl. In other words, doing this part in Perl essentially would've meant returning big chunks of memory and then fiddling with pack()/unpack() on the Perl side to create the objects as needed. This certainly wouldn't have made it less fragile. Changing the C side wasn't an option, as it was a closed source third party library.

        Well, I don't want to go into further details. I'd just like to point out that in the end, the flexibility of XS did allow me to write less and faster, i.e. more-to-the-point code than SWIG did.

        Ugh. The "cool" factor of writing ugly, fragile code.

        Did you notice the smiley?

Re: $perl_extension ? require SWIG : require XS;
by Joost (Canon) on Jan 06, 2007 at 21:57 UTC
    SWIG is nice if you've got a decent C API already and you don't want to be bugged down with the perl internals and/or you want to support multiple languages. XS is great for precise control or if you need to do a lot of customization of the API to make it work perlishly and you can't afford to do all that wrapping in perl. In other words, it depends.

Re: $perl_extension ? require SWIG : require XS;
by Anonymous Monk on Jan 06, 2007 at 17:02 UTC
Re: $perl_extension ? require SWIG : require XS;
by sgt (Deacon) on Jan 08, 2007 at 13:32 UTC

    I think personally that knowing a bit of XS is very useful (especially if at some point you want to do core hacking). Still for easy prototyping Inline::C is just great; for packaging into a a module InlineX::C2XS is useful (or at least the ideas are useful to know what and where you eventually need to patch). You can also try InlineX::XS

    If you want to interface for speed, I think it is important to know that going in-and-out of perl takes time, and that you probably want to minimize that. There are some tricks out there: one of the most useful was mentioned on the p5p list and is used by Imager. Still the most serious problem is using a contract of the type malloc/free, while wanting automatic (end of scope) "freeing" on the perl side; for this it is probably better to use ideas like that of the Scope::Guard module (and the documentation mentioned therein). Arena-like programming can be ok too: you "malloc" (for each new object) on the C-side and "free" only in bunchs on the perl side (in the end it is probably better to do this via special objects -- thus part of the perl API).

    hth --stephan