Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked

Rationale for why successive Perl releases are not XS-compatible?

by vrk (Chaplain)
on Mar 17, 2017 at 12:04 UTC ( #1185014=perlquestion: print w/replies, xml ) Need Help??

vrk has asked for the wisdom of the Perl Monks concerning the following question:

Dear monks,

at $work, we use SWIG to generate a Perl wrapper for a large C++ library. SWIG creates glue code in XS and pure Perl based on the C++ class definitions, and the XS code is then compiled against the header files of a particular version of Perl. This works great and you can use the same SWIG interface file to generate a Python, Java and C# wrapper as well.

The problem is, when the XS code is compiled against a particular version of Perl, say 5.18, the resulting compiled library is only compatible with the same (major) version of Perl. Our customers use a range of Perl versions, from 5.10 through to the latest one, so we have to compile and provide support for an ever increasing number of versions, even though the actual C++ library is exactly the same in each case. It was manageable for 5.6 and 5.8, which were long-lived versions, but it's become exasperating since the move to annual Perl releases.

Can you point me to any resources that describe or discuss the rationale for why XS code could not be made binary compatible between Perl versions? Or why there couldn't be some kind of reduced XS API that would stay binary compatible even if some parts change? Which authority should I petition to make it so?

Replies are listed 'Best First'.
Re: Rationale for why successive Perl releases are not XS-compatible?
by salva (Canon) on Mar 17, 2017 at 12:21 UTC
    Mostly because perlapi is not a thoroughly and well designed API but just a subset of the Perl internals exposed, including functions, structures and macros.

    That means that whenever the representation of any structure used by perl internally changes, all the code must be recompiled and as you can imagine, evolving the language requires changing its internals...

Re: Rationale for why successive Perl releases are not XS-compatible?
by marto (Cardinal) on Mar 17, 2017 at 12:34 UTC

      chromatic's blog post is a good read, but as he points out, it would actually be beneficial to have an extra layer between perl's internals and user modules to prevent XS code from accessing the internals. I don't disagree with the necessity for changing the internals periodically. It would just be nice to have a binary compatible interface that shields you from those changes...

        The perl api mostly consists of inline macros, for speed. It should be relatively simple to design a semistable binary api: just provide out-of-line-compiled functions for each of the inline macros. There would be a slight speed penalty, which is why nobody has done it.
Re: Rationale for why successive Perl releases are not XS-compatible?
by stevieb (Canon) on Mar 17, 2017 at 13:22 UTC

    Perl 5 Porters are the authority you speak of. You can contact them through their mailing list. Perhaps it would be prudent to peruse the archives (in the same link) before reaching out in case it's been brought up before.

      A quick search with a favourite search engine shows it has been discussed a couple of times in the past few years, so I'll need to spend some time reading through the archives.

Re: Rationale for why successive Perl releases are not XS-compatible?
by Anonymous Monk on Mar 17, 2017 at 16:27 UTC
    Just spin up a server, install Perlbrew on it and install versions of Perl as per your customers' needs. Use Jenkins to build your software on the target Perl. It's easier than you might think.
      easier when it works, if it works
Re: Rationale for why successive Perl releases are not XS-compatible?
by vrk (Chaplain) on Mar 27, 2017 at 07:55 UTC

    Some blue sky thinking... FFI::Platypus is a libffi interface for Perl. You could modify SWIG to produce pure Perl glue code targeting FFI::Platypus instead of producing XS. Then you only need to worry about having FFI::Platypus installed on customer systems. But it would only solve the problem if 1) someone wrote a SWIG backend (some months of work) and 2) FFI::Platypus is easy to install (haven't checked).

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://1185014]
Approved by haukex
Front-paged by LanX
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others contemplating the Monastery: (5)
As of 2023-02-06 20:57 GMT
Find Nodes?
    Voting Booth?
    I prefer not to run the latest version of Perl because:

    Results (36 votes). Check out past polls.