tsee++

Well done, Steffen for the sterling work you are doing with this, and the recent developments with PAR.

Although I have a C background, I've been too scared to date, to get my hands dirty with coding XS, and at least one of my CPAN modules would benefit from some C routines. I've also been scared of Inline::C and the voodoo involved in getting it working with CPAN and package managers, so I welcome your insights.

I'm wondering if it's a design defect of Inline::C that it needs to runtime compile the code. This is making a big assumption, that your target machine has a working C compiler. I'm very interested in your workaround, especially one that will work with CPAN and package managers.

Please also consider all the package managers that are out there: e.g. from the Linux world - dpkg, rpm and evolve (and others).

I'm wondering if it's a design defect of Inline::C that it needs to runtime compile the code

I don't really see it that way. It only needs to "runtime compile the code" the first time the code is run. Subsequent runnings of the code simply load what was compiled the first time ... no need to re-compile it. In that way it's no different to a perl extension - ie it needs to be compiled only once.

This is making a big assumption, that your target machine has a working C compiler

This statement implies to me that you're thinking Inline::C will work without a C compiler - which is not the case. (Sorry ... perhaps you meant something entirely different.)

Cheers,
Rob

I think you've not appreciated my point, though I admit I probably didn't express it very well.

A CPAN style distribution has a 4 phase install process: configure, build, test, install.

perl Makefile.PL  # Configure
make  # build
make test # test
make install # install
[download]

or

perl Build.PL  # Configure
./Build  # build
./Build test # test
./Build install # install
[download]

For an XS or swig module, compilation of C code takes place in the build phase. This phase generates usable deliverables: perl code, binaries, man pages, scripts, etc. under the blib directory. The test harness mechanism works on these deliverables pre-installation, so you can run tests against your distribution before it gets installed.

Please correct me if I'm wrong in any of this - this is based on my understanding of I::C and I could be talking rubbish here. An Inline::C module won't be compiled during the build phase, but the first time it is run. While this should (hopefully!) happen as a side effect of the test phase, these binaries won't be listed as deliverables, and won't be installed as part of the install phase. So, the binaries get built, the first time the module gets called after it has been installed.

This won't happen as root, most of the time, so presumably the compile is redone for each user, the first time they run the application that uses the script. If this isn't the case, there could be permissions problems and all sorts of environmental wierdness happening.

So far, I have been referring to the CPAN model for installing a distribution. When it comes to package managers, the idea (usually) is that there is no compile step done on the target machine. Said target machine could be Windows with a PPM install, Debian with an install via apt-get or whatever. The point is that package managers assume that no compile is needed on the target, and no compiler is necessary. This implies that distributions for such package managers may be platform specific and contain pre-built binaries.

The important distinction here is that the build machine has a C compiler, indeed the full development tool set. But the target machine is not guaranteed to have any of this, it's a run-time environment.

PAR takes this idea a stage further, and the target machine doesn't even need to have perl installed.

Hopefully this has clarified my argument sufficiently.

--

Oh Lord, won’t you burn me a Knoppix CD ?
My friends all rate Windows, I must disagree.
Your powers of persuasion will set them all free,
So oh Lord, won’t you burn me a Knoppix CD ?
(Missquoting Janis Joplin)

Okay, I got a little further. Along the way, various issues of varying potential to make me really sick came up. Supposing I release Module::Inline::C to CPAN, here's the recipe for converting your Inline::C-based CPAN distribution to use Module::Inline::C. This works for MakeMaker-based distributions only at this point.

• Replace all invocations of Inline with M::I::C:

  # use Inline C => '<<HERE';
  # becomes
  use Module::Inline::C <<'HERE';
  ...
  HERE
  [download]

• Replace the 1; that ends the module/.pm file with use Module::Inline::C 'END';
• The version declaration of your module needs to be accessible at compile time and also determinable by MakeMaker which uses a simple regex to extract it. That means you need to do this:

  # our $VERSION = '0.01';
  # becomes
  our $VERSION = '0.01';
  BEGIN {$VERSION = '0.01'};
  [download]

  (Similarly if you prefer use vars '$VERSION'.)
• You need to add a bit to your Makefile.PL. For module Foo::Bar, this might look like:

  use ExtUtils::MakeMaker;
  
  my $module_file = 'lib/Foo/Bar.pm';
  WriteMakefile(
      NAME              => 'Foo::Bar',
      VERSION_FROM      => $module_file,
      PREREQ_PM         => {
         'Module::Inline::C' => '0', # instead of Inline
      },
      ($] >= 5.005 ?
        (ABSTRACT_FROM  => $module_file,
         AUTHOR         => 'Your Name <and@e.mail>') : ()),
      dist => {
          PREOP => "perl -MModule::Inline::C::MM=\$(DISTNAME)-\$(VERSION
  +) -c $module_file",
      },
  );
  [download]

  The dist => ... part specifies that before tar-ing up the distribution file, the Inline::C to XS conversion should take place. The MANIFEST of the generated distribution is updated automatically.

That should be mostly it. Afterwards, anybody downloading and installing your distribution should be able to do the usual

perl Makefile.PL
make # compiles XS => so/dll into blib/
make test # doesn't use Inline!
make install # no more Inline for this module, ever.
[download]

Of course, the user is free to delete the .xs files from the distribution and use Inline instead. Using Inline::C can be much more enjoyable for development (IMHO).

There's still a lot of cleaning up to do as well as adding documentation and implementations for Module::Build and Module::Install. Additionally, if you have a distribution with several XS (or rather: Inline::C) modules, you might run into trouble with this. But you're welcome to read up on the issue. It's not specific to this but rather common to all XS distributions.

Steffen

Update: I forgot one item in the list.

A first cut of a complete Inline::XS (formerly Module::Inline::C) implementation is ready.

I haven't uploaded it to CPAN yet since I want the inline community's blessing for the namespace choice before doing that. Until it hits CPAN, you can grab a copy of it from my website. The link might go down as soon as the real thing hits CPAN.

Steffen