Beefy Boxes and Bandwidth Generously Provided by pair Networks
XP is just a number
 
PerlMonks  

Re: PerlIO::Layers 0.011 fails to build in macOS 10.13.3

by bliako (Monsignor)
on Mar 26, 2018 at 13:39 UTC ( [id://1211741]=note: print w/replies, xml ) Need Help??


in reply to PerlIO::Layers 0.011 fails to build in macOS 10.13.3

The problem is when macro:
/* concatenating with "" ensures that only literal strings are accepte +d as * argument */ #define STR_WITH_LEN(s) ("" s ""), (sizeof(s)-1)

is passed a non-literal string (i.e. something which is not quoted, for example the XS_VERSION => 0.011 (via XS_VERSION_BOOTCHECK) which is just a naked 0.011 . Actually this is mentioned in the comments before the macro def (see above code).

By double-quoting contents of -DVERSION and -DXS_VERSION, the command line you pasted succeeds for me:

-DVERSION="\"0.011\"" -DXS_VERSION="\"0.011\""
The complete command line:
cc -I/opt/perlbrew/perls/perl-5.18.0/lib/5.18.0/darwin-2level/CORE -DV +ERSION="\"0.011\"" -DXS_VERSION="\"0.011\"" -c -fno-common -DPERL_DAR +WIN -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include + -I/opt/local/include -arch x86_64 -framework System -Wa,--noexecstac +k -I/opt/local/include -I/opt/X11/include -O3 -o lib/PerlIO/Layers.o + lib/PerlIO/Layers.c

Of course I am not sure if "causes no error" or "succeeds" is the same as "works as intended". But an unquoted 0.011 can be found in the resultant object file:

$ strings lib/PerlIO/Layers.o 0.011 v5.18.0 ...
bliako

Replies are listed 'Best First'.
Re^2: PerlIO::Layers 0.011 fails to build in macOS 10.13.3
by syphilis (Archbishop) on Mar 27, 2018 at 01:28 UTC
    The problem is when macro ... is passed a non-literal string

    Good catch.
    I had seen the comments in handy.h, and noted the absence of quotes in perlboy_emeritus' build - but still failed to put it all together properly.
    (I had myself thinking that the strange ("" s "") construct must have been magically enforcing the requisite stringification but, of course, it merely throws the error if s is not a string.)

    So the bug is in the version of Module::Build that perlboy_emeritus has installed - and any perl extension that is built with ExtUtils::MakeMaker (instead of Module::Build) will be free from this issue.
    The version of Module::Build that I was using (on Windows) is 0.4214, which was fine - so I guess another alternative might have been to simply update Module::Build to a later version.

    Here's a little C demo that I ran just to satisfy my limited understanding:
    C:\_32\C>type try.c #include <stdio.h> int main (void) { #define FOO(s) printf("%s\n", "" s ""); FOO(VER); return 0; } C:\_32\C>gcc -o try.exe -DVER="\"0.1\"" try.c C:\_32\C>try 0.1 C:\_32\C>gcc -o try.exe -DVER=0.1 try.c try.c: In function 'main': try.c:5:2: error: expected ')' before numeric constant
    Cheers,
    Rob

      with your little c-program:

      gcc -DVER="0.1" -E try.c

      does the preprocessor pass and expands FOO to:

      printf("%s\n", "" 0.1 "")

      and

      gcc -DVER="\"0.1\"" -E try.c

      expands it to:

      printf("%s\n", "" "0.1" "")

      so, whereas

      printf("%s\n", 0.1)

      would have caused just a warning at compile time and a crash later,

      printf("%s\n", "" 0.1 "")

      causes an error right away

      and
      printf("%s\n", "" "0.1" "")

      is legitimate.

      bliako
        printf("%s\n", "" "0.1" "") didn't immediately look like valid C to me (and it's not valid perl).
        But C does allow that type of concatenation - whereas I guess perl doesn't really need to.
        Even in assignment one can do char *x = "hello" "0.01" "world";

        Cheers,
        Rob
Re^2: PerlIO::Layers 0.011 fails to build in macOS 10.13.3
by perlboy_emeritus (Scribe) on Mar 26, 2018 at 17:02 UTC
    Bravo bliako :-) However, the behavior of that module's build process is still bizarro. I went into ./_build and grepped for '0.011' and found file build_params to contain:
    'dist_abstract' => 'Querying your filehandle\'s capabilities', 'dist_version' => '0.011', 'build_bat' => 0, 'dynamic_config' => 1, 'c_source' => undef, 'extra_compiler_flags' => [], 'verbose' => undef, 'build_class' => 'Module::Build', 'quiet' => undef, 'install_sets' => {}, 'use_rcfile' => 1, '_added_to_INC' => [ '/Users/carly/perl5/lib/perl5/5.18.2/darwin-thread-multi-2level' +, '/Users/carly/perl5/lib/perl5/5.18.2', '/Users/carly/perl5/lib/perl5/darwin-thread-multi-2level', '/Users/carly/perl5/lib/perl5', '/Users/carly/src/perl5/lib' ],
    among other expressions. This is obviously not a hand-generated file. So, I double quoted that version number => "0.011" and then did 'perl ./Build.PL' and then Build and 'Build test'. Like magic, all was well, all tests were successful but... get this, the module Module::Build as in:
    # This should have just enough arguments to be able to bootstrap the r +est. my $build = Module::Build->resume ( properties => { config_dir => '_build', orig_dir => $orig_dir, }, ); $build->dispatch;
    reset that double-quoted string to '0.011'. Man, talk about perverse. I poked Leon Timmerman's, the author/maintainer of PerlIO::Layers with this issue but have yet to hear back. How do we alert all of the Perl-heads who might be inadvertently creating this problem?

      As <<syphilis>> said:

      I notice that XS_VERSION_BOOTCHECK has changed since perl-5.18 and no longer directly calls STR_WITH_LEN.

      If I were you I would upgrade my perl and never think about it again.

      In OSX something that really works for me well is:

      1. perlbrew (note Homebrew and perlbrew are not the same thing, perlbrew allows you to select which version of perl to use and also downloads, installs and uses any version of perl you instruct it to),
      2. cpanm for installing perl modules,
      3. important: if using perlbrew, shebang should now be: #!/usr/bin/env perl

      #1 makes a lot of difference and definetely pokes a wooden pole in the chest of apple's perl.

      glad you found a solution.

        Hello bliako,

        Perlbrew is cool! I tried it and it worked, right out of the box to build 5.20.3. First time I've built Perl from sources and it is amazing. I initially got two failed tests in Memoize but I believe the tests failed due to a time out. I found the failed tests and ran their .t file manually and they passed. Just to be sure, I ran all the tests for that module with a find command with -exec to have perl read all of the test files in that directory and all passed, so I forced the install of 5.20.3.

        However, I can't totally give up on installing PerlIO::Layers in the native perl that ships with High Sierra, as others may be struggling with this too. The problem is that 'version' is defined in only one place and is used to initialize both the .pm file and the .o file by way of compiler -D macros. Setting it for the compiler hoses the .pm and setting it for the module causes the compiler to choke. There is no way to set XS_VERSION and VERSION manually, only indirectly by way of 'dist_version'. Therefore, the automated build/test process is broken. ExtUtils::CBuilder::Base sets compiler -D macros with this fragment of perl I modified:
        #!/usr/bin/env perl -w ##!/usr/bin/env perl -wd use strict; use Data::Dumper; $Data::Dumper::Purity = 1; my %args = ( XS_VERSION => '\'"0.011"\'', VERSION => '\'"0.011"\'', ); print arg_defines(%args), "\n"; exit 0; sub arg_defines { my (%args) = @_; # my ($self, %args) = @_; return map "-D$_=$args{$_}", keys %args; } __END__
        which when run yields:
        $ macro-D.pl -DXS_VERSION='"0.011"'-DVERSION='"0.011"'

        Data::Dumper is syntactic sugar for when I run in Perl's debugger. Whenever anything like these strings is set in 'dist_version', the build fails and 00-compile.t fails.

        To manually build the module and its shared library do: ./Build clean and then ./Build and extract the two cc commands, one to compile and one to link in the bundle, run them manually and all will be well. With this work-around I was able to do what triggered this thread, namely, build PDL.

        So, it isn't absolutely necessary in macOS High Sierra to resort to perlbrew but it is a whole lot easier. I am a hacker at heart but I suspect most mac users are not.

        Rats! I am now running Perl 5.26.1 in Homebrew and building PerlIO::Layers is still broken, only at this point my work-around of editing version => '0.011' expressions and making them "0.011" (three places, *.yml files and ./_build/build_params) and then doing perl ./Build.PL, ./Build and Build test does not yield a shared object. Indeed, Layers.o in this latest build is a file size = 5048 bytes while the one I built using that work-around with Perl 5.18.2 is size = 92676 bytes. The 00.compile.t test fails even though there is a Layers.o file in ../lib. That test uses -Mblib to find the shared object, which obviously isn't a loadable object. And as was noted, XS_VERSION_BOOTCHECK does not appear to be invoked, which means now there is some other reason why the module will not build. Back to the drawing board...

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1211741]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others examining the Monastery: (3)
As of 2024-04-19 15:44 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found