Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer
 
PerlMonks  

"symbol lookup error" message help

by Special_K (Monk)
on Jan 20, 2023 at 20:41 UTC ( [id://11149732]=perlquestion: print w/replies, xml ) Need Help??

Special_K has asked for the wisdom of the Perl Monks concerning the following question:

I have a collection of perl modules that were installed locally (/home/user_foo/perl_modules/lib/perl5) using cpanm (cpanm -L$PWD <module_name>). This particular machine is running RHEL7 and has the following 2 perl versions installed:

/tool/bin/perl (v5.12.2) /usr/bin/perl (v5.16.3)

Scripts created on this machine start with #!/tool/bin/perl and have use lib '/home/user_foo/perl_modules/lib/perl5' to access the locally-installed modules. Note /home/user_foo/perl_modules/lib/perl5 is on a network and accessible to all machines. If I try to run those scripts on a different machine running RHEL8, I receive the following error:

/tool/bin/perl: symbol lookup error: /home/user_foo/perl_modules/lib/p +erl5/x86_64-linux/auto/List/Util/Util.so: undefined symbol: Perl_gv_i +nit

This RHEL8 machine has the following versions of perl installed:

/tool/bin/perl (v5.20.0) /usr/bin/perl (v5.26.3)

Based on some searching it seems the error is related to a mismatch between the version of perl being run vs the version of perl used to originally install the modules. If I create a new local perl module installation directory on the RHEL8 machine:

/new_location/perl_modules/5.20/lib/perl5

then reinstall the modules using cpanm (cpanm -L$PWD <module_name>), I notice the following things:

1. If I change my scripts to have use lib '/new_location/perl_modules/5.20/lib/perl5' but keep the #!/tool/bin/perl, the scripts work fine.
2. If I change my scripts to have use lib '/new_location/perl_modules/5.20/lib/perl5' and also change #!/tool/bin/perl to #!/usr/bin/perl, the scripts still work fine even though the perl version is now different.
3. The /new_location/perl_modules/5.20/lib/perl5 has a 5.26.3/x86_64-linux-thread-multi subdirectory that is empty

What exactly is causing the original error above? I suspect it is some sort of mismatch issue between the version of perl used to install the modules vs the version being run, but that leads to the following questions:

1. How do I specify which version of perl is being used when I install a module using cpanm?
2. Why do the new modules I installed to /new_location/perl_modules/5.20/lib/perl5 work for both /tool/bin/perl (v5.20.0) and /usr/bin/perl (v5.26.3) if module installations are version-dependent? Or am I misunderstanding the real issue here?

Replies are listed 'Best First'.
Re: "symbol lookup error" message help
by choroba (Cardinal) on Jan 20, 2023 at 21:09 UTC
    Pure Perl modules can work in different versions of Perl. XS modules need to be compiled when installed, using the same compiler as the Perl they are being installed to.

    cpanm (or any other client) installs the modules where you tell it to do it. It will use the perl you installed it for, i.e. the perl that runs it.

    map{substr$_->[0],$_->[1]||0,1}[\*||{},3],[[]],[ref qr-1,-,-1],[{}],[sub{}^*ARGV,3]
      > XS modules need to be compiled when installed, using the same compiler as the Perl they are being installed to.

      Can you please elaborate on this? I'm not familiar with XS, but a brief search turns up the "xsubpp" utility, which is apparently used to compile Perl XS code into C code. If I run xsubpp -v on each of the two machines, I receive the following output:

      RHEL7: xsubpp version 2.21 RHEL8: xsubpp version 3.24

      These xsubpp versions seem completely separate from any version of perl installed on the system. Are you saying that when an XS module such as List::Util is installed, whatever version of xsubpp happens to be installed on the system is used to compile the C code and from then on, only machines with that same version of xsubpp installed can use the module?

        Don't care about xsubpp. When you upgrade Perl, simply reinstall all the modules for the new version.

        Alternatively, use PerlBrew to keep several Perl versions. Each version needs its own set of modules (again, in fact, only XS modules need to be different, but if you don't know which these are, just install all of them for each version).

        map{substr$_->[0],$_->[1]||0,1}[\*||{},3],[[]],[ref qr-1,-,-1],[{}],[sub{}^*ARGV,3]

      > XS modules need to be compiled when installed, using the same compiler as the Perl they are being installed to.

      What exactly is being compiled into what in this context, and by what? Is it C/C++ -> assembly? Perl -> something else? Is xsubpp is the compiler in the context of the quoted message? I'm just trying to understand why the same version of perl is required to run modules that use XS as was used to install them.

        The source code in XS is being compiled into the machine code. See perlxs for details, but it's rather advanced, so maybe start with Wikipedia that has a nice basic introduction.

        map{substr$_->[0],$_->[1]||0,1}[\*||{},3],[[]],[ref qr-1,-,-1],[{}],[sub{}^*ARGV,3]

        You could roughly define XS as a macro language - it is only a little bit more than pure C with a bunch of predefined macros. It is preprocessed by xsubpp into C, then compiled with the appropriate C compiler into an object file (i.e. machine code).

        It needs the same version of perl mainly because it deals with a bunch of perl's internal data structures - when compiled, that bakes in knowledge of the size of those structures, and the offset of various elements within them. It also bakes in knowledge of the signature of the various perl internal functions it may call, along with compiler-specific calling conventions.

Re: "symbol lookup error" message help
by ikegami (Patriarch) on Jan 20, 2023 at 22:10 UTC

    How do I specify which version of perl is being used when I install a module using cpanm?

    You should have two cpanm, one installed with each perl. They will have different shebang (#!) lines.

    It's simply a question of using the right cpanm (e.g. by using its full path to launch it).

    head -n 1 /tool/bin/cpanm # #!/tool/bin/perl /tool/bin/cpanm Some::Module # Installs for /tool/bin/perl

    The cpanm script itself should work with any perl, so you could explicitly specify the perl to use as a fallback.

    /tool/bin/perl "$( which cpanm )" Some::Module

    Why do the new modules I installed to /new_location/perl_modules/5.20/lib/perl5 work for both /tool/bin/perl (v5.20.0) and /usr/bin/perl (v5.26.3)

    There's no guarantee that using different build will fail.

    There's merely no effort made to ensure that it won't fail.

      Given a directory where perl modules are already installed (ex: /home/user_foo/perl_modules/lib/perl5 from my original message), how do I determine what version of perl was used to install them?
        Use Module::Metadata (core) to examine the path:
        /usr/bin/perl -MModule::Metadata -le 'print Module::Metadata->find_mod +ule_by_name("Class::Struct")'
        /System/Library/Perl/5.18/Class/Struct.pm
      > There's no guarantee that using different build will fail.

      > There's merely no effort made to ensure that it won't fail.

      So is the best practice to just have a complete modules directory corresponding to each version of installed perl, no matter how minor the version update? For example, if I have the following (hypothetical) versions installed:

      5.26.1 5.26.2 5.26.3 5.26.4

      Then each one of those should have its own complete modules directory?

        Perl actually guarantees that you can upgrade between minor versions without needing to recompile. But that implies that the new minor version was compiled with exactly the same compiler and compiler options as the old one.

      Here's what I don't understand:
      I ran the following:


      > which cpanm /bin/cpanm > head -n 1 /bin/cpanm #!/usr/bin/perl

      I know that /usr/bin/perl on this machine is 5.26.3, yet if I look at the most recent cpanm build.log file, it says:

      cpanm (App::cpanminus) 1.7044 on perl 5.026003 built for x86_64-linux- +thread-multi

      Why does the log file list perl version 5.026003 when /usr/bin/perl is 5.26.3?

        Because $], the variable containing the Perl version number has the value 5.026003. See the documentation in perlvar.

        5.026003 is 5.26.3.

        5.026003 |_||_| 26 3
        See "perldoc version" (version on the CPAN).

        perl -le 'print $]'
        5.026003
        perl -le 'print $^V'
        v5.26.3
        perl -Mversion -le 'print version->parse("5.026003")->normal'
        v5.26.3
        perl -Mversion -le 'print version->declare("v5.26.3")->numify'
        5.026003

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://11149732]
Approved by choroba
Front-paged by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others romping around the Monastery: (5)
As of 2024-04-19 15:40 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found