Beefy Boxes and Bandwidth Generously Provided by pair Networks
Don't ask to ask, just ask
 
PerlMonks  

XSLoader/DynaLoader Can't Locate Library

by wbirkett (Acolyte)
on Jun 10, 2020 at 17:58 UTC ( [id://11117912]=perlquestion: print w/replies, xml ) Need Help??

wbirkett has asked for the wisdom of the Perl Monks concerning the following question:

Hello, I built an XS module that interfaces the Rmath statistics library. This was straightforward, and works fine, when installed in the normal location (/Library/Perl/5.18 on my Mac, the Rmath library is in /usr/local/lib). Now, I want to relocate the module and library to another location, in this case a TextMate bundle at ~/Library/Application Support/TextMate/Pristine Copy/Bundles. With the library in /usr/local/lib, the module works fine. But when I move the library to another location, the loading fails with this message:

Can't load '/Users/wbirkett/Library/Application Support/TextMate/Prist +ine Copy/Bundles/PressCal.tmbundle/Support/lib/perl5/darwin-thread-mu +lti-2level/auto/ICC/Support/Rmath/Rmath.bundle' for module ICC::Suppo +rt::Rmath: dlopen(/Users/wbirkett/Library/Application Support/TextMat +e/Pristine Copy/Bundles/PressCal.tmbundle/Support/lib/perl5/darwin-th +read-multi-2level/auto/ICC/Support/Rmath/Rmath.bundle, 1): Library no +t loaded: libRmath.dylib Referenced from: /Users/wbirkett/Library/Application Support/TextMat +e/Pristine Copy/Bundles/PressCal.tmbundle/Support/lib/perl5/darwin-th +read-multi-2level/auto/ICC/Support/Rmath/Rmath.bundle Reason: image not found at /System/Library/Perl/5.18/XSLoader.pm lin +e 71. at /Users/wbirkett/Library/Application Support/TextMate/Pristine Copy +/Bundles/PressCal.tmbundle/Support/lib/perl5/darwin-thread-multi-2lev +el/ICC/Support/Rmath.pm line 183. Compilation failed in require at /Users/wbirkett/Desktop/Rmath_test.pl + line 1. BEGIN failed--compilation aborted at /Users/wbirkett/Desktop/Rmath_tes +t.pl line 1.

From the documentation for XSLoader.pm, "To reduce the overhead as much as possible, only one possible location is checked to find the extension DLL (this location is where make install would put the DLL). If not found, the search for the DLL is transparently delegated to DynaLoader , which looks for the DLL along the @INC list." The library is definitely located in an @INC location, and the delegation to DynaLoader fails. I've tried many ideas to get this working, including setting the @DynaLoader::dl_library_path variable, and using DynaLoader.pm rather that XSLoader.pm, with no success.

I am completely stumped, and appeal to your monk wisdom for a solution.

Replies are listed 'Best First'.
Re: XSLoader/DynaLoader Can't Locate Library
by bliako (Monsignor) on Jun 10, 2020 at 19:47 UTC

    Perhaps there are some extra files libRmath.dylib depends on which you have not moved? Perhaps this can tell you more, if it works: export LD_DEBUG=all . Or use the nuclear deterrent: strace -f -e trace=open,close,read,write,connect,accept perl xx.pl | grep libRmath (and without the grep perhaps).

    Have you tried to use only DynaLoader, re: https://metacpan.org/pod/XSLoader#Migration-from-DynaLoader

    What does this show?

    require DynaLoader; print "@DynaLoader::dl_library_path\n";

    bw, bliako

      Thank you, bliako, for your suggestions. Unfortunately, strace is not available on my Mac. I wasn't able to coax any additional debug information. I also tried setting the $dl_debug flag.

      I did modify the Rmath.pm file to use the DynaLoader module instead of XSLoader, sort of a reverse migration. The result was the same, even when pushing the exact .dylib folder path onto @DynaLoader::dl_library_path. When I add code to the .bs file, it is executed, so I used that to modify @DynaLoader::dl_library_path.

      The default values for @DynaLoader::dl_library_path are '/usr/lib' and '/usr/local/lib'. The library loads correctly from either of those locations. I was able to load the library from other locations using the dl_load_file() function.

        When I add code to the .bs file, it is executed, so I used that to modify @DynaLoader::dl_library_path.

        What about adding that code in the actual .pm file you created (Rmath.pm?) like:

        package Rmath; ... our @ISA = qw( ... DynaLoader ... ); require DynaLoader; push @DynaLoader::dl_library_path, 'my-nonstandard-lib-path'; print "XXX=$_\n" for @DynaLoader::dl_library_path; bootstrap <Yourmodule> $VERSION;

        $dl_debug : I meant doing export LD_DEBUG=all in your terminal/shell/prompt before testing your program (on that same terminal).

        There are two very remote possibilities: 1) the dylib extension is not tried at all in enumerating possible library names (unlikely because it finds it in standard lib location), 2) the spaces in the library path (in OSX hmm that's not likely).

        ps. the reason I suggested this reverse-migration is to investigate DynaLoader's behaviour without the front-man.

        >strace is not available on my Mac

        man dtruss

        «...strace is not available on my Mac.»

        dtruss?

        «The Crux of the Biscuit is the Apostrophe»

        perl -MCrypt::CBC -E 'say Crypt::CBC->new(-key=>'kgb',-cipher=>"Blowfish")->decrypt_hex($ENV{KARL});'Help

Re: XSLoader/DynaLoader Can't Locate Library
by perlfan (Vicar) on Jun 11, 2020 at 01:42 UTC
    Try making sure your alternative paths are added to your environment via LD_LIBRARY_PATH (with /path/to/static-or-shared-libraries/lib) and to LD_INCLUDE_PATH contains your /path/to/headerfiles/to/include. You talk of DLLs and Mac don't make sense to me, but what I do know is that if you're compiling with gcc, then those two environmental variables are used by the linker and if you're linking a shared library rather than compiling it statically, then you're gonna need LD_LIBRARY_PATH set correctly at run time. As far as I know, this is all controlled outside of whatever Perl harness you're using to bootstrap the compilation and facilitating the running of your XS module.

      Thanks for the suggestion, perlfan.

      I exported the path to the folder containing the .dylib, and here is the result

      WBBirkett-M:~ wbirkett$ export LD_LIBRARY_PATH=/Users/wbirkett/Library +/Application\ Support/TextMate/Pristine\ Copy/Bundles/PressCal.tmbund +le/Support/lib/perl5 WBBirkett-M:~ wbirkett$ env TERM_PROGRAM=Apple_Terminal SHELL=/bin/bash TERM=xterm-256color TMPDIR=/var/folders/kf/hd1w1x888xj_x1018f8m3f2h0000gn/T/ Apple_PubSub_Socket_Render=/private/tmp/com.apple.launchd.le70PQDJKe/R +ender TERM_PROGRAM_VERSION=404.1 TERM_SESSION_ID=CC263E8D-8730-4F07-9752-F779AA2A8ADD USER=wbirkett LD_LIBRARY_PATH=/Users/wbirkett/Library/Application Support/TextMate/P +ristine Copy/Bundles/PressCal.tmbundle/Support/lib/perl5 SSH_AUTH_SOCK=/private/tmp/com.apple.launchd.L73CgT5E19/Listeners PATH=/Users/wbirkett/.rbenv/shims:/usr/local/bin:/usr/local/sbin:/usr/ +local/mysql/bin/:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/opt/X1 +1/bin:/usr/local/git/bin:/Library/TeX/texbin PWD=/Users/wbirkett LANG=en_US.UTF-8 XPC_FLAGS=0x0 RBENV_SHELL=bash XPC_SERVICE_NAME=0 SHLVL=1 HOME=/Users/wbirkett LOGNAME=wbirkett DISPLAY=/private/tmp/com.apple.launchd.pfkv04jVPR/org.macosforge.xquar +tz:0 _=/usr/bin/env WBBirkett-M:~ wbirkett$ perl -I/Users/wbirkett/Library/Application\ Su +pport/TextMate/Pristine\ Copy/Bundles/PressCal.tmbundle/Support/lib/p +erl5 /Users/wbirkett/Desktop/Rmath_test.pl Can't load '/Users/wbirkett/Library/Application Support/TextMate/Prist +ine Copy/Bundles/PressCal.tmbundle/Support/lib/perl5/darwin-thread-mu +lti-2level/auto/ICC/Support/Rmath/Rmath.bundle' for module ICC::Suppo +rt::Rmath: dlopen(/Users/wbirkett/Library/Application Support/TextMat +e/Pristine Copy/Bundles/PressCal.tmbundle/Support/lib/perl5/darwin-th +read-multi-2level/auto/ICC/Support/Rmath/Rmath.bundle, 1): Library no +t loaded: libRmath.dylib Referenced from: /Users/wbirkett/Library/Application Support/TextMat +e/Pristine Copy/Bundles/PressCal.tmbundle/Support/lib/perl5/darwin-th +read-multi-2level/auto/ICC/Support/Rmath/Rmath.bundle Reason: image not found at /System/Library/Perl/5.18/XSLoader.pm lin +e 71. at /Users/wbirkett/Library/Application Support/TextMate/Pristine Copy +/Bundles/PressCal.tmbundle/Support/lib/perl5/darwin-thread-multi-2lev +el/ICC/Support/Rmath.pm line 183. Compilation failed in require at /Users/wbirkett/Desktop/Rmath_test.pl + line 1. BEGIN failed--compilation aborted at /Users/wbirkett/Desktop/Rmath_tes +t.pl line 1.

      DynaLoader gets its initial list of library paths from Perl's %Config hash. On my Mac, the paths are /usr/lib and /usr/local/lib.

        Thank you for replying, glad it gave you something to try. I am going through your OP to pick out any information that I might have missed the first time.

        First thing that strikes me is that you moved the library. We know nothing about how you built the XS libararies, so did you literally move it or did you rebuild it in the new location? I suspect that there is an issue with the libaries you built thinking they are in the original location (just a stretch guess).

        Second thing is that DynaLoader is for Perl modules, so I was confused that this was being used to create and export your XS libraries. After educating myself a bit on DynaLoader, I see that it says:

        It must be stressed that the DynaLoader, by itself, is practically useless for accessing non-Perl libraries because it provides almost no Perl-to-C 'glue'. There is, for example, no mechanism for calling a C library function or supplying arguments. A C::DynaLib module is available from CPAN sites which performs that function for some common system types. And since the year 2000, there's also Inline::C, a module that allows you to write Perl subroutines in C. Also available from your local CPAN site..

        My suggestion is to take DynaLoader out of this altogether. Seems like you don't need it at all. XS based modules should work as normal if you just did a use My::XS::Module. On CPAN, XS based modules are not distributed as pre-compiled blobs, they are compiled and installed when you "install" the module. So get back to the basics:

        1. make sure you've rebuilt your XS modules for the new location
        2. get it to work in a simple script without DynaLoader with, use My::XS::Module
        3. make sure the lib directory in /path/to/lib/My::XS::Module is in @INC

        Then iterate on that until you get it to work, once that happens you can translate what you did in the simple program. I am getting the feeling that the root of the issue is that you may have literally mv'd the libraries to the new location rather than rebuilding them.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://11117912]
Approved by marto
Front-paged by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others wandering the Monastery: (4)
As of 2024-03-29 04:51 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found