wbirkett has asked for the wisdom of the Perl Monks concerning the following question:
Hello, I built an XS module that interfaces the Rmath statistics library. This was straightforward, and works fine, when installed in the normal location (/Library/Perl/5.18 on my Mac, the Rmath library is in /usr/local/lib). Now, I want to relocate the module and library to another location, in this case a TextMate bundle at ~/Library/Application Support/TextMate/Pristine Copy/Bundles. With the library in /usr/local/lib, the module works fine. But when I move the library to another location, the loading fails with this message:
Can't load '/Users/wbirkett/Library/Application Support/TextMate/Prist
+ine Copy/Bundles/PressCal.tmbundle/Support/lib/perl5/darwin-thread-mu
+lti-2level/auto/ICC/Support/Rmath/Rmath.bundle' for module ICC::Suppo
+rt::Rmath: dlopen(/Users/wbirkett/Library/Application Support/TextMat
+e/Pristine Copy/Bundles/PressCal.tmbundle/Support/lib/perl5/darwin-th
+read-multi-2level/auto/ICC/Support/Rmath/Rmath.bundle, 1): Library no
+t loaded: libRmath.dylib
Referenced from: /Users/wbirkett/Library/Application Support/TextMat
+e/Pristine Copy/Bundles/PressCal.tmbundle/Support/lib/perl5/darwin-th
+read-multi-2level/auto/ICC/Support/Rmath/Rmath.bundle
Reason: image not found at /System/Library/Perl/5.18/XSLoader.pm lin
+e 71.
at /Users/wbirkett/Library/Application Support/TextMate/Pristine Copy
+/Bundles/PressCal.tmbundle/Support/lib/perl5/darwin-thread-multi-2lev
+el/ICC/Support/Rmath.pm line 183.
Compilation failed in require at /Users/wbirkett/Desktop/Rmath_test.pl
+ line 1.
BEGIN failed--compilation aborted at /Users/wbirkett/Desktop/Rmath_tes
+t.pl line 1.
From the documentation for XSLoader.pm, "To reduce the overhead as much as possible, only one possible location is checked to find the extension DLL (this location is where make install would put the DLL). If not found, the search for the DLL is transparently delegated to DynaLoader , which looks for the DLL along the @INC list." The library is definitely located in an @INC location, and the delegation to DynaLoader fails. I've tried many ideas to get this working, including setting the @DynaLoader::dl_library_path variable, and using DynaLoader.pm rather that XSLoader.pm, with no success.
I am completely stumped, and appeal to your monk wisdom for a solution.
Re: XSLoader/DynaLoader Can't Locate Library
by bliako (Monsignor) on Jun 10, 2020 at 19:47 UTC
|
Perhaps there are some extra files libRmath.dylib depends on which you have not moved? Perhaps this can tell you more, if it works: export LD_DEBUG=all . Or use the nuclear deterrent: strace -f -e trace=open,close,read,write,connect,accept perl xx.pl | grep libRmath (and without the grep perhaps).
Have you tried to use only DynaLoader, re: https://metacpan.org/pod/XSLoader#Migration-from-DynaLoader
What does this show?
require DynaLoader;
print "@DynaLoader::dl_library_path\n";
bw, bliako | [reply] [Watch: Dir/Any] [d/l] [select] |
|
Thank you, bliako, for your suggestions.
Unfortunately, strace is not available on my Mac. I wasn't able to coax any additional debug information. I also tried setting the $dl_debug flag.
I did modify the Rmath.pm file to use the DynaLoader module instead of XSLoader, sort of a reverse migration. The result was the same, even when pushing the exact .dylib folder path onto @DynaLoader::dl_library_path. When I add code to the .bs file, it is executed, so I used that to modify @DynaLoader::dl_library_path.
The default values for @DynaLoader::dl_library_path are '/usr/lib' and '/usr/local/lib'. The library loads correctly from either of those locations. I was able to load the library from other locations using the dl_load_file() function.
| [reply] [Watch: Dir/Any] |
|
When I add code to the .bs file, it is executed, so I used that to modify @DynaLoader::dl_library_path.
What about adding that code in the actual .pm file you created (Rmath.pm?) like:
package Rmath;
...
our @ISA = qw( ... DynaLoader ... );
require DynaLoader;
push @DynaLoader::dl_library_path, 'my-nonstandard-lib-path';
print "XXX=$_\n" for @DynaLoader::dl_library_path;
bootstrap <Yourmodule> $VERSION;
$dl_debug : I meant doing export LD_DEBUG=all in your terminal/shell/prompt before testing your program (on that same terminal).
There are two very remote possibilities: 1) the dylib extension is not tried at all in enumerating possible library names (unlikely because it finds it in standard lib location), 2) the spaces in the library path (in OSX hmm that's not likely).
ps. the reason I suggested this reverse-migration is to investigate DynaLoader's behaviour without the front-man. | [reply] [Watch: Dir/Any] [d/l] [select] |
|
|
|
|
>strace is not available on my Mac
man dtruss
| [reply] [Watch: Dir/Any] [d/l] |
|
«...strace is not available on my Mac.»
dtruss?
«The Crux of the Biscuit is the Apostrophe»
perl -MCrypt::CBC -E 'say Crypt::CBC->new(-key=>'kgb',-cipher=>"Blowfish")->decrypt_hex($ENV{KARL});'Help
| [reply] [Watch: Dir/Any] [d/l] [select] |
Re: XSLoader/DynaLoader Can't Locate Library
by perlfan (Vicar) on Jun 11, 2020 at 01:42 UTC
|
Try making sure your alternative paths are added to your environment via LD_LIBRARY_PATH (with /path/to/static-or-shared-libraries/lib) and to LD_INCLUDE_PATH contains your /path/to/headerfiles/to/include. You talk of DLLs and Mac don't make sense to me, but what I do know is that if you're compiling with gcc, then those two environmental variables are used by the linker and if you're linking a shared library rather than compiling it statically, then you're gonna need LD_LIBRARY_PATH set correctly at run time. As far as I know, this is all controlled outside of whatever Perl harness you're using to bootstrap the compilation and facilitating the running of your XS module.
| [reply] [Watch: Dir/Any] [d/l] [select] |
|
Thanks for the suggestion, perlfan.
I exported the path to the folder containing the .dylib, and here is the result
WBBirkett-M:~ wbirkett$ export LD_LIBRARY_PATH=/Users/wbirkett/Library
+/Application\ Support/TextMate/Pristine\ Copy/Bundles/PressCal.tmbund
+le/Support/lib/perl5
WBBirkett-M:~ wbirkett$ env
TERM_PROGRAM=Apple_Terminal
SHELL=/bin/bash
TERM=xterm-256color
TMPDIR=/var/folders/kf/hd1w1x888xj_x1018f8m3f2h0000gn/T/
Apple_PubSub_Socket_Render=/private/tmp/com.apple.launchd.le70PQDJKe/R
+ender
TERM_PROGRAM_VERSION=404.1
TERM_SESSION_ID=CC263E8D-8730-4F07-9752-F779AA2A8ADD
USER=wbirkett
LD_LIBRARY_PATH=/Users/wbirkett/Library/Application Support/TextMate/P
+ristine Copy/Bundles/PressCal.tmbundle/Support/lib/perl5
SSH_AUTH_SOCK=/private/tmp/com.apple.launchd.L73CgT5E19/Listeners
PATH=/Users/wbirkett/.rbenv/shims:/usr/local/bin:/usr/local/sbin:/usr/
+local/mysql/bin/:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/opt/X1
+1/bin:/usr/local/git/bin:/Library/TeX/texbin
PWD=/Users/wbirkett
LANG=en_US.UTF-8
XPC_FLAGS=0x0
RBENV_SHELL=bash
XPC_SERVICE_NAME=0
SHLVL=1
HOME=/Users/wbirkett
LOGNAME=wbirkett
DISPLAY=/private/tmp/com.apple.launchd.pfkv04jVPR/org.macosforge.xquar
+tz:0
_=/usr/bin/env
WBBirkett-M:~ wbirkett$ perl -I/Users/wbirkett/Library/Application\ Su
+pport/TextMate/Pristine\ Copy/Bundles/PressCal.tmbundle/Support/lib/p
+erl5 /Users/wbirkett/Desktop/Rmath_test.pl
Can't load '/Users/wbirkett/Library/Application Support/TextMate/Prist
+ine Copy/Bundles/PressCal.tmbundle/Support/lib/perl5/darwin-thread-mu
+lti-2level/auto/ICC/Support/Rmath/Rmath.bundle' for module ICC::Suppo
+rt::Rmath: dlopen(/Users/wbirkett/Library/Application Support/TextMat
+e/Pristine Copy/Bundles/PressCal.tmbundle/Support/lib/perl5/darwin-th
+read-multi-2level/auto/ICC/Support/Rmath/Rmath.bundle, 1): Library no
+t loaded: libRmath.dylib
Referenced from: /Users/wbirkett/Library/Application Support/TextMat
+e/Pristine Copy/Bundles/PressCal.tmbundle/Support/lib/perl5/darwin-th
+read-multi-2level/auto/ICC/Support/Rmath/Rmath.bundle
Reason: image not found at /System/Library/Perl/5.18/XSLoader.pm lin
+e 71.
at /Users/wbirkett/Library/Application Support/TextMate/Pristine Copy
+/Bundles/PressCal.tmbundle/Support/lib/perl5/darwin-thread-multi-2lev
+el/ICC/Support/Rmath.pm line 183.
Compilation failed in require at /Users/wbirkett/Desktop/Rmath_test.pl
+ line 1.
BEGIN failed--compilation aborted at /Users/wbirkett/Desktop/Rmath_tes
+t.pl line 1.
DynaLoader gets its initial list of library paths from Perl's %Config hash. On my Mac, the paths are /usr/lib and /usr/local/lib. | [reply] [Watch: Dir/Any] [d/l] |
|
Thank you for replying, glad it gave you something to try. I am going through your OP to pick out any information that I might have missed the first time.
First thing that strikes me is that you moved the library. We know nothing about how you built the XS libararies, so did you literally move it or did you rebuild it in the new location? I suspect that there is an issue with the libaries you built thinking they are in the original location (just a stretch guess).
Second thing is that DynaLoader is for Perl modules, so I was confused that this was being used to create and export your XS libraries. After educating myself a bit on DynaLoader, I see that it says:
It must be stressed that the DynaLoader, by itself, is practically useless for accessing non-Perl libraries because it provides almost no Perl-to-C 'glue'. There is, for example, no mechanism for calling a C library function or supplying arguments. A C::DynaLib module is available from CPAN sites which performs that function for some common system types. And since the year 2000, there's also Inline::C, a module that allows you to write Perl subroutines in C. Also available from your local CPAN site..
My suggestion is to take DynaLoader out of this altogether. Seems like you don't need it at all. XS based modules should work as normal if you just did a use My::XS::Module. On CPAN, XS based modules are not distributed as pre-compiled blobs, they are compiled and installed when you "install" the module. So get back to the basics:
- make sure you've rebuilt your XS modules for the new location
- get it to work in a simple script without DynaLoader with, use My::XS::Module
- make sure the lib directory in /path/to/lib/My::XS::Module is in @INC
Then iterate on that until you get it to work, once that happens you can translate what you did in the simple program. I am getting the feeling that the root of the issue is that you may have literally mv'd the libraries to the new location rather than rebuilding them.
| [reply] [Watch: Dir/Any] [d/l] [select] |
|
|