Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw
 
PerlMonks  

Troubleshoot a path problem when multithreading on cluster

by wayne_deng2000 (Initiate)
on Nov 08, 2014 at 21:48 UTC ( #1106582=perlquestion: print w/replies, xml ) Need Help??

wayne_deng2000 has asked for the wisdom of the Perl Monks concerning the following question:

I tried to run several tasks (written in perl) on a 8-core node (divided into 8 tasks), but encounter an error for finding a perl module. I tested single task on one thread, no such problem occurred. The weird thing is that the module path is already in ENV path when checking with perl -V. The perl file that gives error begins like this:
#!/usr/bin/perl use strict; use warnings; use XML::Parser; ...
Error message is like this:
Can't locate XML/Parser.pm in @INC (@INC contains: ... /usr/lib64/per +l5 /usr/share/perl5 .) at ./blast_parser.pl line 4. BEGIN failed--compilation aborted at ./blast_parser.pl line 4. [blastall] FATAL ERROR: File write error
The interesting thing is that XML::Parser.pm is in the path of /usr/lib64/perl5 that is listed in above @INC and in %ENV (I checked). I tried to include this path in perl file (use lib ), but got same error message. Couldn't figure out where is wrong and how to solve. The perl version: v5.10.1 (*) built for x86_64-linux-thread-multi. Linux version: 2.6.32-358.2.1.el6.x86_64 Thanks for any suggestions. Wayne

Replies are listed 'Best First'.
Re: Troubleshoot a path problem when multithreading on cluster
by Loops (Curate) on Nov 08, 2014 at 21:58 UTC

    Hello and welcome to the monastery.

    "File write error" looks very suspicious if all its doing is trying to find a module. What exactly did you do in your "single task on single thread" test that worked? How was the code different than the code that is failing?

      I guess "File writing error" is due to messy data not parsed properly by parser module. Two runs I tested use almost same perl code (except a bit difference for multi-task requiring a submission script (batch jobs in PBS manager). Forgot to mention that XML::Parser also requires another Expat library (.so) to work properly. Just suspect that PBS cluster manager ignored the path in ENV or something related to dynamic library? tried to set ld_Library_Path in script, so far the error still persists. Any input is appreciated.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://1106582]
Approved by boftx
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others contemplating the Monastery: (2)
As of 2022-07-01 11:52 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    My most frequent journeys are powered by:









    Results (98 votes). Check out past polls.

    Notices?