Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw
 
PerlMonks  

Re: Perl startup and excessive "stat" use on module load

by Fletch (Chancellor)
on Dec 31, 2020 at 05:03 UTC ( #11126028=note: print w/replies, xml ) Need Help??


in reply to Perl startup and excessive "stat" use on module load

Check out perlrun for the details on PERL5LIB in the environment and the search path perl uses; also of interest would be the lib module. That /etc/perl directory along with several other variant directories doesn't look normal so that may be something your OS' perl set (if you're using the stock perl; which has its own issues) or you may have something setting PERL5LIB. WRT to the number of stats, that's just the way perl searches; it walks each directory on the search path looking for the requested module. The more directories you put on the search path, the more it's going to search.

So given that if you're seeing unacceptable delays one option would be to trim the search path down. Presuming you've got extra things on PERL5LIB then clearing that out in the environment would be the easiest option. If the extra directories have been compiled into your copy of perl itself then it's not so easy. You'd need to recompile your own perl without those extra directories added to the compiled in search path (but there's also benefits to having your own application perl separate from the OS' copy).

However even if you do build your own perl you'll probably wind up with about 4-5 directories to search. If even that much overhead is an issue then there may not an easy answer short of attempting something like you were thinking with a symlink farm. In that case the better option would be to look at redesigning things to not repeatedly spawn fresh perl processen which are going to need to search; if you can instead work with a persistent process (or maybe one that runs multiple passes; handwaving here without details but maybe if you could give a bit more about your environment and the "real time perl scripting" involved would probably help get you more applicable suggestions). Amortizing the stats across a longer process lifetime would help lessen the hit for any one item you process.

The cake is a lie.
The cake is a lie.
The cake is a lie.

Replies are listed 'Best First'.
Re^2: Perl startup and excessive "stat" use on module load
by haukex (Bishop) on Dec 31, 2020 at 11:31 UTC
    If the extra directories have been compiled into your copy of perl itself then it's not so easy.

    I think it's pretty easy - the perlbrew installs on my machine only have four entries in @INC, and building a copy of Perl without perlbrew is pretty easy too.

    $ wget https://www.cpan.org/src/5.0/perl-5.32.0.tar.xz $ tar -xaf perl-5.32.0.tar.xz $ cd perl-5.32.0 $ sh Configure -de -Dprefix=/opt/perl5.32 $ make -j4 $ TEST_JOBS=4 make test_harness $ make install $ /opt/perl5.32/bin/perl -le 'print for @INC' /opt/perl5.32/lib/site_perl/5.32.0/x86_64-linux /opt/perl5.32/lib/site_perl/5.32.0 /opt/perl5.32/lib/5.32.0/x86_64-linux /opt/perl5.32/lib/5.32.0

      It's not rocket science, Smithers; it's brain surgery! However when compared against (possibly) unsetting an environment variable though that's a little bit more effort.

      Slight snark aside though: yes that's certainly something OP should consider doing anyhoo if they're using the stock perl on their platform, @INC length issues not withstanding. If that also solves their stat overhead problem all the better.

      The cake is a lie.
      The cake is a lie.
      The cake is a lie.

        However when compared against (possibly) unsetting an environment variable though that's a little bit more effort

        You're right of course, if the extra @INC dirs are coming from PERL5LIB then a new build of Perl won't help with that, so best to try that first :-)

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://11126028]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others taking refuge in the Monastery: (6)
As of 2021-04-14 15:57 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?