Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"
 
PerlMonks  

Re^4: Use of do() to run lots of perl scripts

by LanX (Saint)
on Mar 03, 2021 at 10:19 UTC ( [id://11129071]=note: print w/replies, xml ) Need Help??


in reply to Re^3: Use of do() to run lots of perl scripts
in thread Use of do() to run lots of perl scripts

Yes I oversaw the fork part until choroba posted his other benchmark.

Do or even require alone are not fast. Reducing the start up of perl might have a time impact but won't change the RAM consumption.

My bet on the biggest time consumer is the filesystem not the compilation. Precompiling really payed off in the 90s, but now?

So using a RAM-disk could have the best cost benefit ratio.

But we are all speculating here, like others repeated over and over again, the OP should be more explicit

  • what his problems are (startup, ram, ...)
  • how frequently this happens
  • what he benchmarked.

I have my doubts that refactoring 500 scripts is an option and even then...

Precompiling them all into the master process would make them vulnerable to global effects in the BEGIN-phase.

Cheers Rolf
(addicted to the Perl Programming Language :)
Wikisyntax for the Monastery

  • Comment on Re^4: Use of do() to run lots of perl scripts

Replies are listed 'Best First'.
Re^5: Use of do() to run lots of perl scripts
by jcb (Parson) on Mar 04, 2021 at 02:54 UTC
    So using a RAM-disk could have the best cost benefit ratio.

    While our questioner has not told us what operating system this is using, modern systems tend to cache frequently used files in RAM. Linux kernels, in particular, will effectively move frequently-accessed files to a RAM disk, available RAM permitting. This is the "dcache" and Linux has had it for decades.

    I have my doubts that refactoring 500 scripts is an option

    The work could be done incrementally. The Pareto principle suggests that 20% of the scripts are probably used 80% of the time, with a long tail into the noise. Our questioner also mentioned that they are not all Perl scripts, so presumably it is already known that the largest contributors to server load are Perl scripts, otherwise the entire question is pointless.

    Precompiling them all into the master process would make them vulnerable to global effects in the BEGIN-phase.

    ByteLoader does not work like that. You use B::Bytecode to compile the script in advance, load ByteLoader itself in the master process, and use ByteLoader in each child to load and run the precompiled script. (Actually, the precompiled script can include use ByteLoader; so that fork or do "script.plc"; will cause ByteLoader to be correctly invoked.)

    Global effects in the BEGIN-phase could be an issue for refactoring the scripts into modules, but addressing those issues would be part of refactoring scripts into modules. :-)

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://11129071]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others exploiting the Monastery: (3)
As of 2024-04-16 14:18 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found