Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

use less 'CPU'

by powerman (Friar)
on Feb 13, 2003 at 17:54 UTC ( [id://235041]=perlquestion: print w/replies, xml ) Need Help??

powerman has asked for the wisdom of the Perl Monks concerning the following question:

I think I've found real solution to limit perl script by CPU load. This will work this way:
# this script will use 40% CPU instead of 100% use CPU 40; 1 while 1;
Relization based on my old ideas: Simultaneous alarms < 1 sec and resource control: CPU. These ideas was united and greatly improved, simplifyed and optimized.

I will test it, create package and upload to CPAN. But I don't know best suited name for this package. Really, I think it's Right Thing to realize it in standart pragma 'less'. This pragma now unimplemented, and as far as I understand it should be realized in perl itself. I suppose to realize 'less CPU' my way in this pragma and later replace my solution to perl internal solution with same interface. The interface can look like this:

use less CPU => 40, memory => '30MB';
The problem is what there is no AUTHOR section in less.pm, so I don't know who should be contacted about this question. May be one of monks know his email? Or you think I should not touch 'less' pragma and release it in usual module with name .....?

P.S. Maybe name 'limit' is more suited for this pragma than 'less'.

Replies are listed 'Best First'.
Re: use less 'CPU'
by rob_au (Abbot) on Feb 13, 2003 at 23:05 UTC
    Firstly, if you are looking at building upon this core module, I think it would be wise to have a read through perlhack or this article from Nathan Torkington about hacking the Perl core.

    I point this out is because, while I do not want to discourage you, it is unlikely that code similar to that posted at resource control: CPU would be incorporated 'as-is' into core pragmas. The reason for this is outlined in the referenced documents, but basically comes down to these guidelines about submitting code for the core:

    1. Keep it fast, simple, and useful.
    2. Keep features/concepts as orthogonal as possible.
    3. No arbitrary limits (platforms, data sizes, cultures).
    4. Keep it open and exciting to use/patch/advocate Perl everywhere.
    5. Either assimilate new technologies, or build bridges to them.

    Unfortunately, the code posted at resource control: CPU 'as-is' does rely heavily on platform dependent features, namely the usage of /proc/self/stat. Further quoting the above referenced documents - "The worst patches make use of a system-specific feature. It's highly unlikely that nonportable additions to the Perl language will be accepted.". This however may be something which you can work further upon to make your code my universal in its appeal.

    Good luck.

     

    perl -le 'print+unpack("N",pack("B32","00000000000000000000001000110000"))'

      Firstly, I want to thank you. It's rare case to get interesting long reply.

      I 100% agree with importance of all these guidelines. In my work I use all these guidelines except portable limits (because I've no needs and no ability to test my code on non-linux systems and also in many cases make code portable mean make it slow).

      I also agree about my code quality in resource control: CPU - this code was just an example and not really useful because it's too slow (for my tasks at least). But at start of this thread I say "improved, simplifyed and optimized". :-)

      About protable code... I don't see any other realization for limiting CPU load issue. My solution based on using things like fork() and signals. I suppose this solution are not really portable, but should work on all/most *NIX systems. And I've seen other people asking about this feature. Really, any who execute long-running process which is working instead of waiting for some event need this feature. And using plain sleep() to free some CPU is ugly solution because it's very hard to know "how much to sleep" to free some CPU and at some time do it work as quickly as possible. So, I think it better to have *NIX-only solution than have no solution.

      Here is my current code. I don't see any more possible improvements in main logic, but interface will change because I will package it as module or pragma. Can you please point me to anything not adequate to guidelines except porting one?

      # SYNOPSYS # # # start new alarm (fork new process) # $alarm_id = alarm_start(0.01, \$counter); # # # stop alarm (kill forked process) # alarm_stop($alarm_id); # # # sleep without been interrupted by signals # sleepy(0.5); # # # limit CPU usage to 10% # Limit_CPU(); # use Time::HiRes qw(time alarm setitimer ITIMER_PROF); { my %ALARM; sub alarm_start { my ($PARENT, $time, $counter) = ($$, @_); return unless $time > 0; return unless my ($n) = grep { not exists $ALARM{$_} } "NUM50" .. "NUM59"; $SIG{$n} = ref($counter) eq "SCALAR" ? sub { $$counter++ } : ref($counter) eq "CODE" ? sub { local $SIG{$n}="IGNORE"; &$counter } : return; return $n if $ALARM{$n} = fork(); $0 .= " ALARM $n"; $SIG{ALRM} = sub { kill $n, $PARENT or exit }; alarm($time, $time); CORE::sleep 1 while 1; } sub alarm_stop { return unless exists $ALARM{$_[0]}; local $SIG{CHLD} = $SIG{$_[0]} = "IGNORE"; kill 9, $ALARM{$_[0]} and waitpid delete $ALARM{$_[0]}, 0; }} sub sleepy { my $alarm = alarm_start($_[0], \(my $wakeup)) or die "sleepy: alarm failed"; # die or return? CORE::sleep 1 until $wakeup; alarm_stop($alarm); } use constant CPUMAX => 10; # percent, max CPU load by this process use constant INT => 0.1; # sec, how often check for CPU overload { my ($real, $prof, $wait); $SIG{PROF} = sub { $prof++ }; sub Limit_CPU { setitimer(ITIMER_PROF, INT, INT); alarm_start(INT, sub { $wait = ($prof*(100/CPUMAX) - ++$real) / (1/INT); return if INT > $wait; sleepy($wait); $real += int( $wait * (1/INT) ); }); }}
        What is wrong with using "nice" to lower the priority of the long-running process? That way the process gets to use all of the CPU when there is nothing else running but doesn't slow down normal processes. The operating system handles the scheduling efficiently. No polling, busy loops, or extra signals required.
Re: use less 'CPU'
by Mr. Muskrat (Canon) on Feb 13, 2003 at 18:07 UTC

      It's a core module; please contact the Perl 5 Porters mailing list instead. Jarkko is only taking bugfixes for the eventual 5.8.1 at this point.

Re: use less 'CPU'
by extremely (Priest) on Feb 13, 2003 at 19:16 UTC
    I'd recommend "limit", since at first my brain crammed the words together for "useless CPU" :)

    --
    $you = new YOU;
    honk() if $you->love(perl)

Re: use less 'CPU'
by Fletch (Bishop) on Feb 13, 2003 at 19:37 UTC

    Erm, perhaps you'd want to look into BSD::Resource, which will let the OS do more of the monitoring rather than polling yourself (at least that's what resource control: CPU looks to be doing).

    Not that a more pragmatic interface than setrlimit would be a bad thing.

      No, BSD::Resource and setrlimit don't allow to control max CPU load. They allow only control max cpu time. I.e. with my feature script may run forever, but never take more than 40% CPU. With setrlimit (or ulimit in bash) you can limit script to 5 CPU minutes and after script reach this limit kernel will kill it.

        The default SIGXCPU handler dies, but there's nothing stopping you from installing your own handler which does something different (sleeps, alters what's running, sets a different limit).

Re: use less 'CPU'
by pg (Canon) on Feb 14, 2003 at 20:52 UTC
    There is a big chance that, when "use less" is finally implemented, might be in Perl 6, it will be called "use optimize", which makes more sense.

    This example:
    "use less CPU => 40, memory => '30M';"

    does not fit very well with the original purpose of having "use less" or "use optimize".

    The root purpose of "use optimize" or "use less" is to allow Perl to make speed/memory trade off at either run time or compile time (details would be determined by the implementation). If you say "use optimize CPU", Perl would try to use more memory if that speeds up your script; or the other way around, if you specify "use optimize memory". (Be aware that, up till now, Perl usually trade memory for speed)

    To specify the number amount is "a little bit" less meaningful in the real world ;-). For example, what does "use less memory => 30M" really mean? if your program is deemed to use 35M, what should Perl do? Especially if you specify both less memory and less CPU, (with or without number amount attached), Perl would be put into a dilemma, and does not know what to do.
      Yeah, you right. Thanks for explanation!

      About memory limit... to be fair enough, I just copy this word from `perldoc less` and add parameter for fun - I don't mean something special here. But ability to limit memory sometimes very good idea. I don't mean perl optimizations here or OS limits like `setrlimit` (because as far as I know it's impossible to handle these limits - OS just kill the process).

      All you know what perl eat a lot of memory - it trade memory for speed. But user's script sometimes also trade memory for speed. All sort of caches do this. And I think it's a good to know inside script what it reach some memory limits. In this case user can set callback routine or signal handler which will empty caches, flush buffers, etc.

Re: use less 'CPU'
by EvdB (Deacon) on Jun 10, 2003 at 17:32 UTC
    Long after the fact but...

    Instead of specifying the percentage to use I would put in a fraction: so 40 would be 0.4, ie use CPU 0.4;

    Just my personal preference.

    --tidiness is the memory loss of environmental mnemonics

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://235041]
Approved by Mr. Muskrat
Front-paged by broquaint
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others avoiding work at the Monastery: (2)
As of 2024-04-26 07:43 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found