Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl Monk, Perl Meditation

Re: Checking Perl script on server

by Abigail-II (Bishop)
on Jul 30, 2003 at 13:43 UTC ( #279175=note: print w/replies, xml ) Need Help??

in reply to Checking Perl script on server

There's nothing wrong with a program that's using almost 100% of the CPU. It means the CPU is used very effective, and the OS isn't context switching the program away all the time.

Typically, as a system administrator, you want to monitor as much as possible. And what you are looking for are things that are unusual. For a particular program, you first have to get an indication of what you expect of the program, and what it's actually doing. For a program that does a lot of find and replace actions, you'd expect a lot of physical reads (unless most of the filesystem is already buffered), lots of logical reads, lots of logical writes, and depending on your filesystem and buffer size, physical writes.

90 to 95% CPU time could indicate many things. It could be that most of the files acted on are already buffered (hence little physical I/O, and no I/O waits). It could also mean that your program is written very inefficiently.

There's just too little information to say much about this situation.

As for tools, there's top, or its Solaris nephew prstat. You already mentioned it. You also mentioned truss, vmstat and ps. Good choices, and check out the manual page of truss, it has some useful options; counts of system calls can be very informative. HP has glance for HP-UX, which is, IMO, a wonderful tool. I think it has been ported to Solaris, but I'm not sure. However, if it is, it won't be free.


Replies are listed 'Best First'.
Re: Re: Checking Perl script on server
by waswas-fng (Curate) on Jul 30, 2003 at 13:55 UTC
    Thereare also mpstat (view stats on a per cpu basis) and iostat (to see io activity). You can see where the the system is doing more work by looking at usr/sys/idle/wt. usr is userland stuff, sys is any kernel related activity (semephore, context switching, memory allocation etc) wt is io wait (physical io blocking). and idle is the amount the cpu is sitting "idle". As Abigail pointed out a proccess using 95 or 100% of cpu is not always bad, but you may want to renice it if you have other proccesses on your server that usually use that cpu and are being hit by your perl script running. Also take a moment to verify the size of your proccess loaded into memory. Do you have it build huge arrays/hashtables? are you forcing the machine to swap, and causing slowdown on the other services on the machine?

      Thanks to both of you for the info. Yes I am building huge arrays in my finding and changing data recursively. Please advise if this way is using alot of memory?
      sub mySub { if( $_ =~ /\.html) { my $name = $File::Find::name; open ( F, $name ) || warn "Can\'t open File $name: $!\n"; while($line = <F>) { for $hit ($line =~ /matchdata/gi) { push @files, $name; } } close F; } } find( \&mySub, $dir ); foreach (@files) { open(LDATA, "$_") || warn "File does not open: $!\n"; @data = (<LDATA>); close(LDATA); open(LDATA, ">$_") || warn "File Write problem $_: $!\n"; foreach (@data) { s/OLD/NEW/gi; print LDATA $_; } close(LDATA); }
        I can't speak directly to efficiency as it's hardly my specialty, but I would suggest a change to this chunk:
        while($line = <F>) { for $hit ($line =~ /matchdata/gi) { push @files, $name; } } close F;
        Logically it seems like you want to check to see if the file you're currently examining has data that needs to be updated (you're saving the filename in @files for later processing) -- but you're pushing the filename onto your list once for each match in the file! Once you find a match in a particular file you should (based on my understanding of your goal) (1) push the filename onto your to-be-processed stack, and then (2) move on to the next file. So:
        while($line = <F>) { if ($line =~ /matchdata/i) { push @files, $name; last; } } close F;
        Note that I dropped the global flag for the match operator, too -- you just want to know if there's something there, anywhere, to be fixed later, and then move on.

        Naturally, if I misunderstand your goals this could be way off base :)

        Couple of things:
        1. if /matchdata/ has any capturing () than $name is pushed into @files for each ().
        2. When you do @data = (<LDATA>) you read the entire file into memory. A better way to do this would be:
        foreach (@files) { open(LDATA, "$_") || warn "File does not open: $!\n"; open(TMP, ">$_.tmp") || warn "File Write problem $_.tmp: $!\n"; while (<LDATA>) { s/OLD/NEW/gi; print TMP $_; } close(LDATA); close(TMP); rename("$_.tmp", "$_") or warn "Could not rename '$_.tmp' to '$_': + $!\n"; }
        Another side advantage of this is that if, for some reason, your script dies half way through then you still have our old file rather than a half written new file. hth.

        $will->code for @food or $$;

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://279175]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others avoiding work at the Monastery: (5)
As of 2022-12-04 18:01 GMT
Find Nodes?
    Voting Booth?

    No recent polls found