Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

I am running a Perl script that does alot of recursive find and replaces on my Solaris Ultra 250 server. I really want to know what you all use to monitor system resources when you run a Perl script on a web server. Here are the unix commands I am using to check:
ps -afe | grep truss -p 2345 --> 2345 is my process number vmstat 3 top
Please advise if all these are going to monitor my script and what specific numbers/columns I should be paying attention to? My script seems to be using about 90 to 95% of cpu.

Replies are listed 'Best First'.
Re: Checking Perl script on server
by Abigail-II (Bishop) on Jul 30, 2003 at 13:43 UTC
    There's nothing wrong with a program that's using almost 100% of the CPU. It means the CPU is used very effective, and the OS isn't context switching the program away all the time.

    Typically, as a system administrator, you want to monitor as much as possible. And what you are looking for are things that are unusual. For a particular program, you first have to get an indication of what you expect of the program, and what it's actually doing. For a program that does a lot of find and replace actions, you'd expect a lot of physical reads (unless most of the filesystem is already buffered), lots of logical reads, lots of logical writes, and depending on your filesystem and buffer size, physical writes.

    90 to 95% CPU time could indicate many things. It could be that most of the files acted on are already buffered (hence little physical I/O, and no I/O waits). It could also mean that your program is written very inefficiently.

    There's just too little information to say much about this situation.

    As for tools, there's top, or its Solaris nephew prstat. You already mentioned it. You also mentioned truss, vmstat and ps. Good choices, and check out the manual page of truss, it has some useful options; counts of system calls can be very informative. HP has glance for HP-UX, which is, IMO, a wonderful tool. I think it has been ported to Solaris, but I'm not sure. However, if it is, it won't be free.


      Thereare also mpstat (view stats on a per cpu basis) and iostat (to see io activity). You can see where the the system is doing more work by looking at usr/sys/idle/wt. usr is userland stuff, sys is any kernel related activity (semephore, context switching, memory allocation etc) wt is io wait (physical io blocking). and idle is the amount the cpu is sitting "idle". As Abigail pointed out a proccess using 95 or 100% of cpu is not always bad, but you may want to renice it if you have other proccesses on your server that usually use that cpu and are being hit by your perl script running. Also take a moment to verify the size of your proccess loaded into memory. Do you have it build huge arrays/hashtables? are you forcing the machine to swap, and causing slowdown on the other services on the machine?

        Thanks to both of you for the info. Yes I am building huge arrays in my finding and changing data recursively. Please advise if this way is using alot of memory?
        sub mySub { if( $_ =~ /\.html) { my $name = $File::Find::name; open ( F, $name ) || warn "Can\'t open File $name: $!\n"; while($line = <F>) { for $hit ($line =~ /matchdata/gi) { push @files, $name; } } close F; } } find( \&mySub, $dir ); foreach (@files) { open(LDATA, "$_") || warn "File does not open: $!\n"; @data = (<LDATA>); close(LDATA); open(LDATA, ">$_") || warn "File Write problem $_: $!\n"; foreach (@data) { s/OLD/NEW/gi; print LDATA $_; } close(LDATA); }
Re: Checking Perl script on server
by l2kashe (Deacon) on Jul 30, 2003 at 14:57 UTC

    The Devel:: family of modules can also help to determine whats going on in the code itself and where the program is spending the majority of its time. This can lead to refactoring for different goals (I.e from readability -> performance oriented) in the critical sections

    use perl;

Re: Checking Perl script on server
by JSchmitz (Canon) on Jul 30, 2003 at 18:43 UTC
    if you are running Solaris 8 you can also use the prstat command - you may also want to look at statit which is a binary you can get from SolarisInternals that really gives you some granular data. A 250 is a pretty old box I imagine maxing out the CPU on that baby is pretty easy.
Re: Checking Perl script on server
by MidLifeXis (Monsignor) on Jul 30, 2003 at 23:02 UTC

    You could also use sar. If you want to look at the disks, you would use sar -d 5 100, different resources take different parameters, see man sar for more info.