network socket send loop is very cpu intensive

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

I am new to Perl and have just written a script that reads text files from a directory and then sends the data on through a socket connection. It's a very simple process but is very cpu intensive so I suspect my loop and file handling is very flawed. Could someone take a little look at this main loop for me and comment. thanks a lot

the process is 'daemonized' first before entering main loop

opendir DH, "/tmp/store" or die "could not open directory: $!";
while (1) {



while($_ = readdir(DH) ) {

    next if $_ eq "." or $_ eq "..";
    my @data = split /_/;
    $mobile = $data[1];
    undef @data;
    chomp $mobile;
    s/^\s+//, s/\s+$//, s/\s\s+/ / for $mobile;
    open FH,"/tmp/sms/$_" or die "could not open: $_  $!";
    undef $/;
    my $data = <FH>;
    chomp $data;
    my @fields = split;
    my $appcode = $fields[0];
    my $message = $data;
    s/^\s+//, s/\s+$//, s/\s\s+/ / for $data;
    close FH;
    sendtolive($mobile,$message); # Sends data through socket connecti
+on
    unlink("/tmp/sms/$_"); # remove file once read
    }

}
closedir DH;
[download]

20031118 Edit by Corion: Added CODE tags, changed title from "cpu intensive"

Comment on network socket send loop is very cpu intensive Download Code

Replies are listed 'Best First'.
Re: network socket send loop is very cpu intensive by AcidHawk (Vicar) on Nov 18, 2003 at 12:48 UTC
Two things I've noticed. You will never get to `closedir DH;` as you are in a while(1){} loop before. You continuously loop in /tmp/store for files never sleeping or giving anything else a chance to run. So even if there are no files you read and read and read... etc you get the picture.. Try adding a `sleep($timeout)` under the `unlink("/tmp/sms/$_"); # remove file once read` where $timeout can be 1 sec If you post the filename and data you are looking for, (to split) we'll have a better idea of what you are doing with the splits etc.. Update1: Something else... you assign $message the value of $data. Then you clean up the whitespace from `$data` but then send `$message` to `sendtolive()`..? Update2: You open the /tmp/store dir and try to read a file int the /tmp/sms dir..? ----- Of all the things I've lost in my life, its my mind I miss the most.	[reply] [d/l] [select]
Re: Re: network socket send loop is very cpu intensive by Anonymous Monk on Nov 18, 2003 at 13:42 UTC
thanks, all useful. I originally did have the close handler inside the main loop but changed it to see if that helped things. I have also experimented with sleeping for a second before reading again and does sort it out. It has helped me to have you talk me through it though. With regards Update 2, just a typo from a version change. Update 1, I'll check into that, thanks.	[reply]
Re: Re: Re: network socket send loop is very cpu intensive by AcidHawk (Vicar) on Nov 18, 2003 at 14:21 UTC
No problem, I like to have one exit point also, so I put this little bit together to illustrate the point... This is UNTESTED though.. #! /usr/bin/perl use strict; use warnings; my $rc = 0; while (1) { if (opendir DH, "/tmp/store") { if (readdir(DH) ) { next if $_ eq "." or $_ eq ".."; my @data = split /_/; my $mobile = $data[1]; undef @data; chomp $mobile; s/^\s+//, s/\s+$//, s/\s\s+/ / for $mobile; if (open FH,"/tmp/store/$_") { undef $/; my $data = <FH>; chomp $data; my @fields = split; my $appcode = $fields[0]; my $message = $data; #Assuming you want to clean up the data that is being +sent to sendtolive() s/^\s+//, s/\s+$//, s/\s\s+/ / for $message; close FH; # Sends data through socket connection assuming the se +ndtolive returns a success or fail # you can test for it and only delete if it is success +ful. Do you want to delete the file if it cant send the sms..? if (sendtolive($mobile,$message)) { # remove file once read unlink("/tmp/store/$_"); } else { print "Count NOT sent $message: sendtolive failed" +; } } else { print "could not open: $_ $!"; $rc = 3; last; } } else { print "Cannot read dir /tmp/store: $!\n"; $rc = 2; last; } #You have finished with this file loop onto the next. closedir (DH); sleep(1); } else { print "Faile to open dir /tmp/store: $!\n"; $rc = 1; last; } } exit ($rc); [download] ----- Of all the things I've lost in my life, its my mind I miss the most.	[reply] [d/l]
Re: Re: Re: Re: network socket send loop is very cpu intensive by Anonymous Monk on Nov 18, 2003 at 16:16 UTC
Re: Re: Re: Re: Re: network socket send loop is very cpu intensive by AcidHawk (Vicar) on Nov 18, 2003 at 17:37 UTC
Re: Re: Re: Re: Re: network socket send loop is very cpu intensive by l2kashe (Deacon) on Nov 18, 2003 at 21:49 UTC
Re: network socket send loop is very cpu intensive by ptkdb (Monk) on Nov 18, 2003 at 12:51 UTC
Looking closely, you've got your 'opendir/closedir' outside of your while(1) loop. So this loop is going to go once on the first opendir, and then 'spin' out of control as it breaks out of the inner while afterwards, but keeps trying the exhausted DH again and again and again and again. You did step through this code with a debugger before you sent it here didn't you?	[reply]
Re: network socket send loop is very cpu intensive by Art_XIV (Hermit) on Nov 18, 2003 at 13:54 UTC
I wonder what's going on in the sendtolive sub. Are you sure that it's being all that it can be? Is this from a CPAN module or was it written by yourself? Are you creating a new socket/object/whatever each time the sendtolive sub is called? (Avoid that, if you are.) Aside from the directory handle issues, that sub call looks like the biggest suspect. I can't really tell though, w/o code. Hanlon's Razor - "Never attribute to malice that which can be adequately explained by stupidity"	[reply]


go ahead... be a heretic
	PerlMonks