Beefy Boxes and Bandwidth Generously Provided by pair Networks
laziness, impatience, and hubris
 
PerlMonks  

reading output from another script

by Octavian (Monk)
on Jul 23, 2003 at 14:27 UTC ( #277164=perlquestion: print w/replies, xml ) Need Help??

Octavian has asked for the wisdom of the Perl Monks concerning the following question:

Help! I am in need of some divine wisdom!

I have been asked if there is a way to monitor the output of an exsisting script and send an email notification if a certain event occurs. The script that runs dumps its output to standard out, and runs forever or until killed.(Imagine doing a tail -f on syslog.log or something, they want to be notified if a certain condition occurs) I tried throwing it into an array, but that just sits there until it is finished getting STDIN, which in the case of this script, never happens so that doesnt work...

So my question is this: Is there a way to my perl script monitor in real time the output of another script while it is dumping output? they are looking for a way to do a

./script |./monitor.pl or just
have ./monitor.pl call script and watch it...either way works...but I am starting to doubt that it can be done..

Please advise....

update (broquaint): added <code> tags and some further formatting

Replies are listed 'Best First'.
Re: reading output from another script
by Chady (Priest) on Jul 23, 2003 at 14:31 UTC

    Why the array? can't you just pipe it and read one line at a time?

    #script.pl $|++; my $i = 0; while (1) { $i++; print $i % 50000 == 0 ? "Event" : "stuff\n"; } #monitor.pl while (<STDIN>) { if (/Event/) { notify(); } } sub notify { print "Got notification.\n"; } $ ./script.pl | ./monitor.pl
    Update: added sub notify { } in code.
    He who asks will be a fool for five minutes, but he who doesn't ask will remain a fool for life.

    Chady | http://chady.net/
Re: reading output from another script
by NetWallah (Canon) on Jul 23, 2003 at 17:17 UTC
    You did not specify which OS - it looks like Unix, since you mention "tail".

    The simplest way , if you still want to see the STDOUT from the original program, is to use a "tee" program that shows STDOUT on the terminal as well as pipes it to another program (Your monitor Script). This way, you script gets to read its STDIN, and take action when it sees what it wants. The user continues to see the terminal output.

    I have also seen win32 versions of "tee", but it has been years since I have used it.

      NetWallah,
      IO::Tee should be mostly OS independent. I however would have the tee printing to STDOUT and to a log. The reason I would choose a log versus a pipe to the other script is so the secondary script can be stopped and started at will without affecting the primary script. The input for the monitoring script would then use File::Tail to monitor the progress of the log. I guess the only thing then would to remember to set $| to auto-flush.

      Cheers - L~R

        Yeah, I would prefer if the data was written to a log file as well, it would make my life simplier and I wouldnt have needed assistance in figuring out a script to monitor it. However, this is not my script to maintain, I am not even sure what exactly it does, I was just approached with the problem they have, and asked if I could provide a solution. And sorry, yes I am running on an HPUX 11 system, however, there script is on a redhat linux box, but anything I write up should work on their machine. I know how tee works, but the problem is not that they want to see the output as it is being monitored, the problem is that they want to run the script, go home for the night, and if any problems occur, get a page about it. However, when I tried to write a script to watch for what they wanted, it wouldnt work until the input ended. So the end outcome would be that they wouldnt get the page until they came in in the morning and killed the script.
Re: reading output from another script
by Abigail-II (Bishop) on Jul 23, 2003 at 14:35 UTC
    open my $pipe => "tail -f /syslog.log" or die "Fork failed: $!"; while (<$pipe>) { ... whatever ... }

    But beaware of pipe buffers.

    Abigail

      This assumes the stream would be line oriented. Of course that should be true for most circumstances (he even mentions tail -f). But if it is not things get a bit more tricky and he would need to read character by character by some low level system call.
        Well, yeah, and things could even be more tricky if the read in characters are in some strange encoding, and an obscure language. ;-)

        I read the article from the OP as if (s)he had problems with the fact that the source (s)he's reading from continues to produce data - not that (s)he needed help with non line oriented input. But even if you want to read byte by byte, just do a S/ = \1 before reading.

        Abigail

Re: reading output from another script
by ides (Deacon) on Jul 23, 2003 at 14:35 UTC

    I'd suggest looking into IPC::Open2 and IPC::Open3 they allow you to run your script from within the monitor and use the scripts STDIN, STDOUT, and STDERR as file handles.

    -----------------------------------
    Frank Wiles <frank@wiles.org>
    http://frank.wiles.org

Re: reading output from another script
by jmanning2k (Pilgrim) on Jul 23, 2003 at 20:10 UTC
    The other solutions listed here are better, but if you really want to do it in your perl script:

    First way: ./script | ./monitor.pl
    # monitor.pl while(<>) { if(/wantthis/) { dosomething($_); } if(/warning/) {dosomethingelse($_); } print; ## pass on the output }

    Option 2 - run the program from perl
    # monitor.pl open(INPUT, "./script |") or die $!; while(<INPUT>) { # same as above if(/regex/) { dosomething($_); } print; } close INPUT;

    Yet another way: If the script itself is perl, have it monitor it's own output.
    # adapted from Perl Cookbook sub filteroutput { return if my $pid = open(STDOUT, "|-"); die "cannot fork: $!" unless defined $pid; while(<STDIN>) { if(/regex/) { dosomething($_); } print; } exit; #exit forked monitoring process }
    Now add filteroutput(); above the rest of the script's normal code. It's self-monitoring.
      I think I must not have done a good enough job explaining what my problem is, as most suggestions have been dealing with monitoring log files or how to send emails or how to still view the output while piping to perl. The problem I am having is a script is NOT writing to a log file, it is dumping its output to STDOUT. They want me to write a perl script that can either call their script or they will pipe their output to my perl script. There script never stops until it is killed, and the data pumps out pretty fast, about three lines per second. They want my script to watch the output as it comes out and send an email if it see's a certain event occur. I know how to send the email and all that, Its just when I created my array,
      @array= <STDIN>;
      the program would wait until it stopped recieving STDIN before it would process the array, so it wouldnt check the input as it was coming in. The above post seemed to help a bit. (doing a while loop on STDIN) cause it appears that it proccesses a chunk at a time, it looks to me like 50 lines or so at a time, but it still isn't line by line... I will continue to play with the above while loops to see if I can get something acceptable to them...
      Like I said, it dumps about 3 lines per second, so if it does infact do chunks of 50 or so at a time as it is getting it, that may be fine, we can just blame the lag on email ;)
        In that case, the code I included above is exactly what you want.

        As for 50 lines at a time - it's not really happening that way. The above loops will do things line by line as they are output. I bet it's just buffered output that is the problem.

        You may have to unbuffer the original script (if it's perl, use the same solution - if it's not perl, I'm not sure what you can do). However, if it normally produces even output, then just fix the monitor.pl script.

        You can unbuffer the output from your monitor.pl perl script with:
        $| = 1;
        or as an alternative
        use FileHandle; STDOUT->autoflush;
        This should produce smooth, line by line output, as it is generated by the original script.

        As a side note, I bet your email gets sent immediately, it is just the text output to the screen that is buffered.

        ~Jon
      Thanks for your assistance folks, and this thread was accidently duplicated...they are playing with new security "features" around here and it took all day for me to get the question posted, the site kept timing out, so in amongst the time outs, one of em must have made it in and I didnt know it...I dont know who can clean it up, but either this one or the other thread can be removed.
      just putting STDIN in a while loop did the trick, I have by far a lower level of knowledge of perl than some of the brains around here, so I was totally focused on the array and then doing a foreach loop rather than a while loop.
      Thanks again!
Re: reading output from another script
by pzbagel (Chaplain) on Jul 23, 2003 at 17:36 UTC

    It's already been done.

    No sense reinventing the wheel.

Re: reading output from another script
by talexb (Canon) on Jul 23, 2003 at 18:14 UTC

    I can highly recommend Log::Log4Perl. You can configure it to send you E-Mail (among many other methods) on specific log messages.

    --t. alex
    Life is short: get busy!
Re: reading output from another script
by TomDLux (Vicar) on Sep 01, 2003 at 03:41 UTC

    How about

    open OTHER, "other |";

    You could use fork, so one thread keeps an eye out for data from OTHER, while the other does other stuff, maybe interactive stuff. Once data arrives from OTHER, it is stored in a shared variable, so the main loop can make use of it.

    --
    TTTATCGGTCGTTATATAGATGTTTGCA

Re: reading output from another script
by Linuxboy (Novice) on Jul 24, 2003 at 16:47 UTC

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://277164]
Approved by dga
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chanting in the Monastery: (3)
As of 2021-12-05 01:24 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    R or B?



    Results (30 votes). Check out past polls.

    Notices?