Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister
 
PerlMonks  

Best practice ipc and Log::Log4perl

by codeacrobat (Chaplain)
on Jul 03, 2008 at 13:38 UTC ( #695360=perlquestion: print w/replies, xml ) Need Help??

codeacrobat has asked for the wisdom of the Perl Monks concerning the following question:

Hello everyone

I want to process a long running external application, which writes to STDOUT and STDERR. Unfortunately application output is unstructured. So I would like to capture STDOUT and STDERR and somehow delegate it to Log::Log4perls $log->info or $log->error methods.

I tried several Modules IPC::Open3, IPC::Run3, Tie::STDOUT, Tie::STDERR but none easily does what I want ( or maybe I am just blind ).

Has anyone of my fellow monks already written a script as described and could show me an example?


print+qq(\L@{[ref\&@]}@{['@'x7^'!#2/"!4']});

Replies are listed 'Best First'.
Re: Best practice ipc and Log::Log4perl
by starbolin (Hermit) on Jul 04, 2008 at 04:32 UTC

    Could you be more specific as to what you want to do that IPC::Open3, IPC::Run3, Tie::STDOUT, Tie::STDERR don't do? I don't understand.


    s//----->\t/;$~="JAPH";s//\r<$~~/;{s|~$~-|-~$~|||s |-$~~|$~~-|||s,<$~~,<~$~,,s,~$~>,$~~>,, $|=1,select$,,$,,$,,1e-1;print;redo}
      What I want:
    • read unbuffered from a child process ( long running process ).
    • capture childprocess stderr & stdout separately
    • write the childprocess stdout and stderr stream to a logfile and tag lines with a <timestamp> and a tag <INFO> or <ERROR> depending on the source stream.
    • IPC::Run3 almost does what I need.
      #!/usr/bin/perl # timestamp.pl $|++; while(time - $^T<10){ select undef,undef,undef, 0.1; if ( rand()<0.5) { print STDERR time(), " warn\n"; } else { print time(), " info\n"; } } __END__
      use IPC::Run3 qw(run3); my @cmd = ('perl', 'timestamp.pl'); run3 \@cmd, undef, \&out, \&err; sub out { print time(), " out: $_\n"; } sub err { print time(), " err: $_\n"; } __END__ # The problem is that run3 buffers the output. 1215162770 out: 1215162760 info 1215162770 out: 1215162760 info 1215162770 out: 1215162761 info 1215162770 out: 1215162761 info 1215162770 out: 1215162761 info 1215162770 out: 1215162761 info 1215162770 out: 1215162762 info 1215162770 out: 1215162762 info 1215162770 out: 1215162762 info 1215162770 out: 1215162763 info

      print+qq(\L@{[ref\&@]}@{['@'x7^'!#2/"!4']});

        Run3 apparently doesn't do concurrency. IPC:Run can be made to do what you want. This works:

        #!/usr/bin/perl # timestamp.pl while(time - $^T<10){ select undef,undef,undef, 0.1; if ( rand()<0.5) { print STDERR time(), " warn\n"; } else { print time(), " info\n"; } } print "end\n"; __END__ use IPC::Run qw ( start pump finish timeout ); my $in; my $out; my @cmd = qw( perl timestamp.pl); ## Build the harness, open all pipes, and launch the subprocesses my $h = start \@cmd, \$in, \$out, \$out ; $in = "" ; ## Now do I/O. start() does no I/O. pump $h while length $in ; ## Wait for all input to go ## Now do some more I/O. while (1){ pump $h until $out =~ s/(.+)\n$//gm ; last if ("end" eq $1); print time, " : $1\n"; } ## Clean up finish $h or die "cat returned $?" ; __END__ 1215226647 : 1215226647 info 1215226647 : 1215226647 info 1215226647 : 1215226647 warn 1215226647 : 1215226647 info 1215226647 : 1215226647 warn 1215226647 : 1215226647 warn 1215226647 : 1215226647 info 1215226647 : 1215226647 info 1215226647 : 1215226647 info 1215226648 : 1215226648 warn


        s//----->\t/;$~="JAPH";s//\r<$~~/;{s|~$~-|-~$~|||s |-$~~|$~~-|||s,<$~~,<~$~,,s,~$~>,$~~>,, $|=1,select$,,$,,$,,1e-1;print;redo}

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://695360]
Approved by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others rifling through the Monastery: (6)
As of 2022-12-07 09:30 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?