Hash Pipe Problem

NateTut has asked for the wisdom of the Perl Monks concerning the following question:

I have reduced my problem to the following: I have two scripts, A.ksh & B.ksh. Each is identical and they look like:

#!/usr/bin/ksh
Count=0
Limit=30
while [ $Count -lt $Limit ] ;
do
   Count=$(($Count + 1))
   sleep 1
   echo $0 has Slept for \[$Count\] Seconds so Far...
done
[download]

My goal is to write a Perl script that executes them in parallel and prints their output. This is what I have:

#!/usr/bin/perl
use strict;
use warnings;

my @Types = ('A', 'B');

my %ScriptName;
my %IsFinished;
my %ProcesseHandle;
foreach my $Type (@Types)
   {
   $ScriptName{$Type} = $Type . '.ksh';
   $IsFinished{$Type} = 0;
   if(!open($ProcesseHandle{$Type}, "$ScriptName{$Type}|"))
      {
      WarnLog("Unable to run Script:[$ScriptName{$Type}]:\n$^E:\n$!:\n
+Error Number $?:$@ ");
      }

   }

while(!$IsFinished{'A'} or !$IsFinished{'B'})
   {
   my %Line;
   foreach my $Type (@Types)
      {
      if(!$IsFinished{$Type})
         {
         $Line{$Type} = <$ProcesseHandle{$Type}>;
         if(!defined($Line{$Type}))
            {
            $IsFinished{$Type} = 1;
            }
         print("\[$Type\]\[$Line{$Type}\]\n");
         }
      }
   }
[download]

And this is what I get:

[A][GLOB(0x2ae38)]
Use of uninitialized value in concatenation (.) or string at HashPipe.
+pl line 33.
[B][]
[A][GLOB(0x2ae38)]
Use of uninitialized value in concatenation (.) or string at HashPipe.
+pl line 33.
[A][]
[download]

And of course it hangs there. Obviously $Line{$Type} = <$ProcesseHandle{$Type}>; isn't doing what I expect. Why?

Also is there a better way to do while(!$IsFinished{'A'} or !$IsFinished{'B'}) so I don't have to hardcode the keys?

Thanks!

Comment on Hash Pipe Problem Select or Download Code

Replies are listed 'Best First'.
Re: Hash Pipe Problem by Roy Johnson (Monsignor) on Feb 26, 2009 at 19:29 UTC
I think that you will find IO::Select helpful. Caution: Contents may have been coded under pressure.	[reply]
Re: Hash Pipe Problem by BrowserUk (Patriarch) on Feb 26, 2009 at 19:53 UTC
T'is easier with threads: `#! perl -sw use strict; use threads; use Thread::Queue; $\|++; my $Q = new Thread::Queue; async { open my $fh, q[ perl -le"$\|++; sleep(1), print qq[$$: $_] for 1 .. 100" \| ] or die $!; $Q->enqueue( $_ ) while <$fh>; $Q->enqueue( undef ); }->detach; async { open my $fh, q[ perl -le"$\|++; sleep(1), print qq[$$: $_] for 1 .. 100" \| ] or die $!; $Q->enqueue( $_ ) while <$fh>; $Q->enqueue( undef ); }->detach; for ( 1 .. 2 ) { print while defined( $_ = $Q->dequeue ); }` [download] Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. "Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."	[reply] [d/l]
Re^2: Hash Pipe Problem by ikegami (Patriarch) on Feb 26, 2009 at 20:23 UTC
For comparison, here's a version without threads: #! perl -sw use strict; use threads; use IO::Select; $\|++; my $sel = IO::Select->new(); my %children; for (1..2) { open my $fh, q[ perl -le'$\|++; sleep(1), print qq[$$: $_] for 1 .. 10' \| ] or die $!; $sel->add($fh); $children{$fh} = ""; } while (%children) { for my $fh ( $sel->can_read() ) { our $buf; local buf = \( $children{$fh} ); # alias sysread($fh, $buf, 4096, length($buf)) or do { print("$buf\n") if length($buf); $sel->remove($fh); delete($children{$fh}); next; }; print $1 while $buf =~ s/^(.\n)//; } } [download] Definitely more complex and less scalable.	[reply] [d/l]
Re^3: Hash Pipe Problem by BrowserUk (Patriarch) on Feb 26, 2009 at 20:41 UTC
Hm. I tried your code above--the only change was 's for "s in the command line and...it consumes 100% cpu, never produces any output and I had to ^C to terminate it. Like I said, t'is easier with threads. Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. "Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."	[reply]
Re^4: Hash Pipe Problem by ikegami (Patriarch) on Feb 26, 2009 at 20:44 UTC
Re: Hash Pipe Problem by Anonymous Monk on Feb 26, 2009 at 19:24 UTC
`D:\>perl -MO=Deparse -e" < $ProcesseHandle{$Type} > use File::Glob (); glob(' ' . $ProcesseHandle{$Type} . ' '); -e syntax OK` [download] You want readline	[reply] [d/l]
Re^2: Hash Pipe Problem by NateTut (Deacon) on Feb 26, 2009 at 19:31 UTC
Thanks, that works, but I'm still cornfused why the original version didn't work. I'm new to Deparse so I don't know what it is telling me.	[reply] [d/l]
Re^3: Hash Pipe Problem by Anonymous Monk on Feb 26, 2009 at 19:35 UTC
Its telling you how that bit of code was parsed. per perlop, If what's within the angle brackets is neither a filehandle nor a simple scalar variable containing a filehandle name, typeglob, or typeglob reference, it is interpreted as a filename pattern to be globbed `<FOO> <$FOO>` is readline, but `<$foo{barr}>` is glob	[reply] [d/l] [select]
Re^4: Hash Pipe Problem by NateTut (Deacon) on Feb 26, 2009 at 19:38 UTC
Re^5: Hash Pipe Problem by almut (Canon) on Feb 26, 2009 at 19:45 UTC
Some notes below your chosen depth have not been shown here
Re^5: Hash Pipe Problem by Anonymous Monk on Feb 26, 2009 at 19:51 UTC
Some notes below your chosen depth have not been shown here
Re: Hash Pipe Problem by GrandFather (Saint) on Feb 26, 2009 at 20:08 UTC
Be warned: Hash pipes are illegal in some places. True laziness is hard work	[reply]
Re^2: Hash Pipe Problem by NateTut (Deacon) on Feb 26, 2009 at 20:24 UTC
Indeed and too much time on the hash pipe leads to bade code!	[reply] [d/l]
Re: Hash Pipe Problem by Anonymous Monk on Jan 05, 2015 at 07:02 UTC
Not exactly the better way but you can reduce one operation overhead. (!$IsFinished{'A'} or !$IsFinished{'B'}) can be written as (!($IsFinished{'A'} and $IsFinished{'B'})). The previous one has 3 operations while the latter has only 2 operations	[reply]


Pathologically Eclectic Rubbish Lister
	PerlMonks