• Ask nicely in a quiet voice
• Suggest firmly in a modest voice
• Demand loudly
• Give up and do it yourself

Note that if you use pipe-open, open returns the PID of the child. To determine a child terminating, set up a signal handler to catch SIGCLD. This signal doesn't tell you the pid, but if you do a wait or a waitpid in the handler, you do. Given the PID of the terminated child, you know which task was assigned to it (assuming you stored this information when dealing out the tasks).

Study man perlipc, it's a real treasure trove. It should have enough examples to get you on your way.

You are surprisingly right,

I somehow always regarded

$pid = open(KID_TO_WRITE, "|-");
[download]

trick as a way to talk with only one child, but if we rewrite it like this:

$fh = new FileHandle;
my $sleep_count = 0;

do {
        $pid = open($fh, "|-");
        unless (defined $pid) {
                warn "cannot fork: $!";
                die "bailing out" if $sleep_count++ > 6;
                sleep 10;
        }
} until defined $pid;

my (@some_data)=qw[a be ce];
if ($pid) {
         print $fh @some_data;
         close($fh) || warn "kid exited $?";
} else {
        while (<STDIN>) {
                print "GOT: $_\n"; # child's STDIN is parent's KID
        }
        exit;  # don't forget this
};
[download]

then we get to hold whole array of different $fd's, each for talking with one child. This would be clean and reliable, exactly what I wanted.

Thanx.

on second thought..this doesn't work that well with more then one fd open...it hangs on close($fd). Maybe there's an error in my code:

print "I'm $$\n;";
use FileHandle;

$fh = new FileHandle;
$fh1 = new FileHandle;
my $sleep_count = 0;

do {
        $pid = open($fh, "|-");
        unless (defined $pid) {
                warn "cannot fork: $!";
                die "bailing out" if $sleep_count++ > 6;
                sleep 10;
        }
} until defined $pid;
if ($pid) {
        print "I'm master($$) $pid...\n";
} else {
        print "Waiting for commands($$)..\n";
        while (<STDIN>) {
                print "GOT: $_\n"; # child's STDIN is parent's KID
        }
        exit;  # don't forget this
};


do {
        $pid1 = open($fh1, "|-");
        unless (defined $pid1) {
                warn "cannot fork: $!";
                die "bailing out" if $sleep_count++ > 6;
                sleep 10;
        }
} until defined $pid1;
if ($pid1) {
        print "I'm master($$) $pid1...\n";
} else {
        print "Waiting for commands($$)..\n";
        while (<STDIN>) {
                print "GOT: $_\n"; # child's STDIN is parent's KID
        }
        exit;  # don't forget this
};


my (@some_data)=qw[a be ce];
print "($$)PID: $pid\n";
print $fh @some_data;
print "($$) printing done, let's close:\n";
close($fh) || warn "kid exited $?";

print "($$)PID: $pid1\n";
print $fh1 @some_data;
print "($$) printing done, let's close:\n";
close($fh1) || warn "kid exited $?";


print "END FH TEST\n\n\n";
[download]

It's pretty rare to search my site and not find an article that contains a particular topic, but if you do, please email me, because I can always use a new suggestion or two.

It's pretty rare to search my site and not find an article that contains a particular topic

Oh, Merlyn, you're such a spoil-sport. It is us other Monks which have to say that we're not surprised you have written a column about it!

;-)

CountZero

"If you have four groups working on a compiler, you'll get a 4-pass compiler." - Conway's Law

I would use a database as the communication mechanism. That way, you get easy and convenient control over the jobs, when they were started and when they were finished (or hanging). For the job description, you can either come up with a keyword mechanism or consider to put the actual code to be run into the database - this makes for convenient extensibility.

I'm sorry, but it seems like you're playing with databases and everything looks like a nail.

Proposing additional and significant external dependencies for a nice self-contained program is not very smart.

I'm sorry that I misunderstood your problem. I thought that your statement:

The hard part seems to be communicating with children, with one child, I would probably use STDIN/STDOUT, servers like apache have symmetrical children, so I'm kind of lost as to what is the best way to do something like that.

implied that your children are not symmetrical and maybe even distributed across machines, and that children die-ing from unknown reasons might be common.

Maybe you should discuss the scope of your program in the original post to some larger extent - I found DBD::SQLite a very convenient, lightweight and self-contained database, for example.

You know it might actauly be a nail in some cases. It is odd that you ask a question and then are soo rude when you get an answer. Databases can be the right tool and they might not be. That depends entirely on your situation. If you wanted to have 100 children spread over 100 computers then a database would be a perfect solution. You said you wanted "possibly non-blocking and reliable way". Database is *one* of the answers that fits the question. For instances I work on windows and linux boxes and never feel comforatble with forking or "traditional" IPC. However I am always working with databases and most scripts I write use databases for something (not out of habit, i work with large sets of scheduling and accounting data in a database). In my case lots of times a DB is the easiest, fastest and most comfortable route for my problems. As far as we knew you where already connected to a DB and this would be a good solution.

"Proposing additional and significant external dependencies for a nice self-contained program is not very smart." -- Considering we don't know the current dependencies of your program, we could hardly deduce that a database wouldn't fit your needs.

Ask a question, get lots of answers, don't critise an answer or mock the person giving it just because it doesn't fit your need. We can't read your mind or know your particular needs or desires unless you tell us first. Sorry for the rant but its realy hurts to see someone try to help and provide a fair answer and be treated the way you tried corion.


___________
Eric Hodges

On (4.), what is killing your children before their task is done?

There are lots of ways to talk to child processes. If there is a small fixed set of messages, signaling with kill is effective, with %SIG copy-on-write. If you need more extensive communication, pipes are my favorite, but sockets are also good. Each child will need to parse what you send it. See perlipc for details.

Another way is to only launch a child when it and the parent will know what it's supposed to do. That's best on systems, like Linux, where fork is cheap.

A: You (as a designer) don't know.

Anyhoo, some tasks may take forever, thus you need to keep it running indefinitely.

I don't know how launching a child only when task is ready is going to help, you still need to tell your newborn child what kind of tasks you prepared for them ( ie, pass some argument, I would go for $ARGV[], this seems simplest ).

I like using threads and thread queues You can give each child thread its own instruction queue to assign it tasks or have children picking tasks up off a common jobs to be done queue (depends how specialist your children are). The parent can be signalled results back through a common queue for all children. Children can also post heartbeat messages to this queue so you know if one is hanging/dead.

POE may be worth a look. I had a similar job to do a couple of years ago and POE looked ideal but management refused installation of any (non-core)modules, yawn, so I never got to really use it. Management actualy refused any open source but had tons coded in Perl, go figure.

Cheers,
R.

actually, threads won't cut it for my problem(to little separation, and reliability), and same goes for POE (because some tasks are blocking).

I'd be interested to know if this is useful for you, or if there are any issues with it. (No one has commented on it in the nearly 2 years it's been posted, so either 1) no one has seen it, 2) it's useful, or 3) it doesn't help, and no one has complained.)