Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer

Re: Receiving data asynchronously in a long-run process

by rcaputo (Chaplain)
on Sep 21, 2009 at 17:45 UTC ( #796583=note: print w/replies, xml ) Need Help??

in reply to Receiving data asynchronously in a long-run process

If you use a named pipe and file locking to synchronize its access, then the writer (or writers) will block until a reader is ready. In your case, if no workers are ready to read from the named pipe, then the job issuer will block.

It's important to consider that workers may need to poll, even if you'd prefer them not to. If no workers are ready, then the job issuer will need to save the job somewhere reliable. That's generally a database or spool directory. In that case, workers must poll for new jobs when the become ready again.

  • Comment on Re: Receiving data asynchronously in a long-run process

Replies are listed 'Best First'.
Re^2: Receiving data asynchronously in a long-run process
by Annirak (Novice) on Sep 21, 2009 at 19:37 UTC

    After thinking about it some more, I think the way to do what I've proposed is to implement some locking between the worker and the issuer. Doing this requires a manager in the middle somewhere. The manager would need to hold the jobs in a queue and wait for the workers to request jobs/indicate readiness to receive jobs.

    I'm also not sure that my approach is the best. It might be better to provide for the option of spreading my workers across multiple machines. Naturally, that will mean implementing TCP connections instead of pipes. It will likely also mean a more complex startup process, and much more complex management.

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://796583]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chanting in the Monastery: (3)
As of 2022-09-27 04:59 GMT
Find Nodes?
    Voting Booth?
    I prefer my indexes to start at:

    Results (118 votes). Check out past polls.