Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things
 
PerlMonks  

Re: Threaded Web requests

by rcaputo (Chaplain)
on Jan 02, 2010 at 23:05 UTC ( [id://815353]=note: print w/replies, xml ) Need Help??


in reply to Threaded Web requests

If you were using something like POE::Component::Client::HTTP (see these recipes), I would recommend starting a polite number of parallel requests, perhaps 10 or 20, and then firing off a new request for each response that arrives. This is a handy way to limit parallelism without a lot of bookkeeping.

You may still be able to do this. It depends whether you can join() on "the next thread to finish". For every thread that joins, start another with the next request. Exit when you've run out of @addresses and @threads.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://815353]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others exploiting the Monastery: (9)
As of 2024-04-23 18:31 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found