Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical
 
PerlMonks  

Re^3: Speeding up/parallelizing hundreds of HEAD requests

by perrin (Chancellor)
on Sep 16, 2007 at 20:06 UTC ( [id://639292]=note: print w/replies, xml ) Need Help??


in reply to Re^2: Speeding up/parallelizing hundreds of HEAD requests
in thread Speeding up/parallelizing hundreds of HEAD requests

You mentioned caching the requests instead of actually doing them every time. This is a good way to do that. It's more efficient than a bunch of database queries, and makes it very easy to control timeouts on the cache, in case you only want to check every 2 hours or so.
  • Comment on Re^3: Speeding up/parallelizing hundreds of HEAD requests

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://639292]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others about the Monastery: (5)
As of 2024-04-18 05:35 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found