Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"
 
PerlMonks  

Re: LWP::UserAgent gets 403 when other browser does not

by pileofrogs (Priest)
on Jul 26, 2010 at 16:33 UTC ( [id://851404]=note: print w/replies, xml ) Need Help??


in reply to LWP::UserAgent gets 403 when other browser does not

Thanks all!

I decided that I would try multiple times with different settings before declaring a link bad. Some servers don't do HEAD requests, others need a useragent and some want a referrer.

This makes the whole thing slower, but that doesn't matter when it runs automatically at night.

Thanks again!

--Pileofrogs

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://851404]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others goofing around in the Monastery: (8)
As of 2024-03-28 09:42 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found