Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses
 
PerlMonks  

Re^2: LWP::UserAgent Bad and Forbidden requests

by 1arryb (Acolyte)
on Dec 15, 2011 at 20:07 UTC ( [id://943825]=note: print w/replies, xml ) Need Help??


in reply to Re: LWP::UserAgent Bad and Forbidden requests
in thread LWP::UserAgent Bad and Forbidden requests

Hi Anonymous,

In general, changing your user agent string to spoof Firefox, for example, is not a legitimate workaround. LWP::UserAgent is kind of a grey area, since you could use it to build interactive, as well as automated web applications. However, the folks at sciencedirect.com are clearly taking a hard line against it. If taioma chooses to spoof his user agent from a spider, and it is detected, he could get himself (and his whole company) banned.

Cheers,

Larry

  • Comment on Re^2: LWP::UserAgent Bad and Forbidden requests

Replies are listed 'Best First'.
Re^3: LWP::UserAgent Bad and Forbidden requests
by tobyink (Canon) on Dec 15, 2011 at 21:06 UTC

    I agree that spoofing another user agent is an ugly hack, however setting an appropriate user agent is always a good idea:

    my $ua = LWP::UserAgent->new(agent => 'MyApp/0.1');
Re^3: LWP::UserAgent Bad and Forbidden requests
by Anonymous Monk on Dec 15, 2011 at 22:11 UTC

    In general, changing your user agent string to spoof Firefox, for example, is not a legitimate workaround.

    says who?

    and his whole company) banned.

    I am not a lawyer, I don't give legal advice

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://943825]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others surveying the Monastery: (2)
As of 2024-04-24 23:52 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found