Beefy Boxes and Bandwidth Generously Provided by pair Networks
more useful options

Re: "406 not acceptable" errors with LWP::UserAgent::get

by Your Mother (Bishop)
on Aug 17, 2020 at 23:44 UTC ( #11120858=note: print w/replies, xml ) Need Help??

in reply to "406 not acceptable" errors with LWP::UserAgent::get

Seems the main thing it doesnít like is the agent. Your code gave me the same error, even tweaked and pruned down until I swapped the agent for something that isnít, apparently, blacklisted.

use 5.10.0; use LWP::UserAgent; my $ua = LWP::UserAgent->new; $ua->agent('DomoArigato/3.0'); my $url = ''; my $response = $ua->get($url); say $response->is_success ? "OK!" : $response->as_string; __END__ OK!

Replies are listed 'Best First'.
Re^2: "406 not acceptable" errors with LWP::UserAgent::get
by thomas895 (Deacon) on Aug 18, 2020 at 00:34 UTC

    Usually to get around these kinds of blocks the easiest way is to set the user agent string to be the same one your browser uses

    "Excuse me for butting in, but I'm interrupt-driven..."

      Disagree in the main. :P People blacklist agents, not whitelist, and a one-off for an agent that is NEVER abusive to a service is less likely to get crapcanned than one that shares a (base) name with 30% of the traffic.

      Thanks for the replies. I tried setting the agent to the user-agent string used by my current version of Firefox and had no trouble. (I'm still not sure why I was getting a 406 error for this podcast instead of a 403 Forbidden error, which I was getting on a number of podcasts with the default agent of "libwww-perl".)

        You can of course do whatever you want but I encourage you to use a unique agent name and version for your script; maybe even a less whimsical one than my original suggestion. :P I meant to expand on this and forgot. Itís more transparentóIíd argue more honestóbetter metrics for the host, and being a good netizen, from an API standpoint, is better for everyone and, I believe, reflects well on Perl compared to some other more, uh, mercenary communities. Update: response codes are at the discretion of the host code and webserver. They are frequently not particularly sensible.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://11120858]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others taking refuge in the Monastery: (3)
As of 2020-11-27 06:49 GMT
Find Nodes?
    Voting Booth?

    No recent polls found