Beefy Boxes and Bandwidth Generously Provided by pair Networks
more useful options
 
PerlMonks  

Re: "406 not acceptable" errors with LWP::UserAgent::get

by Your Mother (Bishop)
on Aug 17, 2020 at 23:44 UTC ( #11120858=note: print w/replies, xml ) Need Help??


in reply to "406 not acceptable" errors with LWP::UserAgent::get

Seems the main thing it doesnít like is the agent. Your code gave me the same error, even tweaked and pruned down until I swapped the agent for something that isnít, apparently, blacklisted.

use 5.10.0; use LWP::UserAgent; my $ua = LWP::UserAgent->new; $ua->agent('DomoArigato/3.0'); my $url = 'http://makeoursmarvel.com/feed/podcast/'; my $response = $ua->get($url); say $response->is_success ? "OK!" : $response->as_string; __END__ OK!

Replies are listed 'Best First'.
Re^2: "406 not acceptable" errors with LWP::UserAgent::get
by thomas895 (Deacon) on Aug 18, 2020 at 00:34 UTC

    Usually to get around these kinds of blocks the easiest way is to set the user agent string to be the same one your browser uses

    -Thomas
    "Excuse me for butting in, but I'm interrupt-driven..."

      Disagree in the main. :P People blacklist agents, not whitelist, and a one-off for an agent that is NEVER abusive to a service is less likely to get crapcanned than one that shares a (base) name with 30% of the traffic.

      Thanks for the replies. I tried setting the agent to the user-agent string used by my current version of Firefox and had no trouble. (I'm still not sure why I was getting a 406 error for this podcast instead of a 403 Forbidden error, which I was getting on a number of podcasts with the default agent of "libwww-perl".)

        You can of course do whatever you want but I encourage you to use a unique agent name and version for your script; maybe even a less whimsical one than my original suggestion. :P I meant to expand on this and forgot. Itís more transparentóIíd argue more honestóbetter metrics for the host, and being a good netizen, from an API standpoint, is better for everyone and, I believe, reflects well on Perl compared to some other more, uh, mercenary communities. Update: response codes are at the discretion of the host code and webserver. They are frequently not particularly sensible.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://11120858]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others taking refuge in the Monastery: (3)
As of 2020-11-27 06:49 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?