Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot

Re: getstore failure for BBC website

by tangent (Vicar)
on Jan 28, 2018 at 22:54 UTC ( #1208016=note: print w/replies, xml ) Need Help??

in reply to getstore failure for BBC website

When I enter that URL it gets turned in to:
Note the https.

Using LWP::UserAgent (which will follow redirects) I got it to work:

use LWP::UserAgent; my $url = ' +/13'; my $ua = LWP::UserAgent->new(ssl_opts => {verify_hostname => 0}); my $res = $ua->get( $url ); my $html = $res->content; print $html;

Replies are listed 'Best First'.
Re^2: getstore failure for BBC website
by merrymonk (Hermit) on Jan 29, 2018 at 13:33 UTC
    Many thanks for that - I think it means that the BBC have changed the 'structure' of their web pages during the last 6 years.
    I have tried to use your Perl but I got an message saying that "LWP will support https URLs if the LWP::Protocol::https module is installed"
    I have done that but get message about another missing module
    Therefore I am talking to my Perl supplier since I do not want to go down a 'false' trail.

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1208016]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chanting in the Monastery: (3)
As of 2022-05-21 06:03 GMT
Find Nodes?
    Voting Booth?
    Do you prefer to work remotely?

    Results (76 votes). Check out past polls.