Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

Re: getstore failure for BBC website

by tangent (Vicar)
on Jan 28, 2018 at 22:54 UTC ( #1208016=note: print w/replies, xml ) Need Help??


in reply to getstore failure for BBC website

When I enter that URL it gets turned in to:
https://www.bbc.co.uk/schedules/p00fzl7j/2015/10/13
Note the https.

Using LWP::UserAgent (which will follow redirects) I got it to work:

use LWP::UserAgent;
my $url = 'http://www.bbc.co.uk/radio4/programmes/schedules/fm/2015/10
+/13';
my $ua = LWP::UserAgent->new(ssl_opts => {verify_hostname => 0});
my $res = $ua->get( $url );
my $html = $res->content;
print $html;

[download]

Replies are listed 'Best First'.
Re^2: getstore failure for BBC website
by merrymonk (Hermit) on Jan 29, 2018 at 13:33 UTC
    Many thanks for that - I think it means that the BBC have changed the 'structure' of their web pages during the last 6 years.
    I have tried to use your Perl but I got an message saying that "LWP will support https URLs if the LWP::Protocol::https module is installed"
    I have done that but get message about another missing module
    Therefore I am talking to my Perl supplier since I do not want to go down a 'false' trail.
[reply]

In Section Seekers of Perl Wisdom

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?

www.com | www.net | www.org
Node Status?
node history
Node Type: note [id://1208016]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chanting in the Monastery: (3)
As of 2022-05-21 06:03 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Do you prefer to work remotely?



    Results (76 votes). Check out past polls.
    Notices?