Hi Monks,
This question has been asked before. I've tried the suggested solutions without success, so I'm trying again.
I'm trying to use perl to connect to http://python.org/download, which redirects to https://python.org/download.
I can download this page via wget:
$ wget http://python.org/download
--2015-06-19 13:18:26-- http://python.org/download
Resolving python.org (python.org)... 104.130.43.121, 2001:4802:7901:0:
+e60a:1375:0:5
Connecting to python.org (python.org)|104.130.43.121|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://python.org/download [following]
--2015-06-19 13:18:26-- https://python.org/download
Connecting to python.org (python.org)|104.130.43.121|:443... connected
+.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://www.python.org/download [following]
--2015-06-19 13:18:26-- https://www.python.org/download
Resolving www.python.org (www.python.org)... 23.235.47.223
Connecting to www.python.org (www.python.org)|23.235.47.223|:443... co
+nnected.
HTTP request sent, awaiting response... 301 MOVED PERMANENTLY
Location: https://www.python.org/download/ [following]
--2015-06-19 13:18:26-- https://www.python.org/download/
Reusing existing connection to www.python.org:443.
HTTP request sent, awaiting response... 301 MOVED PERMANENTLY
Location: https://www.python.org/downloads/ [following]
--2015-06-19 13:18:26-- https://www.python.org/downloads/
Reusing existing connection to www.python.org:443.
HTTP request sent, awaiting response... 200 OK
Length: 86849 (85K) [text/html]
Saving to: `download'
... (it worked)
Here's my LWP test program:
#!/usr/bin/perl
use Modern::Perl "2012";
use utf8;
use Dumpvalue;
use LWP::UserAgent;
my $dumper = Dumpvalue->new( dumpReused => 1 );
my $ua = LWP::UserAgent->new();
$ua->ssl_opts( verify_hostnames => 0 );
my $response = $ua->get('http://python.org/downloads');
$dumper->dumpValue($response);
and here is the result:
'_content' => 'Can\'t connect to python.org:443
Network is unreachable at /usr/local/share/perl5/LWP/Protocol/http.pm
+line 47.
'
'_headers' => HTTP::Headers=HASH(0x1acb130)
'client-date' => 'Fri, 19 Jun 2015 20:48:22 GMT'
'client-warning' => 'Internal response'
'content-type' => 'text/plain'
'_msg' => 'Can\'t connect to python.org:443'
'_previous' => HTTP::Response=HASH(0x1c79890)
'_content' => ''
'_headers' => HTTP::Headers=HASH(0x1a43f00)
'client-date' => 'Fri, 19 Jun 2015 20:48:22 GMT'
'client-peer' => '104.130.43.121:80'
'client-response-num' => 1
'connection' => 'close'
'content-length' => 0
'location' => 'https://python.org/download'
'_msg' => 'Moved Permanently'
'_protocol' => 'HTTP/1.1'
'_rc' => 301
'_request' => HTTP::Request=HASH(0x17c05c0)
'_content' => ''
'_headers' => HTTP::Headers=HASH(0x17e07e0)
'user-agent' => 'libwww-perl/6.13'
'_method' => 'GET'
'_uri' => URI::http=SCALAR(0x16a4d08)
-> 'http://python.org/download'
'_uri_canonical' => URI::http=SCALAR(0x16a4d08)
-> 'http://python.org/download'
'_rc' => 500
'_request' => HTTP::Request=HASH(0x1a43dc8)
'_content' => ''
'_headers' => HTTP::Headers=HASH(0x1a43df8)
'user-agent' => 'libwww-perl/6.13'
'_method' => 'GET'
'_protocol' => undef
'_uri' => URI::https=SCALAR(0x1a43c48)
-> 'https://python.org/download'
I tried getting the https url directly (avoiding the redirect) with identical results.
Some have suggested using the environment to pass ssl_opts. I Tried:
export PERL_LWP_SSL_VERIFY_HOSTNAME=0 PERL_LWP_SSL_CA_PATH=/
but this also had no effect.
I've also tried force installing LWP::Protocol::https::Socket and upgrading LWP, LWP::Protocol::https, Net::SSLeay, and Crypt::SSLeay to the latest CPAN versions. None of this had any effect.
I'm running on a Fedora 17 system with Perl 5.14.
I even set my user agent to match wget's in case python.org just didn't like LWP. No joy.
I am now out of ideas. Any help would be appreciated. Thanks!
Larry Barnett