Re: HTTPS returning 400 from Linux but 200 OK from Windows

Replies are listed 'Best First'.
Re^2: HTTPS returning 400 from Linux but 200 OK from Windows by Random_Walk (Prior) on Jan 28, 2017 at 07:00 UTC
I found a 5.18 perl on a network adjacent SuSE 12sp1 server me@linux:/home/rpollard> export HTTPS_PROXY='http://user:pass@proxy:80 +80' me@linux:/home/rpollard> echo $HTTPS_PROXY http://user:pass@proxy:8080 me@linux:/home/rpollard> curl https://www.google.com <HTML><HEAD><meta http-equiv="content-type" content="text/html;charset +=utf-8"> <TITLE>302 Moved</TITLE></HEAD><BODY> <H1>302 Moved</H1> The document has moved <A HREF="https://www.google.se/?gfe_rd=cr&ei=6T-MWJGBLsiq8weZ_YCIA +Q">here</A>. </BODY></HTML> me@linux:/home/rpollard> perl GetTest.pl 400 Bad Request at GetTest.pl line 16. me@linux:/home/rpollard> unset HTTPS_PROXY me@linux:/home/rpollard> echo $HTTPS_PROXY me@linux:/home/rpollard> curl https://www.google.com curl: (6) Could not resolve host: www.google.com me@linux:/home/rpollard> perl GetTest.pl 500 Can't connect to google.com:443 (Bad hostname) at GetTest.pl line +16. me@linux:/home/rpollard> perl -v This is perl 5, version 18, subversion 2 (v5.18.2) built for x86_64-li +nux-thread-multi Copyright 1987-2013, Larry Wall Perl may be copied only under the terms of either the Artistic License + or the GNU General Public License, which may be found in the Perl 5 source ki +t. Complete documentation for Perl, including FAQ lists, should be found +on this system using "man perl" or "perldoc perl". If you have access to + the Internet, point your browser at http://www.perl.org/, the Perl Home Pa +ge. [download] That is using the same test script as in the root node. I think this shows both curl and perl are using the same HTTPS_PROXY here. Now I have a more modern Perl, I will install LWP::ConsoleLogger and see what that tells me. Cheers, R. Pereant, qui ante nos nostra dixerunt!	[reply] [d/l]
Re^3: HTTPS returning 400 from Linux but 200 OK from Windows by Random_Walk (Prior) on Jan 28, 2017 at 08:22 UTC
Server is all locked down, no gcc/cc etc and seos won't even let me install gcc, so I can't cpan LWP::ConsoleLogger. Now I can either go through the full change cycle to get a test server, or use curl in my scripts to pull back the data form the REST API I am after, then parse it from there. Neither choice appeals Cheers, R. Pereant, qui ante nos nostra dixerunt!	[reply] [d/l]
Re^2: HTTPS returning 400 from Linux but 200 OK from Windows by Random_Walk (Prior) on Feb 13, 2017 at 18:54 UTC
It was as simple as this. SuSE Perl did not have it, Windows did... `cpan LWP::Protocol::https` [download] This http://blogs.perl.org/users/brian_d_foy/2011/07/now-you-need-lwpprotocolhttps.html brought me enlightenment. Cheers, R. Pereant, qui ante nos nostra dixerunt!	[reply] [d/l]
Re^3: HTTPS returning 400 from Linux but 200 OK from Windows by hippo (Bishop) on Feb 13, 2017 at 21:52 UTC
SUSE does appear to have it, at least in recent versions.	[reply]
Re^2: HTTPS returning 400 from Linux but 200 OK from Windows by Random_Walk (Prior) on Jan 28, 2017 at 06:41 UTC
Hi Corion, I have HTTP_PROXY and HTTPS_PROXY set with http://user:pass@proxyhost and these are used by curl so must be correct. I tried to install lwp::ConsoleLogger with CPAN, and it failed as my Perl is too old (5.10). I suspect this may be the cause, so I am having a quick look around for a server in the same data center with a newer Perl to test if that is the critical difference. The Perl on my lappy that works is 5.22.1. Cheers, R. Pereant, qui ante nos nostra dixerunt!	[reply]
Re^2: HTTPS returning 400 from Linux but 200 OK from Windows by Random_Walk (Prior) on Feb 13, 2017 at 18:07 UTC
I have LWP::ConsoleLogger installed after much struggle to get it built within the restrictions of the client site. Sadly it has brought me no new enlightenment. My only though now is that Perl does not like my https_proxy being an http url. # curl https://google.com/ <HTML><HEAD><meta http-equiv="content-type" content="text/html;charset +=utf-8"> <TITLE>302 Moved</TITLE></HEAD><BODY> <H1>302 Moved</H1> The document has moved <A HREF="https://www.google.se/?gfe_rd=cr&ei=IvWhWPTvI4b67gSTn7ygA +g">here</A>. </BODY></HTML> # perl test.pl GET https://google.com/ .-----------------+--------------------. \| Request Header \| Value \| +-----------------+--------------------+ \| Accept-Encoding \| gzip \| \| User-Agent \| WWW-Mechanize/1.83 \| '-----------------+--------------------' ==> 500 Can't connect to google.com:443 .-----------------+-------------------------------. \| Response Header \| Value \| +-----------------+-------------------------------+ \| Client-Date \| Mon, 13 Feb 2017 18:04:23 GMT \| \| Client-Warning \| Internal response \| \| Content-Type \| text/plain \| '-----------------+-------------------------------' .--------------------------------------------------------------------- +--------------------------------------------------------------------- +---. \| Content + + \| +--------------------------------------------------------------------- +--------------------------------------------------------------------- +---+ \| Can't connect to google.com:443 + + \| \| + + \| \| Name or service not known at /usr/lib/perl5/site_perl/5.18.2/LWP/Pro +tocol/http.pm line 47, <DATA> line 1. + \| '--------------------------------------------------------------------- +--------------------------------------------------------------------- +---' .--------------------------------------------------------------------- +--------------------------------------------------------------------- +---. \| Text + + \| +--------------------------------------------------------------------- +--------------------------------------------------------------------- +---+ \| Can't connect to google.com:443 + + \| \| + + \| \| Name or service not known at /usr/lib/perl5/site_perl/5.18.2/LWP/Pro +tocol/http.pm line 47, <DATA> line 1. + \| '--------------------------------------------------------------------- +--------------------------------------------------------------------- +---' Error GETing https://google.com/: Can't connect to google.com:443 at t +est.pl line 22. [download] Thanks for taking the time. I have at least learned a bit more about the web tools available in Perl :) Cheers, R. Pereant, qui ante nos nostra dixerunt!	[reply] [d/l]
Re^3: HTTPS returning 400 from Linux but 200 OK from Windows by Corion (Patriarch) on Feb 13, 2017 at 18:32 UTC
Unfortunately, LWP::ConsoleLogger doesn't seem to be talkative about when it is trying to connect to a proxy. You can try to find which proxy LWP will try to use: `#!perl -w use strict; use LWP::ConsoleLogger::Easy 'debug_ua'; use Data::Dumper; use LWP::UserAgent; my $ua=LWP::UserAgent->new(); warn Dumper $ua->proxy('https'); my $logger=debug_ua($ua,5); print Dumper \%ENV; $ua->env_proxy; warn Dumper $ua->proxy('https'); $ua->get('https://google.com')` [download] This should show you that your proxy URL is in `%ENV` and also picked up from there in the `$ua`. Maybe your proxy does not let Perl through? Have you tried masquerading your Perl script as `curl` (because that seems to work?)?: `$ua->default_headers->header('User-Agent' => 'curl/7.10.x (i386-redhat +-linux-gnu) libcurl/7.10.x OpenSSL/0.9.7a ipv6 zlib/1.1.4');` [download]	[reply] [d/l] [select]


Welcome to the Monastery
	PerlMonks