Beefy Boxes and Bandwidth Generously Provided by pair Networks
Welcome to the Monastery
 
PerlMonks  

Help with LWP & SSL 500 Internal response error

by dcsastry (Initiate)
on May 12, 2015 at 15:33 UTC ( #1126436=perlquestion: print w/replies, xml ) Need Help??

dcsastry has asked for the wisdom of the Perl Monks concerning the following question:

Hi All,

I have a vexing problem with Perl LWP and accessing an internal company HTTPS site.

I was having lots of problems getting an SSL handshake to work, and I finally got around that.

Then I was having problems with credentials not working, and I finally got around that.

Now I am having some kind of internal issue with HTTP. It seems like the SSL handshake is done, but I cannot pull data from the website after that. I just get this 500 internal error on the client side.

I am not sure what I can do to debug this issue - any ideas for things I need to look at? Thanks!

Here is the code I used (edited for posting)

use LWP::UserAgent; use LWP::Protocol::https; use HTTP::Request; use HTTP::Cookies; $IO::Socket::SSL::DEBUG=3; our $ua = LWP::UserAgent->new(keep_alive=>1); $ua->agent("Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; AS; rv:11 +.0) like Gecko"); $ua->ssl_opts(SSL_ca_file => 'C:\OpenSSL-Win32\cert\companyx Root CA B +ase64.cer'); $ua->ssl_opts(verify_hostname=>1); $ua->cookie_jar(HTTP::Cookies->new(file => "$ENV{HOME}/.cookies.txt")) +; $ENV{http_proxy} = ''; our $host = "cdis.companyx.com:9283"; our $url1 = "https://$host/cdis/default.asp"; $ua->credentials("$host",'', "AMR\\username", 'password'); $ua->timeout(10); my $response1 = $ua->get($url1); if ($response1->is_success) { print $response1->content; } else { print "Error: " . $response1->status_line . "\n"; print $response1->headers()->as_string(), "\n"; print $response1->content; } exit(0);

Here is the output

DEBUG: .../IO/Socket/SSL.pm:2633: new ctx 62187576 DEBUG: .../IO/Socket/SSL.pm:549: socket not yet connected DEBUG: .../IO/Socket/SSL.pm:551: socket connected DEBUG: .../IO/Socket/SSL.pm:573: ssl handshake not started DEBUG: .../IO/Socket/SSL.pm:606: using SNI with hostname cdis.companyx +.com DEBUG: .../IO/Socket/SSL.pm:641: request OCSP stapling DEBUG: .../IO/Socket/SSL.pm:660: set socket to non-blocking to enforce + timeout=10 DEBUG: .../IO/Socket/SSL.pm:676: Net::SSLeay::connect -> -1 DEBUG: .../IO/Socket/SSL.pm:686: ssl handshake in progress DEBUG: .../IO/Socket/SSL.pm:696: waiting for fd to become ready: SSL w +ants a read first DEBUG: .../IO/Socket/SSL.pm:716: socket ready, retrying connect DEBUG: .../IO/Socket/SSL.pm:2536: did not get stapled OCSP response DEBUG: .../IO/Socket/SSL.pm:2489: ok=1 cert=62234208 DEBUG: .../IO/Socket/SSL.pm:2489: ok=1 cert=62234544 DEBUG: .../IO/Socket/SSL.pm:2489: ok=1 cert=62234432 DEBUG: .../IO/Socket/SSL.pm:2489: ok=1 cert=62234320 DEBUG: .../IO/Socket/SSL.pm:1581: scheme=www cert=62234320 DEBUG: .../IO/Socket/SSL.pm:1591: identity=cdis.companyx.com cn=cdis.c +ompanyx.com alt=2 fm1seagwebpd01.amr.corp.companyx.com 2 fm1seagwebpd +02.amr.corp.companyx.com 2 fm1seagwebpd03.amr.corp.companyx.com 2 cdi +sprod-node1.companyx.com 2 cdisprod-node2.companyx.com 2 cdisprod-nod +e3.companyx.com 2 cdis.companyx.com DEBUG: .../IO/Socket/SSL.pm:676: Net::SSLeay::connect -> -1 DEBUG: .../IO/Socket/SSL.pm:686: ssl handshake in progress DEBUG: .../IO/Socket/SSL.pm:696: waiting for fd to become ready: SSL w +ants a read first DEBUG: .../IO/Socket/SSL.pm:716: socket ready, retrying connect DEBUG: .../IO/Socket/SSL.pm:676: Net::SSLeay::connect -> 1 DEBUG: .../IO/Socket/SSL.pm:731: ssl handshake done DEBUG: .../IO/Socket/SSL.pm:2666: free ctx 62187576 open=62187576 DEBUG: .../IO/Socket/SSL.pm:2671: free ctx 62187576 callback DEBUG: .../IO/Socket/SSL.pm:2678: OK free ctx 62187576 Error: 500 Status read failed: Unknown error Content-Type: text/plain Client-Date: Tue, 12 May 2015 15:19:46 GMT Client-Warning: Internal response Status read failed: Unknown error at C:/Perl/site/lib/Net/HTTP/Methods +.pm line 276.

Replies are listed 'Best First'.
Re: Help with LWP & SSL 500 Internal response error
by boftx (Deacon) on May 12, 2015 at 15:58 UTC

    Have you checked the server log to see what the internal error is?

    You must always remember that the primary goal is to drain the swamp even when you are hip-deep in alligators.
      Unfortunately I don't have access to the server log. This is a internal company site that I do not manage - I am just trying to access specific pages of the site, but it requires an HTTPS connection.
        Error: 500 Status read failed: Unknown error Content-Type: text/plain Client-Date: Tue, 12 May 2015 15:19:46 GMT Client-Warning: Internal response

        This error was generated on the client side by LWP::UserAgent (see last line), not by the server. It seems the communication with the HTTPS server failed. Are you sure your LWP installation supports HTTPS? Try connecting to https://www.google.com/ or similar.

        Maybe the HTTPS server uses a self-signed certificate? (But that should give a different error message ...)

        Alexander

        --
        Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)
        A reply falls below the community's threshold of quality. You may see it by logging in.
Re: Help with LWP & SSL 500 Internal response error
by noxxi (Pilgrim) on May 12, 2015 at 20:15 UTC
    According to the log the SSL handshake itself succeeded:
    DEBUG: .../IO/Socket/SSL.pm:676: Net::SSLeay::connect -> 1
    DEBUG: .../IO/Socket/SSL.pm:731: ssl handshake done
    
    Which means the error must be at the HTTP layer. Check if the same URL is accessible with a browser, curl, wget ... - it can be that the server restricts access to specific user agents.
Re: Help with LWP & SSL 500 Internal response error
by Anonymous Monk on May 12, 2015 at 23:10 UTC
    You should upgrade all the modules (LWP / IO::Socket::SSL / Net...) and try again :)

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1126436]
Approved by ww
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others romping around the Monastery: (7)
As of 2020-09-24 18:11 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    If at first I donít succeed, I Ö










    Results (135 votes). Check out past polls.

    Notices?