Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister
 
PerlMonks  

LWP::UserAgent Get timing out after few request

by sannag (Sexton)
on Aug 17, 2017 at 14:58 UTC ( [id://1197564]=perlquestion: print w/replies, xml ) Need Help??

sannag has asked for the wisdom of the Perl Monks concerning the following question:

Update

**************

I don't the underlying cause. I fixed the issue on my program by using REST::Client instead of LWP::UserAgent....details of the code are in one of my reply below

***************

My Get request is timing out after processing 20 or 30 request...I have over 500 records to process. I did confirm that there is no limit on number of GET requested placed to API. I also tried using chrom extension postman to repeatably place GET request successfully. Yet when I place GET request though my perl program it failed after processing few requests. Any help is greatly appreciated

error says following:

500 can't connect to api.xxx.com:443. (A connection attempt failed because the connected party did not properly respond after a periold of time, or established connection failed because connected host has failed to respond

LWP::Protoclo::https::Socket: connect: a connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond at C:/Dwimperl/perl/site/LWP/Protocol/http.pm"

my $ua = LWP::UserAgent->new( ssl_opts => { verify_hostname => 0 }, ); $ua->timeout('1000'); sub getUserInfo { my($userId) = @_; my $partnerId = 'B53765B23456678C1' my $userURI = 'https://xxx.com/api/m1/request/'. $partnerId. '/users/ +'. $userId; print $userURI, "\n"; #sleep (3); my $userResponse = $ua -> request(GET($userURI, Authorization => $aut +hHeaders)); # I am getting a token which is passed in the header. if ($userResponse->is_success) { my $userRecord = decode_json($userResponse->content); print $userResponse->status_line, "\n"; return ($userRecord); } else { displayMsg ( "No user Records to process...................."); print $userResponse->status_line, "\n"; print $userResponse->decoded_content, "\n"; error ("Error: ", $userResponse->status_line, " " , $userResponse- +>decoded_content); exit 1; } }

Replies are listed 'Best First'.
Re: LWP::UserAgent Get timing out after few request
by Your Mother (Archbishop) on Aug 17, 2017 at 15:33 UTC

    Without being able to test or read the API docs I would suggest uncommenting your sleep and raising it from 3 to 5 or even 10. Also, setting $ua->timeout(120) or something in case it's just the service bogging down under any load Missed that you already had a much higher one. Might not be the case but it is possible to configure unlimited GETs while restricting the frequency or amount of data transferred.

      Thanks but had not luck I increased the sleep time to 10 and 15 made no difference. Also played around with timeout (both increasing and decreasing seem to make no difference)

        Is this a public API that any of us could play with or something closed or paid?

Re: LWP::UserAgent Get timing out after few request
by Mr. Muskrat (Canon) on Aug 17, 2017 at 15:33 UTC

    You specified a one second timeout. Have you tried increasing that?

    Update: Ignore this. I had a brain fart.

      Timmeout is in seconds, not milliseconds so it's a thousand.

        D'oh! Thanks for pointing out my blunder.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://1197564]
Approved by davies
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others pondering the Monastery: (5)
As of 2024-04-19 23:34 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found