Beefy Boxes and Bandwidth Generously Provided by pair Networks
laziness, impatience, and hubris
 
PerlMonks  

HTTP::Request::Common Retry Option

by Anonymous Monk
on Sep 17, 2010 at 23:06 UTC ( #860573=perlquestion: print w/replies, xml ) Need Help??

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

I would like to be able to have the option to retry when I am running the code below. Is there an option in HTTP::Request::Common to do that? Also, is there a way to adjust the response time of the module so that it will wait for example, 5 seconds, before giving up?
#!/usr/bin/perl use warnings; use strict; use HTTP::Request::Common; use LWP::UserAgent; my $ua = LWP::UserAgent->new( cookie_jar => '' ); my $request = HTTP::Request->new( 'GET', 'http://www.google.com' ); my $result = $ua->request( $request ); print $result->status_line;

Replies are listed 'Best First'.
Re: HTTP::Request::Common Retry Option
by Your Mother (Archbishop) on Sep 17, 2010 at 23:16 UTC

    I'm not sure your question makes any sense. What exactly is the problem you are having? What is the proof/output it is a problem? The agent will wait for the server to respond for $ua->timeout($seconds) and I think the default is three minutes.

    It's against Google's ToS to do automated hits against it without a developer key/agreement and they will block agents that seem to be violating that.

Re: HTTP::Request::Common Retry Option
by Khen1950fx (Canon) on Sep 18, 2010 at 00:42 UTC
    You can control the number of retries by using LWP::UserAgent::Determined. I set the timing string at 5 seconds, 10, and 20. I also added the codes that I wanted it to retry.
    #!/usr/bin/perl use strict; use warnings; use LWP::UserAgent::Determined; my $url = 'http://www.google.com'; my $browser = LWP::UserAgent::Determined->new( cookie_jar => '' ); my $timing_string = $browser->timing("5,10,20"); my $http_codes_hr = $browser->codes_to_determinate(); $http_codes_hr->{408} = 1; $http_codes_hr->{500} = 1; $http_codes_hr->{567} = 1; $browser->before_determined_callback( sub { print "Trying ", $_[4][0]->uri, " at ", scalar(localtime), "...\n"; }); $browser->after_determined_callback( sub { print "Just tried ", $_[4][0]->uri, " at ", scalar(localtime), ".\n"; }); my $response = $browser->get( $url ); print "That gave: ",$response->status_line, "\n";
      I was just about to ask a question along these lines. Good thing somebody beat me to it.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://860573]
Approved by ahmad
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chanting in the Monastery: (3)
As of 2022-12-03 01:28 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Notices?