http://qs321.pair.com?node_id=1212998

AlwaysSurprised has asked for the wisdom of the Perl Monks concerning the following question:

My grabbing URLs to sniff out some data is working pretty much as I expect it to work, but just occasionally my script aborts.

Invariablly, it crashes out at.

my $dom = $ua->get($targetURL) -> result;

giving an error of "inactivity timeout" or "premature connection close". Now I'm not bothered what the cause is, but I would like a way of reacting to the error. I've hunted about but I can't see anything I can grab on to. What am I missing?

Replies are listed 'Best First'.
Re: Reacting to Mojo::UserAgent Errors
by choroba (Cardinal) on Apr 16, 2018 at 15:40 UTC
    Wrap the call to get into an eval, or Try::Tiny's try, or similar. You can even try several times before giving up.

    ($q=q:Sq=~/;[c](.)(.)/;chr(-||-|5+lengthSq)`"S|oS2"`map{chr |+ord
}map{substrSq`S_+|`|}3E|-|`7**2-3:)=~y+S|`+$1,++print+eval$q,q,a,

    [download]
[reply]
[d/l]
[select]
Re: Reacting to Mojo::UserAgent Errors
by marto (Cardinal) on Apr 16, 2018 at 15:45 UTC
[reply]
[d/l]

      Perhaps they have wised to me, but at 5 URL/s they don't seem to have taken offence. It would be nice if Mojo::UserAgent allowed the useragent details to be set, but that doesn't appear to be the case.

      I have read from the book of Mojolicious at the temple of CPAN and the code you cite but I can't get it to work (or not work). I haven't found specific details so I interpret the if tree to be "look for 200" (aka is_success is a 200 response code), if not look for a connection problem (aka is_error means the GET timesout for some reason).

      Specifically, if I run 

      use Mojo::UserAgent;

# Fine grained response handling (dies on connection errors)
my $ua  = Mojo::UserAgent->new;
my $res = $ua->get('mojoliciousness.org/perldoc')->result;
if    ($res->is_success)  { print $res->body }
elsif ($res->is_error)    { print $res->message }
elsif ($res->code == 301) { print $res->headers->location }
else                      { print 'Whatever...' }
print "Got to the end\n";

      [download]

      with the correct URL, it works correctly and spits out the perldoc page and the extra message. Nowif I misspell the URL, like I have above, I'll get a "Can't connect: No such host is known" message but it doesn't show the "Got to the end" message i.e. the script terminates at is_error and I can't recover and move on.

      Am I missing something here?

[reply]
[d/l]

        "It would be nice if Mojo::UserAgent allowed the useragent details to be set, but that doesn't appear to be the case."

        See transactor:

        # Change name of user agent
$ua->transactor->name('MyUA 1.0');

        [download]

        Here I used eval to catch errors. I'll check the situation you mention above at a sane hour.

[reply]
[d/l]
[select]
        Am I missing something here?

        Due to your misspelling, no IP address could be resolved.

        $ua->get('mojoliciousness.org/perldoc')

        returns an object of type Mojo::Transaction::HTTP which contains an attribute called  original_remote_address.
        Checking if this attribute has a defined value should solve your problem.

        use Mojo::UserAgent;

# Fine grained response handling (dies on connection errors)          
+                                                        
my $ua  = Mojo::UserAgent->new;
#my $res = $ua->get('mojoliciousness.org/perldoc')->result;           
+                                                        
my $res = $ua->get('mojoliciousness.org/perldoc');

if (defined ($res->original_remote_address))
{
    if    ($res->result->is_success)  { print $res->result->body }
    elsif ($res->result->is_error)    { print $res->result->message }
    elsif ($res->result->code == 301) { print $res->result->headers->l
+ocation }
    else                             { print 'Whatever...' }

    print "remote address: ".$res->original_remote_address."\n";
}

else { print "failed to resolve url ...\n"; }

print "Got to the end\n";

        [download]
[reply]
[d/l]
[select]

      Forgive me for I have sinned. I have resolved my problem by wrapping my call to the perl script in a DOS batch file.

      Now when the get fails and I'm dropped out of perl, I trap the %errorlevel% and restart the call to perl. I feel dirty, but it works.

[reply]
[d/l]
[select]

Back to Seekers of Perl Wisdom