Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister
 
PerlMonks  

Reacting to Mojo::UserAgent Errors

by AlwaysSurprised (Novice)
on Apr 16, 2018 at 15:32 UTC ( #1212998=perlquestion: print w/replies, xml ) Need Help??

AlwaysSurprised has asked for the wisdom of the Perl Monks concerning the following question:

My grabbing URLs to sniff out some data is working pretty much as I expect it to work, but just occasionally my script aborts.

Invariablly, it crashes out at.

my $dom = $ua->get($targetURL) -> result;

giving an error of "inactivity timeout" or "premature connection close". Now I'm not bothered what the cause is, but I would like a way of reacting to the error. I've hunted about but I can't see anything I can grab on to. What am I missing?

Replies are listed 'Best First'.
Re: Reacting to Mojo::UserAgent Errors
by choroba (Archbishop) on Apr 16, 2018 at 15:40 UTC
    Wrap the call to get into an eval, or Try::Tiny's try, or similar. You can even try several times before giving up.

    ($q=q:Sq=~/;[c](.)(.)/;chr(-||-|5+lengthSq)`"S|oS2"`map{chr |+ord }map{substrSq`S_+|`|}3E|-|`7**2-3:)=~y+S|`+$1,++print+eval$q,q,a,
Re: Reacting to Mojo::UserAgent Errors
by marto (Cardinal) on Apr 16, 2018 at 15:45 UTC

      Perhaps they have wised to me, but at 5 URL/s they don't seem to have taken offence. It would be nice if Mojo::UserAgent allowed the useragent details to be set, but that doesn't appear to be the case.

      I have read from the book of Mojolicious at the temple of CPAN and the code you cite but I can't get it to work (or not work). I haven't found specific details so I interpret the if tree to be "look for 200" (aka is_success is a 200 response code), if not look for a connection problem (aka is_error means the GET timesout for some reason).

      Specifically, if I run

      use Mojo::UserAgent; # Fine grained response handling (dies on connection errors) my $ua = Mojo::UserAgent->new; my $res = $ua->get('mojoliciousness.org/perldoc')->result; if ($res->is_success) { print $res->body } elsif ($res->is_error) { print $res->message } elsif ($res->code == 301) { print $res->headers->location } else { print 'Whatever...' } print "Got to the end\n";

      with the correct URL, it works correctly and spits out the perldoc page and the extra message. Nowif I misspell the URL, like I have above, I'll get a "Can't connect: No such host is known" message but it doesn't show the "Got to the end" message i.e. the script terminates at is_error and I can't recover and move on.

      Am I missing something here?

        "It would be nice if Mojo::UserAgent allowed the useragent details to be set, but that doesn't appear to be the case."

        See transactor:

        # Change name of user agent $ua->transactor->name('MyUA 1.0');

        Here I used eval to catch errors. I'll check the situation you mention above at a sane hour.

        Am I missing something here?

        Due to your misspelling, no IP address could be resolved.

        $ua->get('mojoliciousness.org/perldoc')

        returns an object of type Mojo::Transaction::HTTP which contains an attribute called  original_remote_address.
        Checking if this attribute has a defined value should solve your problem.

        use Mojo::UserAgent; # Fine grained response handling (dies on connection errors) + my $ua = Mojo::UserAgent->new; #my $res = $ua->get('mojoliciousness.org/perldoc')->result; + my $res = $ua->get('mojoliciousness.org/perldoc'); if (defined ($res->original_remote_address)) { if ($res->result->is_success) { print $res->result->body } elsif ($res->result->is_error) { print $res->result->message } elsif ($res->result->code == 301) { print $res->result->headers->l +ocation } else { print 'Whatever...' } print "remote address: ".$res->original_remote_address."\n"; } else { print "failed to resolve url ...\n"; } print "Got to the end\n";

      Forgive me for I have sinned. I have resolved my problem by wrapping my call to the perl script in a DOS batch file.

      Now when the get fails and I'm dropped out of perl, I trap the %errorlevel% and restart the call to perl. I feel dirty, but it works.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1212998]
Approved by marto
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others imbibing at the Monastery: (3)
As of 2020-10-25 06:48 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    My favourite web site is:












    Results (249 votes). Check out past polls.

    Notices?