Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses
 
PerlMonks  

WWW::Mechanize - error on geting non-existing page

by Anonymous Monk
on Nov 05, 2011 at 16:15 UTC ( [id://936163]=perlquestion: print w/replies, xml ) Need Help??

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Hi all
I try to get a few URLs with WWW::Mechanize. This works fine as long as the URL exists. If I hit an non-existing URL the script dies.
How can I get the script to go on with the next URL after the non-existing one?
Thanks for your help
  • Comment on WWW::Mechanize - error on geting non-existing page

Replies are listed 'Best First'.
Re: WWW::Mechanize - error on geting non-existing page
by Limbic~Region (Chancellor) on Nov 05, 2011 at 18:20 UTC
    Anonymous Monk,
    With current versions of WWW::Mechanize, autocheck is enabled causing $mech to die in these situations. Here is how you might want to handle it:
    eval { $mech->get($url); }; if ($@) { if ($mech->status == 404) { # handle page not found } else { die "Mech failed to fetch '$url': $@"; } }

    Cheers - L~R

      You can also just set onerror => undef which is easier and less error prone than eval/try style code.

        Your Mother,
        I honestly can't think of any reason this might be true. Do you also establish a DBI connection without RaiseError set to true? It is akin to not checking the return code of open. If something goes wrong, I want to know about it very loudly instead of blindly assuming everything is ok.

        Can you give me an example of where setting it to false is better? I assure you I haven't adopted this strategy as a cargo cult practice and am genuinely interested in hearing your views on how this is better.

        Cheers - L~R

      You also might want to switch to WWW::Mechanize::GZip, since some versions of the original WWW::Mechanize seem to announce to handle HTTP compression but don't in reality. This might result in "unparseable" results returned from the webserver.
      Don't use '#ff0000':
      use Acme::AutoColor; my $redcolor = RED();
      All colors subject to change without notice.

        You also might want to switch to WWW::Mechanize::GZip, since some versions of the original WWW::Mechanize seem to announce to handle HTTP compression but don't in reality. This might result in "unparseable" results returned from the webserver.

        I doubt this ever happens :) but if it did, the solution is to always install the latest WWW::Mechanize , LWP, Compress::Zlib

        If you were going to install WWW::Mechanize::GZip, you might install the latest WWW::Mechanize , LWP, Compress::Zlib instead

      Thanks a bunch, problem solved!
Re: WWW::Mechanize - error on geting non-existing page
by moritz (Cardinal) on Nov 05, 2011 at 16:51 UTC
Re: WWW::Mechanize - error on geting non-existing page
by bichonfrise74 (Vicar) on Nov 05, 2011 at 16:47 UTC
    Can you show us what you have done so far?

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://936163]
Approved by moritz
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others exploiting the Monastery: (3)
As of 2024-04-20 02:40 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found