Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid
 
PerlMonks  

Re: WWW::Mechanize follow_link not working

by ig (Vicar)
on Oct 13, 2013 at 04:39 UTC ( #1058035=note: print w/replies, xml ) Need Help??


in reply to WWW::Mechanize follow_link not working

The code you posted is not complete and has syntactic errors, making it difficult to be sure what your problem might have been.

In the following code, I have fixed a few obvious errors to make it compile and run. Otherwise I replaced 'Next' with 'Avanti' as, on the pages I got back (Google may return different content to you) the button at the bottom of the page, to proceed to the next page of results, is labeled Avanti. Perhaps this working example will help you get your code working as you wish.

use strict; use warnings; use Data::Dumper::Concise; use WWW::Mechanize; my $mech = WWW::Mechanize->new(); my $title = "Raumchemie der festen Stoffe"; $mech->get("http://scholar.google.it/scholar?q=" . $title ); unless($mech->success()) { die $mech->status(); } my $response = $mech->response(); my $content = $response->decoded_content(); print Dumper($content); my $link_result = $mech->follow_link( url_regex => qr/cites/i, n => 1 +); unless($link_result) { die "link not found"; } my $result = $mech->content; my $indi = $mech->uri(); #my $rest = $out->scrape( $result, $indi ); for my $i (2..5) { print "page : ".$i."\n"; $mech->follow_link( text_regex => qr/Avanti$/) or die("finished on page : ".$i."\n"); my $result = $mech->content; my $indi = $mech->uri(); print $indi->as_string() . "\n"; sleep(5); }

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1058035]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others examining the Monastery: (4)
As of 2020-09-27 01:50 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    If at first I donít succeed, I Ö










    Results (142 votes). Check out past polls.

    Notices?