Keep It Simple, Stupid | |
PerlMonks |
comment on |
( [id://3333]=superdoc: print w/replies, xml ) | Need Help?? |
Yep, robots.txt / user-agent exclusion is the problem $mech->agent_alias( 'Windows IE 6' ); works with wikipedia but for some reason not gnu.org $mech->agent_alias('Linux Mozilla'); works for both. I guess if wikipedia doesn't want mech scraping, I won't do it. Thanks for your help planetscape, In reply to Re^2: Test::WWW::Mechanize page_links_ok fails on wikipedia entry external links
by mandog
|
|