Beefy Boxes and Bandwidth Generously Provided by pair Networks
Syntactic Confectionery Delight

Re: To mechanize WWW::Mechanize: a scraping language?

by Jenda (Abbot)
on Aug 25, 2004 at 22:22 UTC ( #385858=note: print w/replies, xml ) Need Help??

in reply to To mechanize WWW::Mechanize: a scraping language?

Why create a new language? Let's use Perl!

You may create a module that will work as a wrapper around WWW::Mechanize. It will create the object and do all the other preliminary stuff, it will provide you with helper functions and throught AUTOLOAD it will allow you to call the $agent's methods as functions. So you end up with something like

use WWW:Mechanize::Simple; get url => "", output="login.html"; submit form_name => "login", user=>"", passwd=>"", button=>"login", output=>"index.htm; get url=>" Nodes", output=>"newest.html";

Aint that simple enough? And it should not be that hard to implement, you just need to initialize the object, create a few functions and use AUTOLOAD to pass the others to the $agent as methods. And that'll be it.

Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live.
   -- Rick Osborne

Replies are listed 'Best First'.
Re^2: To mechanize WWW::Mechanize: a scraping language?
by dragonchild (Archbishop) on Aug 26, 2004 at 01:22 UTC
    This is exactly what I did when our DBA came to me and asked me for a load tester, to determine which parameter changes were best, given a specified set of SQL statements and the order they should run in.

    We are the carpenters and bricklayers of the Information Age.

    Then there are Damian modules.... *sigh* ... that's not about being less-lazy -- that's about being on some really good drugs -- you know, there is no spoon. - flyingmoose

    I shouldn't have to say this, but any code, unless otherwise stated, is untested

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://385858]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others examining the Monastery: (5)
As of 2023-03-26 21:56 GMT
Find Nodes?
    Voting Booth?
    Which type of climate do you prefer to live in?

    Results (63 votes). Check out past polls.