Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer
 
PerlMonks  

Re: Lexis-Nexis Academic spider

by BigRare (Pilgrim)
on Apr 26, 2005 at 19:58 UTC ( [id://451748]=note: print w/replies, xml ) Need Help??


in reply to Lexis-Nexis Academic spider

The easiest way to get this to work is to use the following modules: HTTP::Proxy, and HTTP::Recorder

Check the CPAN documentation on them.

HTTP::Proxy sets up a proxy server, which you can set your browser to use.
HTTP::Recorder sits in the proxy and records your actions and generates a script for WWW::Mechanize to use.

Example code using these modules:
use HTTP::Proxy; use HTTP::Recorder; my $file = /path/to/file/to/log; my $proxy = HTTP::Proxy->new(port => 3128); # create a new HTTP::Recorder object my $agent = new HTTP::Recorder; # set the log file (optional) $agent->file($file); # set HTTP::Recorder as the agent for the proxy $proxy->agent( $agent ); # start the proxy $proxy->start();

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://451748]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others drinking their drinks and smoking their pipes about the Monastery: (6)
As of 2024-04-19 16:25 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found