Beefy Boxes and Bandwidth Generously Provided by pair Networks
more useful options
 
PerlMonks  

LWP::Simple - cookies?

by Anonymous Monk
on Sep 27, 2001 at 21:38 UTC ( [id://115179]=perlquestion: print w/replies, xml ) Need Help??

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Hello. I want to retrieve a whole bunch of pages from one site, the problem is that it requires a cookie, from a windows browser, u enter the username/passwd & proceed, never being asked again (until the cookie expires). How can I surf to the www page from a perl script and make it retrieve the page as though the cookie has been set? I'll have to set the cookie first time? save it somewhere? any particular modules I should look at?

Replies are listed 'Best First'.
Re: LWP::Simple - cookies?
by cacharbe (Curate) on Sep 27, 2001 at 21:59 UTC
    Yes, you should be looking at:
    It isn't actually that hard. Here is something I use with frequency lately.
    ##Assumes: use LWP::UserAgent; use HTTP::Request::Common qw(POST); use HTTP::Cookies; sub CreateSession{ ##Takes Ref to user agent object my ($ua) = @_; $$ua->cookie_jar(HTTP::Cookies->new(file => "<FILEPATH>", autosave => 1)); $$ua->proxy(http => '<PROXYINFO - OPTIONAL>'); $$ua->timeout(300); #Use some basic Authentication my $req = POST 'https://<SOME SITE>/login.cgi', [cdsid=>'<USERID>', pw=>'<PASSWORD>',back=>'<BACK>']; $req->proxy_authorization_basic('<PROXY USERID>, <PROXY PASS>); my $res = $$ua->request($req); unless ($res->is_success) { print "Login Failed: : ". $res->status_line . "\n"; } #get the next page $req = POST 'https://<NEXT PAGE>/main1.cgi'; $res = $$ua->request($req); unless ($res->is_success) { print "Main Page Failed: : ". $res->status_line . "\n"; } return 1; }

    C-.

Re: LWP::Simple - cookies?
by merlyn (Sage) on Sep 27, 2001 at 21:50 UTC

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://115179]
Approved by root
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others romping around the Monastery: (6)
As of 2024-04-23 18:51 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found