Re: Using LWP (or some other module) to Dowload HTML with Cookie Session ID

An easy way to get a start is by converting your Curl command to a Perl program using (my) Curl to Perl converter:

#!perl
use strict;
use warnings;

use LWP::UserAgent;

my $ua = LWP::UserAgent->new( 'send_te' => '0' );
my $r  = HTTP::Request->new(
    'GET' => 'https://example.com/',
    [
        'Accept'     => '*/*',
        'User-Agent' => 'curl/7.55.1',
        'Cookie'     => 'session=536...035a'
    ],

);
my $res = $ua->request( $r, ':content_file' => 'HTML_output.txt' );



__END__

Created from curl command line
curl https://example.com --cookie "session=536...035a" -o "HTML_output
+.txt"
[download]

I'm not sure if your Firefox stores its cookies in a cookies.txt file still or if it stores them in an SQLite database nowadays.

Comment on Re: Using LWP (or some other module) to Dowload HTML with Cookie Session ID Select or Download Code

Replies are listed 'Best First'.
Re^2: Using LWP (or some other module) to Dowload HTML with Cookie Session ID by Leudwinus (Scribe) on Nov 28, 2022 at 23:50 UTC
Thank you, Corion for that example and link to your page which I have bookmarked! It looks like the content I need is then stored in `$res->{_content}`. I had to first install LWP::UserAgent and LWP::Protocol::https to get this to work though. I guess I need to get more familiar with the LWP, LWP::UserAgent and HTTP::Request modules. What is the difference between LWP::UserAgent and Mojo::UserAgent, the latter which was also suggested to me to try to solve this? Thanks again!	[reply] [d/l]
Re^3: Using LWP (or some other module) to Dowload HTML with Cookie Session ID by Corion (Patriarch) on Nov 29, 2022 at 08:17 UTC
No - please use `$res->decoded_content` instead of reaching into the HTTP::Response hash. The difference between LWP::UserAgent and Mojo::UserAgent is mostly that LWP::UserAgent doesn't allow for parallel requests. Mojo::UserAgent is a bit more complex due to allowing for parallelism, but it also offers some more convenience stuff like handling JSON replies directly.	[reply] [d/l]
Re^4: Using LWP (or some other module) to Dowload HTML with Cookie Session ID by Leudwinus (Scribe) on Nov 29, 2022 at 22:29 UTC
Ok, thank you again Corion! As you can clearly tell, I have no idea what I am doing! And thank you for clarifying the differences between LWP::UserAgent and Mojo::UserAgent.	[reply]


Just another Perl shrine
	PerlMonks