Unable to access restricted URL resource (GET) 302 redirect msg

asadzzz has asked for the wisdom of the Perl Monks concerning the following question:

Hi Folks, I'm using https://metacpan.org/pod/HTTP::Request::Generator to generate multiple GET resource which is password protected. So my plan is to first use POST followed up with GET request.


use strict;
use warnings;
use HTTP::Request::Generator 'generate_requests';
use LWP::UserAgent;
use LWP::Protocol::https;
use Data::Dumper;
use Test::More;

my $ua = 'LWP::UserAgent'->new(max_redirect => 3);

my $gen = generate_requests(
    method  => 'POST',
    host    => [ 'abc.ai' ],
 #pattern => 'https://abc.ai',
#    port => '443',
     scheme=> ['https'],
    path=>'/login',
    #url => '/login/',
    body_params => {
           user_id => ['dem'],
           user_password=> ['dem'],
       },

    headers => {
    "User-Agent" => 'Mozilla/5.0 (Windows NT 10.0; Win64; x64',
    "TE" => "application/json",
    "Cookie" => 'abc',
  },
    wrap    => sub {
        my ( $req ) = @_;
      #  $req->{headers}->{'TE'} = 'ASAD';

        return $req;

    },
    wrap => \&HTTP::Request::Generator::as_http_request,

);

while ( my $req = $gen->() ) {
    my $response = $ua->request( $req );
  print $response->protocol, ' ', $response->status_line, "\n";
#  print $req->headers->as_string, "\n";
print $response ->header('title',"\n");
print $req->headers->as_string, "\n";
#print $response ->header('title',"\n");
#print $response->content,"\n";
    # Do something with $response here?
if ($response->is_success) {
  #  print $response->decoded_content;
    print $response ->header('title',"\n");
}
else {
  print $response->status_line;
}
}
[download]

I'm getting 302 I want to know how can I marshal above code to perform GET request after POST, do I need to create another LWP object. Thanks.

Comment on Unable to access restricted URL resource (GET) 302 redirect msg - using HTTP::Request::Generator Download Code

Replies are listed 'Best First'.
Re: Unable to access restricted URL resource (GET) 302 redirect msg - using HTTP::Request::Generator by Corion (Patriarch) on Oct 10, 2020 at 18:51 UTC
HTTP::Request::Generator is for generating HTTP requests, for example for scanning multiple hosts for a set of URLs. It is not for executing the sequence of requests and certainly not for following requests. LWP::UserAgent can automatically/transparently follow 302 requests, but I don't know if it also transforms a POST request into a GET request after redirect. If not, you will have to do that yourself, but you will have to do that with any kind of HTTP request, no matter how you got it (manually, or from HTTP::Request::Generator). See the response_redirect handler for where you can implement the change from POST to GET on redirect. Update: If you want to have your Perl script act more like a browser, consider using WWW::Mechanize, which extends LWP::UserAgent to act more like a browser.	[reply]
Re^2: Unable to access restricted URL resource (GET) 302 redirect msg - using HTTP::Request::Generator by tobyink (Canon) on Oct 10, 2020 at 21:01 UTC
"LWP::UserAgent can automatically/transparently follow 302 requests, but I don't know if it also transforms a POST request into a GET request after redirect." For HTTP 302 and HTTP 303 redirects, it transforms non-GET/HEAD requests into GET requests upon redirection. toby döt ink	[reply]
Re: Unable to access restricted URL resource (GET) 302 redirect msg - using HTTP::Request::Generator by tobyink (Canon) on Oct 10, 2020 at 20:53 UTC
As per my comment on StackOverflow, this: `wrap => sub { my ( $req ) = @_; # $req->{headers}->{'TE'} = 'ASAD'; return $req; }, wrap => \&HTTP::Request::Generator::as_http_request,` [download] Does not work. The `wrap` option can only be given once. The second wrapper (`\&HTTP::Request::Generator::as_http_request`) overrides the first one (`sub {...}`) and the first one is completely being ignored so it doesn't need to be there. I'm not sure why you're using HTTP::Request::Generator at all. LWP::UserAgent (perhaps with a cookie jar) seems like it should do all you need. toby döt ink	[reply] [d/l] [select]
Re: Unable to access restricted URL resource (GET) 302 redirect msg - using HTTP::Request::Generator by perlfan (Vicar) on Oct 11, 2020 at 04:01 UTC
post format protip: HTTP::Request::Generator -> `[mod://HTTP::Request::Generator]`	[reply] [d/l]


No such thing as a small change
	PerlMonks

Unable to access restricted URL resource (GET) 302 redirect msg - using HTTP::Request::Generator