How to remove HTTP Keep-Alive 300 header from LWP::UserAgent request

Veltro has asked for the wisdom of the Perl Monks concerning the following question:

Hello, I'd like to remove the Keep-Alive 300 header from the request but I can't figure out how.

I am instantiating the LWP::UserAgent with the intention to use HTTP 1.1 and keeping the connection alive as follows:

my $ua = LWP::UserAgent->new(keep_alive => 1, send_te => 0) ;

send_te => 0 removes the TE header and the header information that I want to send contains tags like:

my @ns_headers = (
    'ACCEPT' => '..',
    'ACCEPT_ENCODING' => '...', 
    'ACCEPT-LANGUAGE => '..',
    'UPGRADE_INSECURE_REQUESTS' => '1',
    'USER-AGENT' => '...',
) ;
[download]

I have tried several methods creating the request but each and one of them sends the Keep-Alive 300 header.

my $response = $ua->get( $link, @ns_headers ) ;

my $response = $ua->request(GET $link) ; # This does not use @ns_headers

my $getReq = HTTP::Request->new( GET => $link, HTTP::Headers->new( @ns
+_headers ) ) ;
my $response = $ua->request( $getReq ) ;
[download]

Tryig to get rid of it by specifying this in the @ns_headers does not work:

'KEEP-ALIVE' => undef,

Anyone knows how to do this? Thanks

Comment on How to remove HTTP Keep-Alive 300 header from LWP::UserAgent request Select or Download Code

Replies are listed 'Best First'.
Re: How to remove HTTP Keep-Alive 300 header from LWP::UserAgent request by jeffenstein (Hermit) on Apr 26, 2018 at 09:55 UTC
Since you've asked for keep-alives in the constructor: `my $ua = LWP::UserAgent->new(keep_alive => 1, send_te => 0) ;` then LWP will add both the `Connection: keep-alive` and the `Keep-Alive: 300`, as the standard specifies According to the LWP::UserAgent docs, the argument to `keep_alive` is the maximum number of connections to cache, so if you do need keep-alives, then 1 is probably not the right number.	[reply] [d/l] [select]
Re^2: How to remove HTTP Keep-Alive 300 header from LWP::UserAgent request by Veltro (Hermit) on Apr 26, 2018 at 14:34 UTC
Sorry but that does not sound right to me. In think that the implementation is incorrect. There is no standard that says the Keep-Alive: 300 header is mandatory. E.g. from the 5 Internet Browsers I have looked at, none of them sends this header. https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Keep-Alive quote: The Keep-Alive general header allows the sender to hint about how the connection and may be used to set a timeout and a maximum amount of requests. https://tools.ietf.org/html/rfc2068#section-19.7.1.1 quote: When the Keep-Alive connection-token has been transmitted with a request or a response, a Keep-Alive header field MAY also be included Another thing is that to me this header looks parameterized in form in both request and response (according to each of the specifications mentioned, including the one you have provided). Thus not of the form 'Header : value' Regards the keep_alive argument I think you are mixing it up. The keep_alive argument is in the 'new' of the LWP::UserAgent and is not related to the "Keep-Alive: 300" header. Quote from LWP::UserAgent: "The keep_alive value is passed on as the total_capacity for the connection cache".	[reply]
Re^3: How to remove HTTP Keep-Alive 300 header from LWP::UserAgent request by jeffenstein (Hermit) on Apr 26, 2018 at 15:41 UTC
The actual code that adds the header is in Net::HTTP::Methods, at line 180 in the current version. `if ($self->keep_alive) { if ($peer_ver eq "1.0") { # from looking at Netscape's headers push(@h2, "Keep-Alive: 300"); unshift(@connection, "Keep-Alive"); } }` [download] So, I would guess that Net::HTTP believes your connection is http/1.0, and is therefore sending the http/1.0 header. If it's not, maybe you can make a bug report against the module?	[reply] [d/l]
Re^4: How to remove HTTP Keep-Alive 300 header from LWP::UserAgent request by Veltro (Hermit) on May 01, 2018 at 22:04 UTC
Re^4: How to remove HTTP Keep-Alive 300 header from LWP::UserAgent request by Anonymous Monk on Apr 27, 2018 at 02:28 UTC
Re^3: How to remove HTTP Keep-Alive 300 header from LWP::UserAgent request by Anonymous Monk on Apr 27, 2018 at 02:14 UTC
E.g. from the 5 Internet Browsers I have looked at, none of them sends this header. Are any of these browsers from 1995? First came keep-alive, then came the RFC, thats history for you	[reply]
Re: How to remove HTTP Keep-Alive 300 header from LWP::UserAgent request by Anonymous Monk on Apr 27, 2018 at 02:18 UTC
Hi Well, since you responded to jeffenstein, here you go Veltro LWP header -> Re: LWP is there any way to get "real" outgoing headers? #!/usr/bin/perl -- use strict; use warnings; use LWP; use Data::Dump(); sub LWP::Protocol::http::SocketMethods::format_request { package LWP::Protocol::http::SocketMethods; my( $socket, $method, $fullpath, @h ) = @_; #~ my $req_buf = $socket->Net::HTTP::Methods::format_request($meth +od, $fullpath, @h, 'Keep-Alive',666); ## fail, adds another header my $req_buf = $socket->Net::HTTP::Methods::format_request($method, + $fullpath, @h ); $req_buf =~ s{Keep-Alive: 300}{Keep-Alive: 30}g; ## win, removes e +xisting header Data::Dump::dd( $req_buf ); return $req_buf; } LWP::UserAgent->new( keep_alive=>1 )->get( q{http://127.0.0.1:80/} )-> +dump; __END__ "GET / HTTP/1.1\r\nTE: deflate,gzip;q=0.3\r\nKeep-Alive: 30\r\nConnect +ion: Keep-Alive, TE\r\nHost: 127.0.0.1:80\r\nUser- Agent: libwww-perl/6.15\r\n\r\n" HTTP/0.9 200 Assumed OK Client-Date: Tue, 24 Apr 2018 23:25:20 GMT Client-Peer: 127.0.0.1:80 Client-Response-Num: 1 Echo: GET / HTTP/1.1\r Echo: TE: deflate,gzip;q=0.3\r Echo: Keep-Alive: 30\r Echo: Connection: Keep-Alive, TE\r Echo: Host: 127.0.0.1:80\r Echo: User-Agent: libwww-perl/6.15\r Echo: \r\n [download]	[reply] [d/l]
Re^2: How to remove HTTP Keep-Alive 300 header from LWP::UserAgent request by Veltro (Hermit) on May 01, 2018 at 22:10 UTC
Thanks for showing how to override format_request. Very useful for introspection. I have also posted my reply to jeffenstein with different solution here for more details	[reply]
Re^3: How to remove HTTP Keep-Alive 300 header from LWP::UserAgent request by Anonymous Monk on May 02, 2018 at 07:01 UTC
Neat, thanks. I hope I can count on your vote, and I hope you enjoy the pastrami on rye.	[reply]
Re^4: How to remove HTTP Keep-Alive 300 header from LWP::UserAgent request by Veltro (Hermit) on May 02, 2018 at 07:46 UTC
Re: How to remove HTTP Keep-Alive 300 header from LWP::UserAgent request by Anonymous Monk on Apr 24, 2018 at 21:56 UTC
Why? You tell LWP you want keep_alive but then you dont want the header? Those goals are contradictory	[reply]
Re^2: How to remove HTTP Keep-Alive 300 header from LWP::UserAgent request by Veltro (Hermit) on Apr 24, 2018 at 22:18 UTC
I'll clarify. There are two different things: 1. Connection: keep-alive 2. Keep-Alive 300 I don't want 2	[reply]
Re^3: How to remove HTTP Keep-Alive 300 header from LWP::UserAgent request by Anonymous Monk on Apr 24, 2018 at 23:23 UTC
I'll clarify. There are two different things: 1. Connection: keep-alive 2. Keep-Alive 300 I don't want 2 :) That doesn't explain why "Keep-Alive: 300" need to be removed. Maybe you can teach LWP something about HTTP. So please explain why, thats my price for showing you how.	[reply]
Re^4: How to remove HTTP Keep-Alive 300 header from LWP::UserAgent request by karlgoethebier (Abbot) on Apr 25, 2018 at 08:49 UTC
Re^5: How to remove HTTP Keep-Alive 300 header from LWP::UserAgent request by Anonymous Monk on Apr 25, 2018 at 22:50 UTC
Some notes below your chosen depth have not been shown here

Back to Seekers of Perl Wisdom