$ cat lwp-headers.pl
#!/usr/bin/perl

package MyLWP;
use base 'LWP::UserAgent';

sub progress {
    my($self, $status, $resp) = @_;
    if ($resp) {
        my @headers = $resp->header_field_names;
        print "response headers: @headers\n";
    }
}

package main;

my $ua = MyLWP->new;
my $url = shift || 'http://gmail.google.com';
my $rp = $ua->get($url);
print $rp->status_line, "\n";

$ ./lwp-headers.pl http://www.google.com
response headers: Cache-Control Connection Date Location Content-Lengt
+h Content-Type Client-Peer Client-Response-Num Set-Cookie Title X-Cac
+he X-Cache-Lookup
response headers: Cache-Control Connection Date Location Content-Lengt
+h Content-Type Client-Date Client-Peer Client-Response-Num Set-Cookie
+ Title X-Cache X-Cache-Lookup
response headers: Cache-Control Connection Date Content-Type Client-Pe
+er Client-Response-Num Set-Cookie Title X-Cache X-Cache-Lookup
response headers: Cache-Control Connection Date Content-Type Client-Pe
+er Client-Response-Num Set-Cookie Title X-Cache X-Cache-Lookup
response headers: Cache-Control Connection Date Content-Type Client-Pe
+er Client-Response-Num Set-Cookie Title X-Cache X-Cache-Lookup
response headers: Cache-Control Connection Date Content-Type Client-Pe
+er Client-Response-Num Set-Cookie Title X-Cache X-Cache-Lookup
response headers: Cache-Control Connection Date Content-Type Client-Pe
+er Client-Response-Num Set-Cookie Title X-Cache X-Cache-Lookup
response headers: Cache-Control Connection Date Content-Type Client-Pe
+er Client-Response-Num Set-Cookie Title X-Cache X-Cache-Lookup
response headers: Cache-Control Connection Date Content-Type Client-Da
+te Client-Peer Client-Response-Num Set-Cookie Title X-Cache X-Cache-L
+ookup
200 OK

$ ./lwp-headers.pl
response headers: Cache-Control Connection Date Pragma Location Conten
+t-Type Client-Peer Client-Response-Num Title X-Cache X-Cache-Lookup
response headers: Cache-Control Connection Date Pragma Location Conten
+t-Type Client-Date Client-Peer Client-Response-Num Title X-Cache X-Ca
+che-Lookup
response headers: Cache-Control Connection Date Pragma Location Conten
+t-Type Client-Peer Client-Response-Num Set-Cookie Title X-Cache X-Cac
+he-Lookup
response headers: Cache-Control Connection Date Pragma Location Conten
+t-Type Client-Date Client-Peer Client-Response-Num Set-Cookie Title X
+-Cache X-Cache-Lookup
response headers: Cache-Control Connection Date Pragma Location Conten
+t-Type Client-Peer Client-Response-Num Set-Cookie Title X-Cache X-Cac
+he-Lookup
response headers: Cache-Control Connection Date Pragma Location Conten
+t-Type Client-Peer Client-Response-Num Set-Cookie Title X-Cache X-Cac
+he-Lookup
response headers: Cache-Control Connection Date Pragma Location Conten
+t-Type Client-Date Client-Peer Client-Response-Num Set-Cookie Title X
+-Cache X-Cache-Lookup
response headers: Cache-Control Connection Date Pragma Server Content-
+Length Content-Type Client-Peer Client-Response-Num Client-SSL-Cert-I
+ssuer Client-SSL-Cert-Subject Client-SSL-Cipher Client-SSL-Warning Se
+t-Cookie
.....
response headers: Cache-Control Connection Date Pragma Server Content-
+Length Content-Type Client-Date Client-Peer Client-Response-Num Clien
+t-SSL-Cert-Issuer Client-SSL-Cert-Subject Client-SSL-Cipher Client-SS
+L-Warning Set-Cookie Title
200 OK
[download]

I get the GET line. (Keep in mind that WWW::Mechanize is a subclass of LWP::UserAgent.)

use LWP::UserAgent qw( );
my $ua = LWP::UserAgent->new();
my $response = $ua->get('http://www.google.com/');
print($response->request()->as_string(), "\n");
[download]

GET http://www.google.ca/
User-Agent: libwww-perl/5.805
[download]

Update: hum, is that really what it sends? Is that even valid?
Update: yeah, it works if I send that raw. I'm not used to HTTP/0.9

GET http://www.google.com/
User-Agent: libwww-perl/5.805
[download]

200 OK
[download]

I use LWP::UserAgent version 2.036 from distribution libwww-perl-5.808. The method progress was added since libwww-perl-5.806. Upgrade your version and try again. I wonder however my code runs at all with libwww-perl-5.805 (that included LWP::UA 2.033 which didn't provide method progress yet).

Update: How stupid of me. My code runs with LWP::UA 2.033 because the method progress isn't called at all

---

Open source softwares? Share and enjoy. Make profit from them if you can. Yet, share and enjoy!

Thanks.
I upgraded and succeeded to make this work, unfortunately this method does not bring the result I need using headers_as_string.
I will give you an example:
http://www.clipfish.de/player.php?videoid=MzkyOTN8OTA%3D
has a video inside.
There is no direct link to this video in the HTML source, the location will only be sent within the headers.
Using LWP::UserAgent with method progress does not bring this URL which is as follows (from LiveHTTPheaders-log, snippet from header):


http://pg1.clipfish.de/media/96/4efec90e3f12d631b4f5b490db152596.flv
GET /media/96/4efec90e3f12d631b4f5b490db152596.flv HTTP/1.1
Host: pg1.clipfish.de
...
[download]

Would it be possible to get the "host"-information and URL of the video from the headers using LWP or do I have to use another module or method?