Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister
 
PerlMonks  

Re: Connection through a proxy

by TwistedGreen (Sexton)
on Apr 26, 2004 at 12:52 UTC ( [id://348159]=note: print w/replies, xml ) Need Help??


in reply to Connection through a proxy

Will you need to use authentication to get through the proxy? If so, you may run into some difficulties using the usual modules. I have had to use the CPAN module Net::HTTPTunnel instead of any of the LWP::UserAgent modules to do this.

An example:
use Net::HTTPTunnel; use HTTP::Request; my $proxy_host = "192.168.1.1"; my $proxy_port = "83"; my $proxy_user = "username"; my $proxy_pass = "password"; my $ht = Net::HTTPTunnel->new( 'proxy-host' => $proxy_host, 'proxy-port' => $proxy_port, 'remote-host' => 'www.perlmonks.org', 'remote-port' => '80', 'proxy-user' => $proxy_user, 'proxy-pass' => $proxy_pass, 'timeout' => 20); $ht or die "request timed out"; # generate an HTTP request my $req = HTTP::Request->new(GET => "http://www.perlmonks.org/index.pl +?node_id=317556"); print $ht $req->as_string; # print the request to t +he socket my $response; while (<$ht>) {$response .= $_;} # read response (the HTML +web page) from the socket close $ht; # close the socket
The page is now stored in $response, to do with as you please.

Replies are listed 'Best First'.
Re: Re: Connection through a proxy
by mce (Curate) on Apr 26, 2004 at 13:35 UTC
    Hi,

    What is the difference with this module, and the Basic authentication that comes with LWP?

    See the credentials function in LWP::UserAgent.


    ---------------------------
    Dr. Mark Ceulemans
    Senior Consultant
    BMC, Belgium
      From what I gathered when I was working on this problem, the main difference is in the level of abstraction of the modules. LWP::UserAgent will not give you a direct socket, while Net::HTTPTunnel will return a TCP socket which is tunneled transparently through the proxy you specify. You can then write what you will to the server, such as the HTTP GET request as shown in the example.

      But if you just want to retrieve a page like I did, the critical difference was that Net::HTTPTunnel worked swimmingly with my proxy while LWP::UserAgent seemed unable to authenticate properly. Perhaps a more experienced monk can explain the underlying mechanics.
Re: Re: Connection through a proxy
by mpolo (Chaplain) on Apr 26, 2004 at 15:31 UTC

    The newer LWP::UserAgent modules accept an http_proxy in the form http://username:password@server:port/ . You can, for instance, set $ENV{HTTP_PROXY} to a string of that form and then use LWP::Simple, etc.

    If you have a Microsoft proxy server that you are going through, you will have to indicate your domain as well, so the format will be http://domain\username:password@server:port/ (careful with quoting of the backslash).

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://348159]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others about the Monastery: (6)
As of 2024-03-28 11:26 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found