Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses

Pass through perl http proxy server (no code, just advice)

by tachyon (Chancellor)
on Feb 28, 2003 at 21:38 UTC ( #239558=perlquestion: print w/replies, xml ) Need Help??

tachyon has asked for the wisdom of the Perl Monks concerning the following question:

I have a need to implement an http filter ie Net Nanny/Surf Control style to protect innocent students from the evils of the internet at large. OK so I want to write a proxy server to accept requests, validate the URL and then do pass through/deny. I know how to do all of this with sockets, fork, redirects and a DBMS. The question is boring. Although I really love reinventing wheels I wish to learn from those who have gone before so any suggestions of what is currently available for me to R&D* would be appreciated.



* R&D == Rip off && Duplicate. Open source == R&D && include attribution && share


  • Comment on Pass through perl http proxy server (no code, just advice)

Replies are listed 'Best First'.
•Re: Pass through perl http proxy server (no code, just advice)
by merlyn (Sage) on Feb 28, 2003 at 22:15 UTC

      I just love getting you to say it! Kinda like Pavlov's dog and ringing that bell. Of course I have to grant it to you, been there done that, sometimes done it twice! Might make a good slogan for stonehenge....




Re: Pass through perl http proxy server (no code, just advice)
by BooK (Curate) on Mar 01, 2003 at 08:51 UTC

    Well, uh, I'm just writing a module that's called HTTP::Proxy. ;-)

    The goal is to be able to pass every request and response though a set of configurable Perl "filters".

    Here's an example script that creates a anonymizing proxy, just like in merlyn's column (his script has been one of my inspirations, along with two others scritps by Abigail and Rafael Garcia-Suarez):

    #!/usr/bin/perl -w use HTTP::Proxy qw( :log ); use strict; # a very simple anonymizing proxy my $proxy = HTTP::Proxy->new; $proxy->logmask( shift || NONE ); # log configuration # the anonymising filter $proxy->push_headers_filter( mime => undef, # apply this on any type of content request => sub { $_[0]->remove_header(qw( User-Agent From Referer Cookie )); }, response => sub { $_[0]->remove_header(qw( Set-Cookie )),; }, ); $proxy->start;

    (This code works with the development version.)

    The proxy is not fully functional yet, though. One of my goal is to create a set of filters that would log the whole session, so that yet another script could (automatically) create a web robot from the session transcript.

    If you are interested, there is a web site (older versions and CVS snapshot) and a mailing-list at I am interested in help, and comments on the API (which is subject to change).

Re: Pass through perl http proxy server (no code, just advice)
by IlyaM (Parson) on Mar 01, 2003 at 13:06 UTC
    In my attempt to learn POE I wrote this simple single-process proxy:
    #!/usr/bin/perl use warnings; use strict; use POE qw(Component::Server::TCP Component::Client::HTTP Filter::HTTPD Filter::Stream); POE::Component::Client::HTTP->spawn(Alias => 'user_agent', Streaming => 1024); POE::Component::Server::TCP->new ( Alias => 'web_server', Port => 8000, ClientFilter => 'POE::Filter::HTTPD', ClientInput => \&client_input, InlineStates => { proxy_response => \&proxy_response } ); $poe_kernel->run(); sub client_input { my ($kernel, $heap, $request) = @_[KERNEL, HEAP, ARG0]; # if request is actually response it just means that we've got an # error if($request->isa('HTTP::Response')) { $heap->{client}->put($request); $kernel->yield('shutdown'); return; } $request->header(Connection => 'close'); $kernel->post(user_agent => request => proxy_response => $request) +; } sub proxy_response { my ($kernel, $heap, $request_packet, $response_packet) = @_[KERNEL, HEAP, ARG0, ARG1]; my $request = $request_packet->[0]; my ($response, $data) = @$response_packet; unless($heap->{got_headers}) { $response->header(Connection => 'close'); $heap->{client}->put($response); $heap->{got_headers}++; } if(defined $data) { $heap->{client}->set_output_filter(POE::Filter::Stream->new()) +; $heap->{client}->put($data); } else { $kernel->yield("shutdown"); } }
    Note that at the time of writing it requires patched POE::Component::Client::HTTP.

    Ilya Martynov,
    CTO IPonWEB (UK) Ltd
    Quality Perl Programming and Unix Support UK managed @ offshore prices -
    Personal website -

Re: Pass through perl http proxy server (no code, just advice)
by zengargoyle (Deacon) on Feb 28, 2003 at 21:56 UTC

    not Perl and a heavyweight (functionwise, not necessarily slow/bloated) is Squid Web Proxy Cache which you could probably make do whatever you can imagine.

Re: Pass through perl http proxy server (no code, just advice)
by traveler (Parson) on Feb 28, 2003 at 22:51 UTC
    Not perl, but a famous solution is dansguardian. It is GPL.

    HTH, --traveler

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://239558]
Approved by pfaut
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others meditating upon the Monastery: (1)
As of 2023-06-01 01:37 GMT
Find Nodes?
    Voting Booth?

    No recent polls found