This is a proxy server that will record all your web browsing
activity. It's based on cool
NetServer::Generic, which lets
you focus on the core of your server app, and leads to compact
servers. Performancewise it may have a slight effect
on the response time of your browser, but it's lightweight
enough not to be noticed if you have a fast machine. It also
takes advantage of
Proc::Daemon to do all those things decent
daemons should, like detaching themselves from controlling
terminals.
Usage: First configure it by changing the variables
on top to suit your browsing habits and bandwidth.
$pageview_range should be a little longer than the
average time it takes your browser to issue requests for all
component files of a page view. $per_page_time is
the average time you spend on a page, for the program to
give you a simple approximation of the time you spend on
the web. $listen_port is the port you want your
proxy to listen on and $logfile should be the path
to the logfile where your web browsing activity is to be
recorded.
If you run this proxy on your own machine you
should configure your browser to use a proxy on
localhost and the port you configured the program
with.
After
doing this just continue your happy browsing and when
you're curious about how much time you spend on the web
(and on any particular site you visit) just go to the
http://stats url and you'll get a nice report
from the proxy.
It works under Linux, and
may also run on NT. This is my first cut at it and
I haven't tested it exhaustively. Please provide any
comments you may have on functionality or style.
Warning:This program could be a very nasty thing
to use on coworkers, but may be of great help in monitoring
a child's use of the web.
Update: I found out that my program will fill the
process table with zombie processes. Looks like a bug in
NetServer::Generic, since the ones I fork are
properly ignored by the parent and do not remain.
Update: Wow.. It turns out that NetServer::Generic
indeed has a bug. If you want to fix it yourself then go to
the source (file Generic.pm) and replace all lines
$SIG{CHLD} = &reap_child(); for
$SIG{CHLD} = \&reap_child;.
I'm using v1.02, which is the most recent. I'll submit
a patch to the author.
#!/usr/bin/perl -w
use strict;
use NetServer::Generic;
use Proc::Daemon;
my $listen_port = 8080;
my $logfile = '/tmp/proxy_log';
my $pageview_range = 20; # seconds
my $per_page_time = 2; # minutes
my $server_cb = sub {
my ($s) = shift ;
my $line1 = <STDIN>;
unless($line1 =~ m[(\w+)\s+http://([^/:]+)(:(\d+))?(\S*)\s+(\S+)]) {
print STDOUT "HTTP/1.0 400 Bad Request\nConnection: close\n\n";
print STDOUT "HTTP/1.0 400 Bad Request\n";
return;
}
my ($method, $serv, $port, $path, $version) = ($1, $2, $4, $5, $6);
if($serv !~ /stats/) {
my $sock = IO::Socket::INET->new(PeerAddr => $serv,
PeerPort => $port || 80,
Proto => 'tcp');
print $sock "$method $path $version\n";
print $sock "Connection: close\n";
$SIG{CHLD} = 'IGNORE';
if(my $pid = fork) {
while(<STDIN>) {
print $sock $_;
}
} else {
while(<$sock>){
print STDOUT $_;
}
}
} else {
my $stats = &getStats();
print STDOUT "HTTP/1.1 200 OK\nContent-type: text/plain\n";
print STDOUT "Connection: close\n\n";
print STDOUT "Your Browsing Stats!\n\n";
print STDOUT "$stats->{DAY} page views in the last day\n";
print STDOUT "$stats->{WEEK} page views in the last week\n";
print STDOUT "$stats->{MONTH} page views in the last month\n";
print STDOUT "$stats->{YEAR} page views in the last year\n\n";
my $avg_time = ($stats->{MONTH} / 30) * $per_page_time;
print STDOUT "At $per_page_time minutes per page that's $avg_time
+minutes per day in the last month.\n\n";
print STDOUT "Your favorite sites:\n\n";
foreach(map {$_->[0]}
sort{$b->[1] <=> $a->[1]}
map{[$_, $stats->{BY_SERVER}{$_}{TOTAL}]}
(keys %{$stats->{BY_SERVER}})) {
print STDOUT "$_\n";
print STDOUT "--------------------------------------------\n";
print STDOUT $stats->{BY_SERVER}{$_}{DAY} || 0,
" page views in the last day\n";
print STDOUT $stats->{BY_SERVER}{$_}{WEEK} || 0,
" page views in the last week\n";
print STDOUT $stats->{BY_SERVER}{$_}{MONTH} || 0,
" page views in the last month\n";
print STDOUT $stats->{BY_SERVER}{$_}{YEAR} || 0,
" page views in the last year\n\n";
}
}
open LOG, ">>$logfile" or die "could not open $logfile";
print LOG $serv, ' ', time, "\n";
close LOG;
};
my ($foo) = new NetServer::Generic;
$foo->port($listen_port);
$foo->callback($server_cb);
$foo->mode('forking');
print "Starting server\n";
&Proc::Daemon::Init();
$foo->run();
sub getStats {
my $day_ago = time - 60 * 60 * 24;
my $week_ago = time - 60 * 60 * 24 * 7;
my $month_ago = time - 60 * 60 * 24 * 7 * 30;
my $year_ago = time - 60 * 60 * 24 * 7 * 30 * 12;
my($serv, $time, %hits);
open LOG, "$logfile" or die "could not open $logfile";
while(<LOG>) {
($serv, $time) = split;
if($time - $hits{BY_SERVER}{$serv}{LAST} > $pageview_range) {
$hits{BY_SERVER}{$serv}{HITS}{$time} = 1;
$hits{BY_SERVER}{$serv}{LAST} = $time;
$hits{BY_SERVER}{$serv}{TOTAL}++;
if($day_ago < $time) {
$hits{BY_SERVER}{$serv}{DAY}++;
$hits{BY_SERVER}{$serv}{WEEK}++;
$hits{BY_SERVER}{$serv}{MONTH}++;
$hits{BY_SERVER}{$serv}{YEAR}++;
$hits{DAY}++;
$hits{WEEK}++;
$hits{MONTH}++;
$hits{YEAR}++;
} elsif ($week_ago < $time) {
$hits{BY_SERVER}{$serv}{WEEK}++;
$hits{BY_SERVER}{$serv}{MONTH}++;
$hits{BY_SERVER}{$serv}{YEAR}++;
$hits{WEEK}++;
$hits{MONTH}++;
$hits{YEAR}++;
} elsif ($month_ago < $time) {
$hits{BY_SERVER}{$serv}{MONTH}++;
$hits{BY_SERVER}{$serv}{YEAR}++;
$hits{MONTH}++;
$hits{YEAR}++;
} elsif ($year_ago < $time) {
$hits{BY_SERVER}{$serv}{YEAR}++;
$hits{YEAR}++;
}
}
}
close LOG;
\%hits;
}
