As long as you don't need a "real" server (and as you're dealing currently with flat files, it dosen't sound like it), DBD::SQLite is a very nice solution to store data in files and access them fairly fast through SQL and Perl.

Of course, as soon as you have more than one process stomping over the data, all sorts of interesting issues appear, so you might want to consider whether you will stay with a single-process solution or file locking, or will want a real solution with concurrent access, which is something that SQLite dosen't do from what I remember.

perl -MHTTP::Daemon -MHTTP::Response -MLWP::Simple -e ' ;    # The  
$d = new HTTP::Daemon and fork and getprint $d->url and exit;#spider
($c = $d->accept())->get_request(); $c->send_response( new   #in the
HTTP::Response(200,$_,$_,qq(Just another Perl hacker\n))); ' #  web
[download]