Beefy Boxes and Bandwidth Generously Provided by pair Networks
go ahead... be a heretic
 
PerlMonks  

Apache::CacheContent and unforeseen side effects

by geektron (Curate)
on Dec 27, 2004 at 17:45 UTC ( [id://417590]=perlquestion: print w/replies, xml ) Need Help??

geektron has asked for the wisdom of the Perl Monks concerning the following question:

one of my client's sites uses an external site to store name, address, etc. information, and rather than maintain that information in 2 places ( our local DB and the 3rd party site ), we use a mod_perl handler to fetch the relevant information and substitute it in templates. requests for http://mysite.com/12345/index.htm are "replicated" by using the ID number in the URI, contacting the 3rd party site, and using the returned information to populate the index.htm template

last week the 3rd party site complained about too many hits to the server, so i implemented Apache::CacheContent for this site. minimized the hits to the 3rd party server, made things faster for endusers because of the lessened amount of 'net traffic, and all seemed fine.

what i didn't realize would be a problem: the sheer volume of these cache files. i now have something like 46,000 cache files ... because of the number of customers that need this replicated data. that's a problem, because even a remote `ls` takes too long.

i want to stuff the cache files somewhere *else*, i.e. one step out of the DocumentRoot ... but i don't see any options in the perldoc to change the location of the cache files. i was thinking about using a DirectoryAlias, but i also need the LocationMatch to trigger the cache/ dynamic content.

anyone done something like this before?

EDIT: the httpd.conf section for the handlers might be useful ...

PerlModule Mysite::Replicate <Directory /home/httpd/mysite.com/html> SetHandler perl-script PerlHandler MySite::Replicate </Directory> <LocationMatch "^/[0-9]"> SetHandler perl-script PerlFixupHandler Mysite::Replicate->disk_cache </LocationMatch>

Replies are listed 'Best First'.
Re: Apache::CacheContent and unforeseen side effects
by The Mad Hatter (Priest) on Dec 27, 2004 at 18:11 UTC
    One solution would be to ditch Apache::CacheContent and modify the code in your mod_perl handler to use something like Cache::Cache which will only put so many files into one directory before creating subdirectories (if you appropriately set the cache_depth option for Cache::FileCache).
      i'm getting there. Apache::CacheContent doesn't expire files, it seems, if the user hasn't revisited the content in the DefaultTTL time ...
        Ah. Could you setup a cron job to manually "expire" (aka, delete) the old files?
Re: Apache::CacheContent and unforeseen side effects
by perrin (Chancellor) on Dec 27, 2004 at 21:05 UTC
    Apache::CacheContent is mostly a proof of concept. I recommend you ditch it and use it a cache reverse proxy instead. In addition to the basic caching features, it will also handle conditional-get (If-Modified-Since) for you. You really should be running a reverse proxy anyway with any mod_perl server that's getting siginificant traffic.
      i did ditch Apache::CacheContent -- in favor of a simple Cache::FileCache solution that just caches the needed data fro mthe 3rd party site. took me about an hour, including reading the perldoc on it ...

      reverse proxy? i don't see how this would solve the problem. our servers are fine ... it's the 3rd party site that has the issues with the hits.

        Apache::CacheContent caches the whole generated page. So does a caching reverse proxy. That's why one is a good substitute for the other.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://417590]
Approved by Arunbear
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others avoiding work at the Monastery: (6)
As of 2024-03-29 12:45 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found