One solution would be to ditch Apache::CacheContent and modify the code in your mod_perl handler to use something like Cache::Cache which will only put so many files into one directory before creating subdirectories (if you appropriately set the cache_depth option for Cache::FileCache).