Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked
 
PerlMonks  

Re: Caching Format

by moritz (Cardinal)
on Jan 11, 2012 at 17:41 UTC ( #947391=note: print w/replies, xml ) Need Help??


in reply to Caching Format

You could just keep the data in memory... or is there a reason not to?

Now on to the question. Any ideas on suggestions to make this quicker and easier?

Quicker to program or quicker to run?

If you want it quicker to run, you could just put the data on a RAM disc instead of a hard disc. If you want it easier to program, you could just use Perl data structures and JSON::XS or Storable for serializing and deserializing.

Currently the use of a DB is not an option.

And what is an option? How much control do you have over your environment? What about non-databases like memcached?

Replies are listed 'Best First'.
Re^2: Caching Format
by hok_si_la (Curate) on Jan 11, 2012 at 20:55 UTC

    Thanks everyone for the input and suggestions.

    The environment I am working with is accredited (read "takes and act of god to add/remove software requirements such as SQL, Postgres", etc) and hardened. Additionally it is hosted on one of our customer's systems. So short answer concerning environment, I have very little control. The images that are being sent are being sent to this one vendor from multiple hands, all with a defined file format, so unless I decide to rename everything I need to store additional info. I should also mention I plan to write a web app to run reports on submitted/non-submitted/incomplete collections. Using memory might be an option, though I do not know how long I need to make sure this information is available. I do know that 10s of thousands come in daily, so I guess I will have to weigh in on File IO choking vs hogging resources.

    hok

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://947391]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others lurking in the Monastery: (2)
As of 2021-12-06 04:41 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    R or B?



    Results (31 votes). Check out past polls.

    Notices?