Databases Stored In memory

mt2k has asked for the wisdom of the Perl Monks concerning the following question:

Hello all you fellow monks out there!

I have heard a lot about databases being stores in memory for quick swapping of information. I have heard about the whole thing of decreased access time and handling, and am now very interested in the subject!

My current web site locks a file at the beginning of the script to ensure that only one request is asked of my database file at once. Unfortunately, my database at the moment is a data structure printed to a file via Data::Dumper. The data structure is a hash (%db) that contains all my data.

I cannot say much about the subject, as I have never dealt with Apache modules before, but if my memory serves me correctly (which I highly doubt!), I remember hearing something about the whole memory subject being related to the Apache modules.

I am very interested in knowing whether it is possible to store this hash in memory, where my CGI scripts could access and update it. If anyone either has some knowledge on this subject or any good reference documents, please let me know!

Many, many thanks in advance!

Comment on Databases Stored In memory

Replies are listed 'Best First'.
Re: Databases Stored In memory by samtregar (Abbot) on May 05, 2002 at 22:58 UTC
It's a little hard to know where to start since you've touched on a lot of different topics in your, um, question. I think you owe it to yourself to do some reading on the topic then return with more specific questions if need be. I suggest CGI Programming with Perl as well as Writing Apache Modules with Perl and C. Both treat the subject of storing data used in web applications on disk and in memory. If you're serrious about building database applications, I suggest you also check out Programming the Perl DBI which will show you how to use a real database in your application. -sam	[reply]
Re: Databases Stored In memory by lhoward (Vicar) on May 06, 2002 at 01:15 UTC
Items that you store in memory with mod_perl are only available to the same apache process, so that may not help you much. You could try using IPC::Shareable or IPC::ShareLite to make your datastructure available across processes. This can buy you a lot of performance, but has a lot of risks (i.e. if you reboot your system the data in shared memory is gone, unless you have a facility to write it to disk). Your best bet is probably to use a real database (Postgres, mysql, etc...) and if it isn't fast enough for you then consider some sort of in-memory caching. mod_perl can buy you a lot with traditional databases performance wise (presistant cached database handles, etc..)	[reply]
Re: Databases Stored In memory by perrin (Chancellor) on May 06, 2002 at 16:40 UTC
You're getting ahead of yourself. Before you try to solve performance problems from databases, you need to get your stuff to use a database. You will probably find that it's fast enough once you do that. A relational database like PostgreSQL or MySQL is a good eventual goal, but the quickest transition for you would probably be to switch to a dbm file. Use MLDBM::Sync and it will handle all the necessary locking for you.	[reply]

Back to Seekers of Perl Wisdom