I realise that RDBMSs are well-tried and easily (if not always cheaply) available, and most OODBMSs are little more than a less than complete veneer over an underlying RDBMS, but is inconceivable to have an object store that doesn't rely upon mapping to a relational DB engine?

That's a great question. I tend to think that relational databases really are a great tool. The thing that makes them so good is that they are both very powerful and very capable. This stems from the fact that, conceptually, they map quite well to the constructs we need in software and that the hardware demands.

Relational databases organize data in a way that allows it to be efficiently managed on real physical data stores (hard disks and hard disk arrays). Think about how a full-table-scan translates to a sequential read off of a spinning disk. Think about how a table with a fixed structure can easily be translated into a fixed-width record and to blocks and sectors on a disk, and so on and so forth (lots more examples... but for the sake of being concise, I'l leave it there). The point is that the structure itself does a wonderful job of abstracting away the hardware, but in such a way that doesn't hide any of the best capabilities of that hardware. In short: they make a great abstraction of the physical persistence layer to your application.

Likewise, what they provide to your application for the purpose of persistence is about the best that can be provided without trying to do too much (and, thereby, doing it poorly and/or making life hard for programmers). If you're not convinced that doing more for applications than they do would be problematic, just look at how much of a mess most object-relational mapping frameworks are. Also (on that note) if the object relational mapping were done at the persistence level, yielding you an object-store rather than a data store... well, then you'd be in a situation like CORBA, which is also proof by counterexample.

There's two halves to saying that it does the right amount, though, and above I only defended the first half, that it didn't do too much... so let me explain why I think that it does enough, as well. Well, think about the data in your applications. In a C-ish world, you have structs, in a java-ish or C++-ish world you have objects' instance data, which is very much like a struct (note: I do not say that an object is like a struct; I say that an object's data is structured like a struct). And what's in a struct? Well, either "primitive" data-types, derived (non-primitive) data types (i.e. other "struct"-like entities). Depending on the details of your language of choice, you may have pointers or references included in your primitive data types or maybe that's how you store other derived data types... but take it as a given that you also (perhaps inclusively) have references or pointers involved as well. A relational database gives you table definitions, which are like struct typedefs, or like class definitions, or what have you... but the point is that they parallel how you declare your application's data structures. In those table definitions, you put primitive data types (numbers, character strings, dates) and you put references to instances of other (or the same) derived data types (i.e. tables). These, in the RDB world, are of course called "foreign keys".

Anyway, summary of that long paragraph is that RDBs give you what you need to model your application data in the form of:

• typedefs / class defs => table defs
• (non-pointer-) primitives => (duh) primitives
• pointers / references / included derived types => foreign keys
• structs / instances => rows

Last of all RDBs give you things for free, that you're likely to end up wanting, regardless. To name a few:

• Visibility into your application's state, and the ability to manipulate it. A sql client is like the world's coolest debugger!
• The ability to make arbitrary reports on your data. For any kind of real business application, this is invaluable. If you weren't using some kind of database for your data, you'd end up having to write a reporting framework, and you wouldn't be able to do it as well.
• Performance: I know a lot of new programmers think they can do better... but again, even if you could, it would be a hell of a lot of work you don't want to have to repeat, when, with an RDB, all you'd have to do is build an index.
• I'm sure there's some better example... but it's late and I'm getting tired
• Update: How could I forget: atomic transactions!

So, to the question, as you framed it: is it conceivable to have an object store that doesn't map to a relational DB engine. Sure it's possible, but I haven't seen much in the way of reasons to do it. The best example I can think of (which I acknowledge as an emerging possibility) are objects in languages, well, like perl... in which you don't necessarily have a fixed object "structure" as you do in java or C++ or C, etc. That is, that the objects are so amorphous that you can't nail down any kind of meaningful "structure" that covers every instance of the class. However, I tend to think of these free-form objects as typically having some kind of structure, it may just be a deeper structure than first appears. Also, there's the issue of subclasses which contain different instance data (and thus define a different structure) than their parents. To that I say: there's almost always a structure that represents the super-set of the structure of all of the relevant, related classes... define your table structure around that, and let the columns you don't need be null.

Anyway... this is a cool discussion, but I'm tired now.

Thanks for taking the time to give such a comprehensive answer. I'm not completely convinced by it yet -- but that's probably no surprise:) I started writing a blow-by-blow counter argument, but stopped as I decided you probably would not be interested in reading it, and that this may well be considered too far OT for this place.

Some parts of your post read a little like "I have a relational hammer, so the whole world looks like a Codd defined nail":) Please take that in the humourous manner in which it was intended. If you do have any interest in looking at an alternate viewpoint, you might like to read The OODBMS manifesto. The word "manifesto" kind of put me off reading it the first time I was refered to it, but I bit the bullet and read it anyway and was glad I did. It's not a that long. A hour or so should do it as a first pass.

The following is what I had written before I stepped back a little. I include it only as an "if your interested" sample of my thinking.

Read more... (4 kB)

---

Examine what is said, not who speaks.

"Efficiency is intelligent laziness." -David Dunham
"Think for yourself!" - Abigail
Hooray!
Wanted!

Some parts of your post read a little like "I have a relational hammer, so the whole world looks like a Codd defined nail":) Please take that in the humourous manner in which it was intended. If you do have any interest in looking at an alternate viewpoint, you might like to read The OODBMS manifesto. The word "manifesto" kind of put me off reading it the first time I was refered to it, but I bit the bullet and read it anyway and was glad I did. It's not a that long. A hour or so should do it as a first pass.

In response to this, and some other OO manifesto thingy, Chris Date and Hugh Darwen wrote The Third Manifesto, which details what a truly relational DBMS should provide. SQL does not adhere to relational theory, and so comments to the effect of "I've used relational DBMS's and they don't do what I need ..." are incorrect, because AFAIK there are no truly relational DBMS's.