But, there is another solution - DBM::Deep is Perl datastructures backed by disk instead of RAM plus transactions. Now that DBM::Deep has transactions, I'm hoping to actually get Presto up and running. It won't be as fast as the hand-tuned code that the major RDBMSes have, but it will be Perlish and be object-centered vs. row-centered.

A bit on a tangent, but you can already use Perl in PostgreSQL. Of course, this has little to with the issue raised here, because Perl is not used there as a data definition or query language; you can just use it in PostgreSQL to store procedures into the database engine.

Also in MySQL: MyPerl

There has been some talk of making Perl 6 RDBMS friendly by taking advantage of their conceptual intersections. The end result would be that what you ordinarily express as SQL could be expressed in Perl natively. I don't know how far beyond talk this has gone exactly as I haven't kept up with it. But it seems interesting given that even if, as some believe, Perl 6 never gets here, many Perl 6 concepts filter their way back into Perl 5. (Maybe the design of DBI2 will be influenced. Maybe Perl 5 will get native SQLian constructs. Who knows?)

You should consider the idea behind Alzabo and then Rolsky's latest work which sounds like Alzabo + a data back end. His work has concentrated on being closer to the relational model than you'll find in implementations in most RDBMs.

I seem to remember a few years ago, some of the mainstream RDBMS vendors (Oracle and Sybase) were extending their products so stored procs could be written in Java -- and a nice side effect was they did it in such a way that stored procs could be written in almost any language. Sybase's attempt was called ESP and there were some rumblings about perl being in the mix. I don't think the extensions (even Java) ever really took off. Maybe the core programmers doing stored proc development never had a vested interest in learning anything other than T-SQL or PL/SQL or maybe products like DBI, ODBC and JDBC scratch more than 80% of the itch.

ESPs in Sybase are extended stored procs, usually written in C and executed via an external server (the XP server).

Java stored procs/functions are available inside of the dataserver as part of the JVM that is included in the dataserver itself.

I wrote a prototype that loaded a perl interpreter into the XP server and allowed the easy creation of RPCs written in perl, but it was rather clunky and the performance wasn't great...

Michael

Not the itch I was thinking about - all that does is move the interface to the server rather than resolve the dissimilarity in data architecture.

-M

Free your mind

dbm-deep has both multi-user and transactions. What more do you want?

---

My criteria for good software:

1. Does it work?
2. Can someone else come in, make a change, and be reasonably certain no bugs were introduced?

Well a lot more! But I'll certainly look into your suggestion and evaluate how much it covers.

Update: and it appears on the face of it that the "more" not covered by DBM::Deep includes, but is not limited to the following, although it could be used as a basis for implementing these on top:-

- Support for Perl stored procedures

- Transaction logging

- System administration tools such as for listing users, locks etc.

- backup/recovery

- server startup/shutdown

- bulk copy

- a security model

- user ownership and privileges model

- group ownership and priviliges

- run levels

- support for logical and/or physical partitioning

- virtual memory configuration and/or other tuning capabilities

- quotas

-M

Free your mind

There are other more pedestrian concerns to consider that have not been mentioned. Most RDBMS's provide infrastructure services for handling backups, recoveries, rollbacks, auditing, etc.. These are arguably somewhat mature.

What I hear you asking for is idiomatic data persistence.

I tried reaching for this recently on a project in the Java world by using Derby. I'm not recommending either - don't misunderstand. The point I want to offer is that though I was able to use this tool with idioms native to the language, the challenges we faced had more to do with supporting a production application than with the efficiency of the programming model.

In the end we went with the more traditional approach, not because it was more convenient to code - it wasn't - but because it was easier to support from an operational standpoint.

I am not trying to discourage you however, merely shedding light on important functionality often considered an afterthought that perhaps should be included in the discussion.

Frankly I'd like to see something like this in a truly robust form. Personally, I find the endless vendor variations of SQL tedious.

Idiomatic data persistence is (mostly) provided by DBM::Deep, which also has transactions. I deliberately did not address the issues of recovery and auditing in order to provide a single-file solution. If those issues are more important to you than idiomatic data persistence, then your choice has been made for you. I have rarely seen the situation where both are absolutely necessary.

---

My criteria for good software:

1. Does it work?
2. Can someone else come in, make a change, and be reasonably certain no bugs were introduced?

Feel free to use one of the existing object-oriented DBs with perl bindings. For example, AcePerl.

That would be the obvious solution if the data is object-oriented - but it's not necessarily so. In my own case, over the last ten years I have been working more often than not with timestamp data, both in regard to realtime market pricing and mobile 'phone billing. Either way, neither the relational nor object-oriented model is ideal for storing and processing the data and relational tends to be used as a sort-of default architecture.

For example, in the case of the billing, it stays in record form as long as possible during the algorithm where Perl rips through like a knife through butter. But eventually the huge 6-million record customer database is needed for lookup and the records have to be split into batches and processed slowly 100 at a time (and that's for best performance) while it trundles through the stored procedure processing before Perl could get its hands on it after that to finish the job off. The ineffectiveness of the relational architecture is costing the operators serious money and an object architecture would not in fact come to the rescue.

-M

Free your mind

The obvious question is: which concepts have to be introduced into Perl by a "RDBMS in Perl"? Which feature of RDBMs make these so attractive to be implemented in perl instead of relying on the mature solutions in this field?

Two concepts come to my mind, which would be an interesting addition to the perl datastructure zoo.

First, the idea of referential integrity, that is, declarative integrity between different objects. For example, we have a class customer, and a class bill, and the programmer can declare in advance the integrity conditions between the customer and his bills.

Second, the idea of declarative programming in general. Example: One declares a class Room and a class guest, and now, given the integrity conditions, the system generates valid "object tuples", from which one could select the most suitable that relates a guest with a corresponding room.

Yes, these are two good additions to the spec. Another is CREATE TRIGGER for functionality that goes beyond what declarative programming can feasibly achieve, e.g. business transaction logging.

-M

Free your mind