I'm curious what you mean by "DBD::SQLite is not a sound database". I know it's a single file, lightweight, minimal-SQL set database, but to me something that isn't sound is buggy. Is that what you're saying? Just curious. (By the way, I just discovered a GPF-producing bug if I switch my code to use selectrow_array() with a pre-prepared statement on Win32, so maybe you've got a point. ;)
And yes, I fully expected the database solution to be much slower than the in-memory hash solution. I mentioned that in my post. I was attempting to quantify how much slower, and how sensitive the database version is to dataset-size.
Wrapping in transactions definately helps. Take the example of my building the database initially. With transactions, it takes the blink of an eye, but without, it takes about 45 seconds to build the database on my system. But on the query side, I'm specifically trying to engage in single queries, and didn't want to see transactions influencing the speed. However, I agree it would be interesting to rewrite the test in such a way that it could take advantage of transactions, and I may do that just to see what comes of it.
Anyway, it's kind of fun to tinker with this stuff. ...hope you agree, even it the database discussion is diverging a little from core-Perl topics.
|Replies are listed 'Best First'.|
Re^3: Hash lookups, Database lookups, and Scalability
by pg (Canon) on Oct 31, 2004 at 18:16 UTC