note
dmitri
The reason I think that SQLite would be useful is that if we want to separate the spider from indexer, finding the articles to update in the index is as simple as
<code>
SELECT * FROM ARTICLES WHERE LAST_UPDATED > $LAST_TIME_I_RAN
</code>
instead of searching the filesystem. Stored on the filesystem, we will need code to
<ul>
<li>search,
<li>store, and
<li>update
</ul>
the documents. SQLite provides all of that for free. Want to move to a different machine? -- The database is a single file. Plus, who knows what other useful things SQLite's flexibility will allow us to do?
620228
620362