In a project like this I really recommend going with a SQL
database of some type. (MySQL and Postgres are both viable
free alternatives see
this column for a good overview of each's strengths and
weakness's. Also, see
this for a good intro to some of the things you need to
be aware of like data modeling and so forth.
If you want to do cross referencing, SQL offers a powerful
methodology for that type of thing. SQL in general makes it
very easy to do very complex tasks with very few words. I
would recommend SQL not merely for performance issues but
for the degree to which it will help you deal with some of
the complexity it sounds like you want to work into this
project.
Good Luck,
Mark
