Basically what you are trying to do is use the XML file as a flat file database, and that doesn't really appear to be a good choice of technology for the problem.

XML is intended as a medium to allow inter-system communication of information; it creates a framework that provides the structure of the data within the data. I don't think XML is providing you any benefit here, and is a slow performing technology for the application. You would be better served by some form of database.

When is XML too much? When you use it for the wrong thing -- much the same issue as any tool.