Those are exactly the type of problems which should cause you to switch away from the CSV file for the underlying "database".

A colon (or comma etc) separated file is fine as long as you have a relatively small number of records, and you will be mostly searching them, only occasionaly adding records.

As soon as you start changing records, deleting them, or trying to manage hunderds or more, a database comes in really handy. A database can complete such operations faster because it isn't hobbled by the CSV format: it can create indices, trade space for speed, and use other strategies which are not available to a CSV using program.

There are a number of very lightweight databases, such as DBD::SQLite which can be installed without fuss and aren't difficult to administer.

If you need the CSV format for other programs, or because humans want to also edit by hand, you can allways emit the CSV from the database after every change.