Well i've got the script running and it seems to be doing ok. We have a pretty capable server so that probably helps. Speed isn't too much of an issue as it's just a background job, and there are plenty of other bottlenecks anyway ;)

Thanks for the replies though, i'll definately consider CSV_XS next time. I've actually used it before, but I was running my scripts on various different servers and I got fed up with having to ask them to install Perl modules, so now try and use the standard ones where possible.

Interesting foreign characters though, i've set it to binary to hopefully deal with that. That's been a problem for me before...