http://qs321.pair.com?node_id=394818


in reply to Parse data into large number of output files.

The real RDMBS suggestion is probably going to be the best way, but if you want to give it a go with multiple files check out FileCache.