You made no mention about it, so I'd guess that the files are not sorted.

If the files are bigsized, it may be worth to first sort them according to the fields you're interested in - even if that may take quite some time too - and then sequentially step _once_ through the sorted files to pick up the mergers.