Given the small number of files you're reading into an array at once (40,000 names isn't a lot, really), I'd grab all of them and then use an array slice to operate on subsets of your array until finished.

Check also here, and here.