Most commonly, I stuff the values as hash-members with a do-nothing value of '1' for each entry, just as you suggested in the OP.   But I have also iterated through a list e.g. with the grep() function and don’t often see much difference ... because my applications are not starving for microseconds (like, e.g. the high-performance applications frequently written by BrowserUK in which this governs the whole design) and the entire list is of such a size that it all fits in real-memory anyhow.   You probably have the luxury of choice in this case, where your decision does not seem to be “forced” by this one consideration alone.

Probably, I would let my design be guided by what else you need to do with this list of numbers, and with the data that is associated with them.   Is it, literally, just a list of numbers, such that you only have to see if the number exists and such that, having done so, there is absolutely nothing else associated with that number?   Questions like that.