Just some random thoughts and questions:
- You do have the stream stored somehow, even if not in memory, ie you can replay it?
- How large is your alphabet? Even if it is not characters are there 100s, 1000s or how many values?
- How do you distinguish short from long repeated sequences? In other words, what is your minimum length for a successful repeat? There could be a repeat of 5 items, is that a success?
- If you have a minimum length, try to do running checksums or such. Obviously, you would need to store position as well, otherwise you cannot go back and retrieve the sequence corresponding to that checksum.
- Do you need perfect matches or is it ok if there are a few deviations? This might be used in the construction of the checksum.
Not very helpful, I am afraid.