i don't think there are any constraints on where a substring may match a larger string.

there can be exact matches. we are removing those using the unique function.

for the current set of sequences (assuming the program is working correctly), we go from 206,737 sequences to 166,089. not sure how many are removed because they are exact matches and how many are removed because they match a substring of a larger sequence. i will add that check.

i think the program is working correctly, but it takes a long time, longer than it seems it should take. (~5 hours on a pretty fast computer) getting the unique sequences is fast, but we have an additional step where we go back and retrieve the sequence name and then do the substring comparison. it is these two steps that are so slow.