http://qs321.pair.com?node_id=760191


in reply to Huge data file and looping best practices

This happens to be a classic computational chemistry problem. When searching large chemical databases, compound characteristics are hashed into long bit strings to minimize the number of expensive graph comparisons. One common type of pairwise bitstring comparison is called a Tanimoto coefficient.

Tanimoto = (A & B) / (A | B)
Ie, the number of characteristics in common divided by the number of characteristics found in either. The bias is that positive information counts higher than negative. This is used to look for closest relatives, find a diverse subset and compare the diversity of collections (comparing the average of each object's closest relative within its collection).

I've probably gone too far with all this Comp Chem stuff but it might be a good way to compare many patents. Could also look for Cosine or Dice coefficients used in other fields.