ML Models & Algorithms
Jaccard Similarity
In shortA similarity measure for two sets: the number of shared items divided by the total number of distinct items, from 0 (nothing shared) to 1 (identical).
Jaccard similarity measures how much two sets overlap. It divides the number of items the sets share by the number of distinct items across both, giving a score from 0 (nothing in common) to 1 (identical). The sets can be words, character n-grams, entities or URLs, which makes it a handy, transparent measure for many SEO comparisons.
How it works
The formula is the size of the intersection divided by the size of the union. Jaccard distance is simply one minus the similarity.
Say set A is {red, running, shoes} and set B is {running, shoes, women}. They share two items (running, shoes) and contain four distinct items in total, so the Jaccard similarity is 2 ÷ 4 = 0.5.
What you put in the sets decides what the score means. Whole words compare vocabulary; character n-grams (such as three-letter chunks) make the score tolerant of typos; URLs from the results page compare how Google ranks two queries.
How it’s used in SEO and marketing
- SERP overlap. Compare the top-10 URL sets of two keywords to decide whether they belong on one page.
- SERP volatility. MLforSEO’s SERP automation guide calculates URL turnover as one minus the Jaccard similarity of two snapshots of the same results page.
- Near-duplicate content. Compare sets of word n-grams (“shingles”) between pages.
- Entity coverage. Compare the entities extracted from your page with those on competing pages.
- Hashtag and tag clean-up. Group variants that share most of their characters.
Example
Two keywords share 6 of their top-10 URLs. The union is 10 + 10 − 6 = 14 distinct URLs, so the Jaccard similarity is 6 ÷ 14, about 0.43. If your threshold for “same intent” is 0.3, these two keywords can be targeted by one page.
Limitations and things to watch
- It ignores frequency and order: a word used once counts the same as a word used twenty times.
- It ignores position, so sharing the number one result counts the same as sharing number ten. Weighted measures fix this when position matters.
- Scores on very short strings jump around, since one item makes a big difference.
- Like other string measures, it has no sense of meaning; synonyms do not overlap.
- For very large collections, approximate methods such as MinHash estimate it quickly.
Learn it on MLforSEO
Where Jaccard Similarity shows up on MLforSEO
Read about it on the blog
Sources and further reading
Other terms from ML Models & Algorithms
- BERT (Bidirectional Encoder Representations from Transformers)Google's 2018 transformer language model that reads text in both directions to produce context-aware embedding…
- BERTopicA modular topic modelling technique that clusters transformer embeddings and labels each cluster with class-ba…
- BIRCH (Balanced Iterative Reducing and Clustering using Hierarchies)BIRCH (Balanced Iterative Reducing and Clustering using Hierarchies) clusters very large numeric datasets by s…
- BM25A classic ranking function that scores documents by how often query words appear, adjusted for word rarity and…
- Boyer-MooreA fast exact string-search algorithm that compares a pattern from right to left and skips ahead after mismatch…
- c-TF-IDFClass-based TF-IDF: treats each cluster as one document to find the words that best describe it, used by BERTo…
- Centroid-based ClusteringA clustering family that groups data around central points (centroids), each the average of its cluster's memb…
- Convolutional Neural Network (CNN)A deep learning architecture that scans data with small learned filters to detect local patterns; the classic…
- DBSCANDensity-based clustering that groups closely packed points, finds the number of clusters itself and labels iso…