Skip to content
Browse courses

ML Models & Algorithms

TF-IDF with Cosine Similarity

In shortComparing texts by turning each into a TF-IDF vector, where rarer terms weigh more, and measuring the angle between vectors with cosine similarity.

Updated 24 Oct 2026· 2 min read

TF-IDF with cosine similarity is a way to compare texts. Each text becomes a TF-IDF vector, in which terms that are rare across the whole collection count for more, and the similarity of two texts is the cosine of the angle between their vectors. It is one of the five string-matching approaches MLforSEO teaches, and the one that brings some context-sensitivity to fuzzy matching.

How it works

  • Tokenise every text, into words or, for fuzzy matching, character n-grams (often trigrams).
  • Term frequency (TF) counts how often each token appears in a text.
  • Inverse document frequency (IDF) down-weights tokens that appear in many texts, usually as the log of the number of texts divided by the number containing the token.
  • Multiply the two to get each text’s vector of weights.
  • Cosine similarity divides the dot product of two vectors by the product of their lengths. For TF-IDF the result runs from 0 (nothing in common) to 1 (the same weighted mix of tokens).

In Python, scikit-learn’s TfidfVectorizer and cosine_similarity do this in a few lines, and setting the analyser to character n-grams makes it tolerant of typos. PolyFuzz uses character-trigram TF-IDF as its default matching model.

How it’s used in SEO and marketing

  • Redirect mapping between old and new URLs or titles.
  • Matching product or entity names across messy data sources.
  • Mapping keywords to the closest existing page.
  • Spotting near-duplicate content and comparing competitor pages.

Example

Imagine mapping 404 URLs on a site where almost every path starts with /blog/. Because /blog/ appears everywhere, its IDF is close to zero and it adds almost nothing to the score. Distinctive tokens such as “fuzzy” and “matching” carry the weight, so /blog/fuzy-matching-guide/ lands on /guides/fuzzy-matching-for-seo/ rather than on any other page that happens to share the /blog/ prefix.

Limitations and things to watch

  • It does not capture meaning: synonyms and paraphrases score low. Use embeddings for semantic similarity.
  • Results depend on the collection, since IDF is calculated from it; add or remove texts and scores shift.
  • It needs preprocessing, such as lowercasing and stripping boilerplate.
  • High-accuracy settings (more n-gram sizes, larger vocabularies) are slower.

Learn it on MLforSEO

Where TF-IDF with Cosine Similarity shows up on MLforSEO

Try it yourself

Sources and further reading

Other terms from ML Models & Algorithms

See all 55 ML Models & Algorithms terms →