Semantic Similarity

Semantic similarity measures how close two pieces of text are in meaning. “Link building” and “backlink outreach” share few letters but score as very similar, while “hard” and “hand” differ by one letter yet mean different things. It is usually calculated by turning each text into an embedding and comparing the vectors, which is what separates it from character-level fuzzy matching.

How it works

An embedding model, often from the Sentence Transformers library such as all-MiniLM-L6-v2 (384 dimensions), converts each keyword, sentence, paragraph or page into a vector. Cosine similarity then compares the direction of two vectors: higher means closer in meaning. Cosine is the default similarity function in Sentence Transformers; on normalised embeddings, a dot product gives the same result.

Scores only make sense relative to the model that produced them. MLforSEO’s internal linking script found that bge-small rates almost every pair of pages above 0.6 while other popular models rarely go that high, so it uses a relative threshold (the top 10% of your own site’s similarity distribution) instead of a fixed number.

How it’s used in SEO and marketing

• Keyword-to-topic mapping: embed keywords and topics with sBERT and assign each keyword to its closest topic, even with no word overlap.
• Internal linking: pages that are similar but not yet linked become link candidates.
• Duplicate and cannibalisation checks: very high similarity (0.95 and up) usually signals a near-duplicate to consolidate.
• No-code audits: Screaming Frog’s SEO Spider (version 22+) flags semantically similar pages (default threshold 0.95) and low relevance content measured against the site’s average embedding.
• AI search: the early stage of retrieval keeps chunks whose vectors sit closest to the query (information retrieval).

Example

In MLforSEO’s internal linking guide, a script scored every page pair on a 79-post blog with cosine similarity, kept unlinked pairs in the top 10% while skipping near-duplicates, then embedded about 5,400 paragraphs to find the best spot for each link. The output was a review file of 128 suggestions, each approved or rejected by a person.

Limitations and things to watch

• Similarity is not relevance to the reader: two pages can be 0.85 similar and still make a pointless link.
• Scores from different models are not comparable, so set thresholds per model and per dataset.
• Topical closeness hides factual differences; a wrong statement on the right topic still scores high.
• Long texts may be truncated (all-MiniLM-L6-v2 cuts input beyond 256 word pieces), so embed paragraphs or chunks for long pages.

Learn it on MLforSEO

• 10 Ways to Automate Internal Linking with AI
• Different ways to Map Keywords to topics, with Supervised and Unsupervised ML approaches
• Map Keywords to Topics: Supervised (sBERT + Fuzzy) vs. Unsupervised (BERTopic) (Notebook)
• Introduction to Machine Learning for SEO (course)