Word2Vec

Word2Vec is a technique, published by Tomas Mikolov and colleagues at Google in 2013, that learns a vector (an embedding) for every word in a corpus by looking at the words around it. Words used in similar contexts end up with similar vectors, so “king” and “queen” or “trainer” and “sneaker” sit close together. It was one of the first widely used ways to represent meaning numerically and paved the way for today’s embedding models.

How it works

Word2Vec trains a shallow neural network on a simple prediction task, then keeps the network’s internal weights as the word vectors. It has two variants:

• Continuous bag of words (CBOW): predict a word from its surrounding words.
• Skip-gram: predict the surrounding words from a word.

After training, each word has a vector of, say, 100 to 300 numbers. You can compare words with cosine similarity, find nearest neighbours and do the well-known vector arithmetic in which king minus man plus woman lands near queen. In Python it is usually trained with Gensim.

Unlike a bag of words, Word2Vec captures that different words can mean similar things. Unlike modern transformer models, it gives each word one fixed vector, whatever the sentence: “bank” gets the same vector in “river bank” and “bank account”.

How it’s used in SEO and marketing

• Information gain: MLforSEO’s Semantic Keyword Research course explains that Google’s information gain patents describe representing documents as semantic feature vectors using models such as Word2Vec and neural networks, to judge how much new information a document adds.
• Finding related terms and synonyms in a niche vocabulary by training on your own corpus of pages, reviews or support tickets.
• Feature input for older classification and clustering pipelines.
• Historical context: it explains how search moved from matching strings towards matching meaning.

Example

Train Word2Vec on 50,000 product reviews for a footwear brand. The nearest neighbours of “pinch” turn out to be “narrow”, “tight” and “toe box”, which surfaces a fit complaint vocabulary customers actually use. Those terms then feed keyword research for a sizing guide.

Limitations and things to watch

• One vector per word means no context sensitivity; ambiguous words blur together.
• Words not seen in training get no vector (later variants such as fastText handle sub-words better).
• It needs a reasonably large corpus to learn useful vectors.
• For most SEO tasks today, sentence-level transformer models such as Sentence-BERT give better results with less effort.

Learn it on MLforSEO

• What Is Information Gain in SEO? How Google Measures Content Uniqueness
• Topic Modelling for SEO with LDA (Latent Dirichlet Allocation)
• Semantic ML-enabled Keyword Research