ML Models & Algorithms
LSA (Latent Semantic Analysis)
In shortA topic modelling technique that uses singular value decomposition (SVD) on a term-document matrix to uncover hidden themes in a set of documents.
Latent Semantic Analysis (LSA) finds hidden (“latent”) themes in a collection of documents by compressing a large table of word counts into a small number of underlying dimensions. Documents and words that tend to appear together end up close to each other in this reduced space, even when they do not share exact words. In information retrieval the same method is called Latent Semantic Indexing (LSI).
How it works
- Build a term-document matrix, usually weighted with TF-IDF, with one row per word and one column per document.
- Apply truncated singular value decomposition (SVD), a linear algebra technique that splits the matrix into components ranked by how much of the structure they explain.
- Keep only the top k components. Each document and each word becomes a short vector in this k-dimensional “concept” space.
- Compare documents with cosine similarity, or read the top-weighted words of each component as a rough topic.
LSA was introduced by Deerwester and colleagues in 1990. In Python, scikit-learn’s TruncatedSVD and Gensim’s LsiModel both implement it.
How it’s used in SEO and marketing
- Content similarity. Find pages about the same theme to suggest internal links or spot overlap.
- Quick topic exploration. A fast, cheap baseline before trying LDA or BERTopic.
- Dimensionality reduction. Compress sparse TF-IDF features before clustering or classification.
Example
You run TF-IDF and truncated SVD with k = 10 on 500 blog posts. One component weights “crawl”, “index”, “robots” and “sitemap” heavily, another “embedding”, “cluster” and “vector”. Posts that score highly on the first component form a technical SEO group, and pairs of posts close together in the ten-dimensional space become internal link candidates.
Limitations and things to watch
- Components can contain negative values, which makes them harder to read as topics than LDA or NMF output.
- It is linear and based on bag of words, so word order is ignored and polysemy is handled poorly.
- You must choose the number of components.
- MLforSEO’s LDA guide notes that LDA’s authors showed it outperforming probabilistic LSI, and modern embedding models usually capture meaning better still.
- The “LSI keywords” advice in SEO folklore borrows the name without using the method, so treat it with caution.
Learn it on MLforSEO
Where LSA (Latent Semantic Analysis) shows up on MLforSEO
Learn it in a course
Read about it on the blog
Sources and further reading
Other terms from ML Models & Algorithms
- BERT (Bidirectional Encoder Representations from Transformers)Google's 2018 transformer language model that reads text in both directions to produce context-aware embedding…
- BERTopicA modular topic modelling technique that clusters transformer embeddings and labels each cluster with class-ba…
- BIRCH (Balanced Iterative Reducing and Clustering using Hierarchies)BIRCH (Balanced Iterative Reducing and Clustering using Hierarchies) clusters very large numeric datasets by s…
- BM25A classic ranking function that scores documents by how often query words appear, adjusted for word rarity and…
- Boyer-MooreA fast exact string-search algorithm that compares a pattern from right to left and skips ahead after mismatch…
- c-TF-IDFClass-based TF-IDF: treats each cluster as one document to find the words that best describe it, used by BERTo…
- Centroid-based ClusteringA clustering family that groups data around central points (centroids), each the average of its cluster's memb…
- Convolutional Neural Network (CNN)A deep learning architecture that scans data with small learned filters to detect local patterns; the classic…
- DBSCANDensity-based clustering that groups closely packed points, finds the number of clusters itself and labels iso…