ML Models & Algorithms
NMF (Non-negative Matrix Factorization)
In shortA topic modelling method that splits a document-word matrix into document-topic and topic-word weights, all non-negative, for readable topics.
Non-negative Matrix Factorization (NMF) is a technique that breaks a large table of numbers into two smaller tables whose values are all zero or positive. Applied to text, it splits a document-word matrix into one table describing how much each document belongs to each topic and another describing which words make up each topic. It is one of the classic topic modelling methods, usually compared with LDA and BERTopic.
How it works
- Build a document-word matrix, usually with TF-IDF weights.
- Choose the number of topics, k.
- NMF finds two matrices, W (documents by topics) and H (topics by words), whose product approximates the original matrix.
Because nothing can be negative, each document is built by adding topics together and each topic by adding words together. Lee and Seung’s 1999 paper in Nature described this as learning the “parts” of objects, and it is why NMF topics are often easy to read. scikit-learn’s NMF and Gensim’s Nmf both implement it.
How it’s used in SEO and marketing
- Topic modelling a site. Group pages into themes to audit coverage and overlap.
- Internal linking. Link pages that share a dominant topic.
- Review and feedback themes. Find the recurring subjects in large sets of reviews or survey answers.
- Benchmarking. Run it alongside LDA and BERTopic and keep whichever gives the most coherent topics for your data.
Example
MLforSEO’s LDA guide compares LDA and NMF on the same documents with eight topics, scored by c_v coherence. On full text, LDA scored 0.66 and NMF 0.65. On titles only, LDA dropped to 0.37 and NMF to 0.33. The lesson: neither is a clear winner on long text, both struggle on short text, and you should test both on your own data rather than relying on one study.
Limitations and things to watch
- You must set the number of topics in advance, which is hard when you do not know how many themes exist.
- It needs pre-processing (stop words, lemmatisation) and works on word counts, so meaning and word order are lost.
- Results can change with initialisation; fix a random seed for repeatable runs.
- Short texts such as titles, queries and tweets give weak topics; BERTopic and Top2Vec, which detect the number of topics automatically, tend to handle these better.
Learn it on MLforSEO
Where NMF (Non-negative Matrix Factorization) shows up on MLforSEO
Learn it in a course
Read about it on the blog
Sources and further reading
Other terms from ML Models & Algorithms
- BERT (Bidirectional Encoder Representations from Transformers)Google's 2018 transformer language model that reads text in both directions to produce context-aware embedding…
- BERTopicA modular topic modelling technique that clusters transformer embeddings and labels each cluster with class-ba…
- BIRCH (Balanced Iterative Reducing and Clustering using Hierarchies)BIRCH (Balanced Iterative Reducing and Clustering using Hierarchies) clusters very large numeric datasets by s…
- BM25A classic ranking function that scores documents by how often query words appear, adjusted for word rarity and…
- Boyer-MooreA fast exact string-search algorithm that compares a pattern from right to left and skips ahead after mismatch…
- c-TF-IDFClass-based TF-IDF: treats each cluster as one document to find the words that best describe it, used by BERTo…
- Centroid-based ClusteringA clustering family that groups data around central points (centroids), each the average of its cluster's memb…
- Convolutional Neural Network (CNN)A deep learning architecture that scans data with small learned filters to detect local patterns; the classic…
- DBSCANDensity-based clustering that groups closely packed points, finds the number of clusters itself and labels iso…