Topic Modelling for SEO with BERTopic

If LDA is the classic that taught search to think in topics, BERTopic is the modern successor that does it with the semantic muscle of transformer embeddings. For most of the text-heavy work SEOs actually do, clustering keywords, reviews, titles and content, it is the approach I reach for first. This post explains what it is, how it works, walks through the actual code so you can run it yourself, and gives you the honest benefits and limitations so you know exactly when it is the right tool.

What BERTopic is

BERTopic is a topic modelling technique that uses BERT embeddings and class-based TF-IDF to create dense, easily interpretable clusters while keeping the most important words in each topic’s description. Its defining feature is that it is built like Lego: you can pick and choose the model for each step, tokenise one way, cluster with HDBSCAN or K-means or BIRCH, swap in a model from Hugging Face. That modularity is what makes it far more versatile than LDA.

How BERTopic works

Like any topic model, it ends up identifying sets of words that form coherent themes. Under the hood, it reverse-engineers those themes through a clean pipeline.

It converts text into 384-dimensional BERT embeddings that capture semantic relationships, uses UMAP to reduce those to a low-dimensional space, applies HDBSCAN to find dense clusters (each a potential topic), and then uses class-based TF-IDF to surface the most representative words per topic, refined with MMR for relevance. You can feed it your own topic labels or none at all, in which case it discovers topics automatically. In short, it is an embedding-based, density-based, soft/fuzzy, unsupervised model. Recent versions from its creator, Maarten Grootendorst, add zero-shot topic modelling, model merging and large language model support, which keep it at the front of the field.

Running BERTopic yourself: the code

Here is the part I want you to actually try. BERTopic is refreshingly little code to get going. In the course you get the fully annotated Colab notebook, the demo dataset and a ready pipeline, but there is no reason you cannot work through the essentials on your own right now. Start with the simplest possible run on a list of documents (your pages, reviews, titles, whatever your corpus is).

!pip install bertopic

from bertopic import BERTopic

docs = df["content"].tolist()          # your pages, reviews or titles as a list of strings

topic_model = BERTopic(language="english", min_topic_size=10)
topics, probs = topic_model.fit_transform(docs)

topic_model.get_topic_info()           # the topics it discovered

That is genuinely all you need for a first pass. The magic, and the reason I love this model, is in the “build your own Lego” step, where you swap the embedding model, the dimensionality reduction and the clustering to suit your data.

from sentence_transformers import SentenceTransformer
from umap import UMAP
from hdbscan import HDBSCAN
from sklearn.feature_extraction.text import CountVectorizer

embedding_model = SentenceTransformer("all-MiniLM-L6-v2")     # 384-dim embeddings
umap_model      = UMAP(n_neighbors=15, n_components=5, metric="cosine")
hdbscan_model   = HDBSCAN(min_cluster_size=15, metric="euclidean", prediction_data=True)
vectorizer      = CountVectorizer(stop_words="english", ngram_range=(1, 2))

topic_model = BERTopic(
    embedding_model = embedding_model,
    umap_model      = umap_model,
    hdbscan_model   = hdbscan_model,
    vectorizer_model= vectorizer,
    calculate_probabilities = True,
    verbose = True,
)
topics, probs = topic_model.fit_transform(docs)

Every one of those blocks is optional and swappable. Prefer K-means because you want a fixed number of topics? Pass it as the hdbscan_model. Working in another language? Swap the sentence transformer. That is the flexibility the slides call “building your Lego”, and it is what puts BERTopic a level above LDA.

What the output actually looks like

Once it has run, you inspect and visualise. In the course demo we deliberately use the same small BBC news dataset (about 100 articles) that we used for LDA and text classification, so you can compare the three approaches directly. BERTopic pulls out six coherent topics from it, family care and support, climate change, a Trump/election topic, and the Ukraine-Russia war among them.

topic_model.get_topic_info()
#    Topic  Count  Name
# 0     -1     11  -1_the_to_of_and                 <- outliers, no clear topic
# 1      0     28  0_ukraine_russia_putin_war
# 2      1     19  1_climate_emissions_energy
# 3      2     15  2_trump_election_republican
# 4      3     14  3_family_care_support_children

topic_model.get_topic(0)
# [('ukraine', 0.041), ('russia', 0.033), ('putin', 0.028), ('war', 0.021), ...]

Notice topic -1: that is the outlier bucket, documents HDBSCAN could not confidently place. It is normal, and it is one of the trade-offs we will get to. From here you visualise, reduce the topic count if it over-produced, and export each document’s topic.

topic_model.visualize_topics()      # interactive inter-topic distance map
topic_model.visualize_barchart()    # top words per topic
topic_model.visualize_hierarchy()   # see which topics could merge

topic_model.reduce_topics(docs, nr_topics=20)   # rein in an over-produced list

df["topic"] = topics                # tag every document
df.to_csv("content_with_topics.csv", index=False)

That CSV, every page or review tagged with its dominant topic and the representative words, is the artefact you actually take into an internal-linking or content-audit project. A quick, important caveat: this is a very limited demonstration of what BERTopic can do. If you are going to use it seriously, the creator’s GitHub repository is a must-read, it is full of tips on scalability, coherence and customising the pipeline.

BERTopic versus LDA and the others

Choosing a topic model is really about your data and your constraints. Here is how the main options compare.

The headline differences: BERTopic and Top2Vec need minimal preprocessing and detect the number of topics automatically, which makes them strong on short texts like reviews, tweets and titles, exactly the data we tend to have. LDA and NMF need more preprocessing and you must predefine the topic count. And BERTopic is usually praised for producing the clearest, most coherent topics, helped by its LLM integration for natural-sounding labels. Whichever you pick, human judgement still matters for validating the topics.

The honest benefits and limitations

I am a fan, but BERTopic is not magic, and knowing where it struggles will save you a bad analysis.

On the benefits side: high topic quality from contextual embeddings, minimal preprocessing, automatic topic count, excellent handling of short text, and clear keyword-labelled topics that align with how a human would group the content. On the limitations side: it is computationally heavy and often wants a GPU; it is weak on small datasets (under about 1,000 documents, like our 100-article demo, it will still form topics but not perfectly coherent ones); it can over-produce topics (feed it 1,000 documents and you might get 200 to 500 topics unless you cap them with reduce_topics or nr_topics); it leaves some documents unassigned as outliers (that topic -1 above); and its biggest real-world catch, in my experience, is that it assigns only one topic per document, which rarely matches how content is actually written.

That last point is exactly why I teach both approaches. BERTopic gives you crisp, coherent top-level topics; LDA, with its soft membership, is better at surfacing the multiple subtopics inside a single document. Run both and marry them and you get a far better picture for something like content categorisation than either gives alone.

Turning topics into SEO wins

The applications mirror the LDA post, because the output serves the same goals: improve internal linking by linking articles in the same topics and subtopics; enhance it with entity extraction to enable entity-based modular linking; run competitor analysis to understand rival content libraries fast; and do market analysis by clustering YouTube or Reddit content to read a competitive landscape quickly. My standing challenge: on your next content audit, run BERTopic, LDA and Google’s text classification on the same set and see which gives the most useful labels.

Where this goes next

Read the companion post on topic modelling with LDA to see the classic approach BERTopic builds on, and what clustering is and where it fits in SEO for the wider context. For hands-on keyword work, our KeyBERT keyword clustering guide is a great next step.

Related glossary terms

• BERTopic
• BERT
• HDBSCAN
• Dimensionality reduction
• Topic modeling