Data Preprocessing (Cleaning)

Data preprocessing, often called data cleaning, is everything you do to raw data before it goes into a model or analysis. It removes errors, duplicates and noise, puts values into consistent formats, and converts the data into a shape the algorithm can work with. Every practical in MLforSEO’s courses has a preprocessing step, because the quality of the output depends on it more than on the choice of algorithm.

How it works

Typical steps depend on the type of data:

• General: remove duplicates and empty rows, fix encodings, standardise case and date formats, handle missing values (drop, fill or flag), and filter out rows that are out of scope.
• Text: strip HTML, URLs, emails and boilerplate; lowercase; remove punctuation and digits where they add nothing; remove stop words (including site-wide words such as your brand name, “menu” or “cookie”); tokenise; and apply lemmatisation or stemming.
• URLs: extract the slug or path, remove protocols, parameters and trailing slashes, and replace hyphens with spaces before comparing.
• Numbers: scale or normalise columns so large values do not dominate distance-based methods such as k-means, and deal with outliers.

Modern embedding models need lighter text cleaning than older methods such as LDA or TF-IDF, because they use context. Removing stop words before creating sentence embeddings can even hurt results.

How it’s used in SEO and marketing

• Topic modelling and clustering of page content, where navigation text and boilerplate otherwise become the “topics”.
• Redirect and 404 mapping, where cleaned URL slugs give far better fuzzy matches than raw URLs.
• Keyword research, where merging exports from several tools needs deduplication, normalised spelling and consistent intent labels.
• Customer and analytics data, where scaling and missing-value handling decide whether segments make sense.

Example

Before running LDA on a blog crawl, MLforSEO’s topic modelling code strips emails and URLs, removes standard stop words plus site-wide words like “cookie”, “menu” and the brand name, then tokenises and lemmatises the text. Without that step, “cookie” and “subscribe” would appear in nearly every topic and the coherence score would fall.

Limitations and things to watch

• Over-cleaning removes signal: dropping numbers, negations or short words can change meaning.
• Keep a copy of the raw data and document each step, so results can be reproduced and checked.
• Apply exactly the same preprocessing to new data that you applied to training data.
• Cleaning cannot fix data that was wrong or biased at collection.

Learn it on MLforSEO

• Topic Modelling for SEO with LDA (Latent Dirichlet Allocation)
• Topic Modelling for SEO with BERTopic
• 404 and Redirect Mapping with Fuzzy Matching in Google Colab (Python)
• ML Data Characteristics Mapped to SEO & ML Tasks (Guide)
• Introduction to Machine Learning for SEO (course)