Overfitting

Overfitting happens when a machine learning model learns its training data too well, memorising details and noise instead of the general pattern. The result is a model that looks excellent on the data it was trained on and disappoints on anything new. The opposite problem, underfitting, is a model too simple to capture the pattern at all.

How it works

Every dataset contains signal (the real relationship you want to learn) and noise (quirks, errors, one-off cases). A model with a lot of flexibility, such as a very deep decision tree or a large neural network trained on a small dataset, can fit the noise as well. You spot it by comparing performance on the training data with performance on data the model has never seen: a big gap is the warning sign.

The standard defences, which MLforSEO covers in its ML fundamentals material:

• Split your labelled data into training, test and validation sets. Validation data is held back during training to tune settings such as learning rate or model depth and avoid overfitting. See train/test split and cross-validation.
• Use more, and more varied, data.
• Simplify the model or constrain it, for example by limiting tree depth, through hyperparameter tuning.
• Use ensembles: a random forest combines many decision trees and takes the most common answer, which improves accuracy and reduces overfitting.

How it’s used in SEO and marketing

• Custom classifiers: a page-type or intent classifier trained on 200 hand-labelled pages from one section may fail on the rest of the site.
• Forecasting: a traffic model that fits last year’s curve perfectly may simply be memorising seasonality and one-off events.
• Topic and sentiment models on short, messy text: MLforSEO’s BERTopic post cites research that topic-level and word-level sentiment models are vulnerable to overfitting because of data sparsity in user-generated short texts.
• Deciding whether to self-train at all: if you don’t have enough data to fill training, test and validation sets, use a pre-trained or fine-tuned model.

Example

A team trains a model to predict which articles will earn backlinks. On the training set it is right 98% of the time; on next quarter’s articles it is right 60% of the time. Inspection shows it learned that articles by one author earned links, which was true of the training sample but not a general rule. Adding more varied data and removing the author feature closed the gap.

Limitations and things to watch

• Overfitting is not always obvious without a proper held-out test set, so never evaluate on training data.
• Testing many settings against the same test set can overfit to that test set too; keep a final untouched sample.
• Fine-tuning a pre-trained model on a tiny dataset can also overfit; watch validation scores as you train.

Learn it on MLforSEO

• The Characteristics of a Machine Learning Problem: A Task, Data and Solution Framework for SEOs
• What Machine Learning Classification Is, and Where It Fits in SEO
• Topic Modelling for SEO with BERTopic
• Introduction to Machine Learning for SEO