Confusion Matrix

A confusion matrix is a table that shows how a classification model’s predictions compare with the correct answers. Each row stands for the true class and each column for the predicted class (or the other way round, depending on the tool), so every cell counts how often one class was predicted as another. It is the starting point for almost every classification metric, including accuracy, precision and recall and the F1 score.

How it works

For a binary classifier the matrix has four cells:

• True positives (TP): items that belong to the class and were predicted as it.
• False positives (FP): items predicted as the class that do not belong to it.
• False negatives (FN): items that belong to the class but were missed.
• True negatives (TN): items correctly left out of the class.

From these, accuracy is (TP + TN) divided by all items, precision is TP divided by (TP + FP), and recall is TP divided by (TP + FN). For multi-class problems, such as sorting keywords into four intent groups, the table grows to one row and one column per class. The diagonal holds the correct predictions, and everything off the diagonal shows which classes the model confuses.

You build it on a held-out test set with ground truth labels, never on the data the model learned from (see train/test split). In Python, scikit-learn’s confusion_matrix function and classification report produce it in one line.

How it’s used in SEO and marketing

• Checking a search intent classifier before you trust its labels across a whole keyword universe.
• Evaluating MLforSEO’s synthetic vs user query classifier, which reports a classification report and confusion matrix after an 80/20 train-test split.
• Comparing tools for sentiment or content categorisation against a hand-labelled sample.
• Choosing which error is cheaper. For spam-style filtering a false positive (hiding a real lead) may cost more than a false negative.

Example

Imagine a model that flags transactional queries, tested on 200 hand-labelled keywords, 50 of which are truly transactional. It predicts 48 as transactional: 40 correctly (TP) and 8 wrongly (FP). It misses 10 transactional queries (FN), and correctly leaves out the other 142 (TN).

• Accuracy: (40 + 142) / 200 = 91%.
• Precision: 40 / 48 = 83%, so about one in six transactional labels is wrong.
• Recall: 40 / 50 = 80%, so one in five transactional queries is missed.

The headline accuracy looks strong, but the matrix shows where the errors sit, and the off-diagonal cells tell you which queries to inspect next, for example commercial comparisons being mistaken for transactional ones.

Limitations and things to watch

• With imbalanced classes, accuracy can look good while the rare class is mostly missed. Read precision and recall per class from the matrix.
• Axis conventions differ: scikit-learn puts true labels in rows and predictions in columns, while some textbooks flip it. Check the labels before reading it.
• The matrix is only as good as the labels it is compared against. Noisy or inconsistent ground truth makes every number misleading.
• A small test set gives unstable counts, so cross-validation helps.

Learn it on MLforSEO

• What Machine Learning Classification Is, and Where It Fits in SEO
• Search-Intent Classification with Machine Learning: The Approaches Explained
• Synthetic vs User Query Classification Notebook with Rule-Based and Machine Learning Detection
• Introduction to Machine Learning for SEO