F1 Score

The F1 score measures how well a classification model performs by combining two numbers: precision (of the items the model labelled positive, how many were right) and recall (of all the truly positive items, how many the model found). It ranges from 0 to 1, and it is high only when both precision and recall are high.

How it works

F1 is the harmonic mean of precision and recall: F1 = 2 × (precision × recall) / (precision + recall). The harmonic mean punishes imbalance. A model with precision 0.9 and recall 0.1 gets an F1 of 0.18, while a simple average would misleadingly say 0.5.

For tasks with more than two classes there are several ways to combine per-class scores:

• Macro-F1: calculate F1 for each class and average them equally, so rare classes count as much as common ones.
• Weighted F1: average per-class F1 weighted by how many examples each class has.
• Micro-F1: pool all decisions first; for single-label tasks this equals accuracy.

In scikit-learn, f1_score(y_true, y_pred, average="macro") computes it in one line, and classification_report shows precision, recall and F1 for every class.

How it’s used in SEO and marketing

• Evaluating intent, sentiment or page-type classifiers against a hand-labelled sample.
• Comparing NLP APIs, open-source models and LLMs on the same data before choosing one.
• Benchmarking entity extraction, where standard tests such as CoNLL-2003 report precision, recall and F1.
• Checking imbalanced tasks where accuracy misleads, such as spotting the few spam reviews among thousands.

Example

In MLforSEO’s sentiment benchmark on 2,000 labelled tweets, the methods scored: VADER 0.52 macro-F1, AFINN 0.53, TF-IDF with logistic regression 0.58 and a fine-tuned Twitter RoBERTa transformer 0.71. Even the best model was wrong about three times in ten, a reminder to look at patterns across many items instead of trusting single labels.

Limitations and things to watch

• F1 treats false positives and false negatives as equally costly. If one is worse for your business, look at precision and recall separately or use an F-beta score.
• Always report which average (macro, weighted, micro) you used.
• A score is only as good as the labels it is measured against, and only valid on data similar to your test set.

Learn it on MLforSEO

• Sentiment Analysis: A Deep Dive Into the Theory, Methods, and Applications
• Comparing NLP APIs for Entity and Sentiment Analysis (vs Generative AI)
• What Machine Learning Classification Is, and Where It Fits in SEO
• Introduction to Machine Learning for SEO