Transfer Learning

Transfer learning is reusing what a model learned on one task or dataset as the starting point for another. Instead of training from scratch, you take a pre-trained model, such as BERT or an image model trained on millions of photos, and adapt it to your own problem with far less data and compute. Most of the practical NLP tools marketers use today depend on it.

How it works

There are two stages. First, a large organisation pre-trains a model on a broad dataset, where it learns general patterns such as how language is structured or what edges and shapes look like in images. Second, you transfer that knowledge to your task, in one of three common ways:

• Use it as it is. A pre-trained model applied directly, for example a transformer used for zero-shot intent classification, which picks an intent from labels you supply without any training on your data.
• Feature extraction. Keep the pre-trained model frozen and use its outputs, such as embeddings, as input to a simpler model. MLforSEO’s image clustering lesson loads a pre-trained network like ResNet50, removes its final classification layers and clusters the feature vectors it produces.
• Fine-tuning. Continue training the pre-trained model on a small labelled dataset from your domain. The original BERT paper describes adapting the pre-trained model with just one additional output layer for a wide range of tasks.

The efficiency gain is the point. The ULMFiT paper (2018) reported that with only 100 labelled examples, its fine-tuned language model matched the performance of training from scratch on 100 times more data in its text classification tests.

How it’s used in SEO and marketing

• Search intent classification: MLforSEO’s comparison of approaches found the zero-shot transformer fast but less accurate out of the box, and notes that fine-tuning improves it.
• Entity extraction: for niche vocabularies, a light fine-tune with spaCy often gets you most of the value of a custom model for a fraction of the effort.
• Meta descriptions and summaries: extractive summarisation with pre-trained BERT models, and newer sentence embedding models, reuse language knowledge learned elsewhere.
• Image and product clustering: pre-trained vision models turn product photos into features you can group.

MLforSEO’s framework for choosing a model puts this in context: pre-trained models for most marketers, self-trained only with a unique domain and lots of data, and fine-tuned models in between.

Example

A SaaS company wants to sort blog search queries into its own six intent labels. Training a classifier from nothing would need a large labelled set. Instead the team labels a few hundred queries, fine-tunes a small BERT-family model with Hugging Face Transformers, checks it on held-out queries and compares it with the zero-shot baseline before using it on the full list.

Limitations and things to watch

• Domain gap. General models can mislabel niche terms; MLforSEO’s entity extraction guide shows terms like “SEO research” classified as “other” because they were not prominent in the training data.
• Inherited bias. Whatever skew is in the pre-training data carries over into your model.
• Small datasets overfit. Always test on examples the model did not see during fine-tuning.
• Tools change. Google stopped training and updating Vertex AI AutoML text models on 15 September 2024 and pointed users to Gemini prompting and tuning instead, so older no-code fine-tuning tutorials may no longer work.

Learn it on MLforSEO

• The Characteristics of a Machine Learning Problem: A Task, Data and Solution Framework for SEOs
• Search-Intent Classification with Machine Learning: The Approaches Explained
• What Entity Extraction Is, and Where It Fits in SEO
• How to Automatically Generate Meta Descriptions with Python: From BERT to LLMs
• Introduction to Machine Learning for SEO (course)