Small Language Model (SLM)

A small language model (SLM) is a language model compact enough to run cheaply, often on a laptop, a phone, in a browser or on a single modest server, instead of in a large data centre. There is no agreed size cut-off: vendors use the label for their smaller model families, such as Microsoft’s Phi models and the smaller sizes of Google’s Gemma. SLMs give up some breadth of knowledge and reasoning in exchange for speed, lower cost and the option to keep data on your own hardware.

How it works

An SLM is built the same way as a large language model, on the transformer architecture, with far fewer parameters. That makes inference faster and cheaper and lets the model fit in the memory of ordinary devices. Common tricks for making models smaller include distillation, where a small model learns to imitate a larger one (the approach behind DistilBERT), and quantisation, where weights are stored at lower precision; Google’s Gemma documentation lists quantised versions alongside the default ones.

A few examples from official pages, as of October 2026:

• Microsoft describes Phi as its family of small language models, created so developers can run AI directly on a device without cloud connectivity, released under the MIT licence.
• Google’s Gemma models come with open weights, and the smallest sizes are built for mobile, edge and browser deployment.
• Gemini Nano runs on Android devices without a network connection, and Chrome uses it for built-in APIs such as summarising, translating and language detection.

How it’s used in SEO and marketing

• High-volume classification: tagging thousands of keywords by intent or pages by topic, where per-call API costs add up.
• Private data: analysing customer reviews, support tickets or CRM notes without sending them to a third-party API.
• Local workflows: Screaming Frog can connect to Ollama as an AI provider, which lets you run open models on your own machine during a crawl.
• Task-specific models: an SLM fine-tuned on your labelled examples can match a much larger general model on one narrow job.

Example

An agency needs to label a few thousand keywords as informational, commercial, transactional or navigational every month. They run a small open-weight model locally with a short instruction and five examples per label, then compare a sample of its labels with ones a strategist assigned by hand. Where it agrees well, they keep it; queries it marks as unclear go to a larger model or a person.

Limitations and things to watch

• Less built-in knowledge, so more factual errors. Avoid asking an SLM for facts it has to remember; give it the text to work on.
• Weaker on long, multi-step reasoning and long documents.
• For classification at scale, MLforSEO’s own testing found purpose-built NLP models beat generative AI on consistency and confidence scores. Test before you switch.
• “Small” is a moving label, and licences differ between model families. Check the terms before commercial use.

Learn it on MLforSEO

• What Machine Learning Classification Is, and Where It Fits in SEO
• How to Automatically Generate Meta Descriptions with Python: From BERT to LLMs
• 10 Ways to Automate Internal Linking with AI, including local models via Ollama
• Introduction to Machine Learning for SEO