Inference

Inference is the stage where a trained model is put to work. Training is when the model learns from data; inference is when it receives new input and produces an output, such as a sentiment score for a review, an intent label for a keyword, or an answer from an LLM. Every time you send a prompt to ChatGPT or call an NLP API, you are running inference.

How it works

The model’s parameters are fixed during inference; it applies what it learned to the input in front of it. For LLMs, the prompt and any retrieved documents are converted to tokens, loaded into the context window, and the model generates output one token at a time. Settings such as temperature change how predictable that output is.

Inference has a cost in compute, time and money, usually charged per token or per request. MLforSEO’s agentic SEO material points out that this cost shapes AI search: systems prefer sources that are cheap to verify, which is one reason clean structured data and clear, self-contained passages are favoured over long, ambiguous prose.

How it’s used in SEO and marketing

• Batch inference: running a trained or pre-trained model over thousands of keywords, pages or reviews in Colab or Google Sheets.
• Real-time inference: on-site search, chatbots and recommendations that respond as users interact.
• Cost planning: estimating API spend before running an LLM over a full site.
• Understanding AI search: retrieval and trust checks happen at “inference time”, when the system can cross-check your claims against other sources it has just retrieved (RAG).

Example

You fine-tune a small classifier on 500 labelled queries (training). You then run it over 50,000 Search Console queries to label each as informational, commercial or navigational (inference). Training took an hour once; inference runs every month on new data in minutes, and its cost scales with the number of queries.

Limitations and things to watch

• A model can only infer from what it learned and what it is given; it cannot know facts outside its training data or context.
• LLM inference is probabilistic, so the same prompt can give different answers.
• Latency and cost grow with input length, which is why long, unfocused pages are less attractive to AI systems.
• In statistics, “inference” also means drawing conclusions about a population from a sample; in ML and AI it means running a model.

Learn it on MLforSEO

• Trust isn’t a schema field: how machines decide to believe you
• Your audience is no longer only human
• How modern search actually works: the five-stage retrieval funnel
• AI Search Optimisation & Agentic SEO (course)
• Introduction to Machine Learning for SEO (course)