Skip to content
Browse courses

ML Models & Algorithms

Encoder Model

In shortA neural network that turns text or other input into numeric representations. In a Google patent, the encoder builds the knowledge used to answer questions.

Updated 24 Oct 2026· 2 min read

An encoder model is a neural network that reads an input, such as a sentence or a document, and turns it into a structured numeric representation that other systems can work with. The term appears in MLforSEO in two related senses: the encoder half of transformer models such as BERT, and the encoder component of a Google patent on answering questions with a knowledge graph.

How it works

In language models. Transformer encoders read all the words in a passage at once and produce an embedding for each token and often one for the whole passage. BERT and Sentence-BERT are encoder-only models. Their output is a vector, not generated text, which is why they are used for search, similarity, classification and clustering. Retrieval systems use them in two forms: bi-encoders embed queries and documents separately for fast matching, and cross-encoders read a query and a passage together to score relevance more precisely, often in a re-ranking step.

In Google’s question-answering patent. The patent “Natural language processing with an N-gram machine” describes a model with an encoder, a decoder and a programmer model. The encoder reads a body of text and outputs a knowledge graph made of short n-gram statements. The programmer model then turns a natural-language question into a small program that searches that graph for the answer. MLforSEO’s knowledge graph lessons describe this as a two-step process: the encoder side builds and maintains the knowledge, and the programmer side answers questions from it.

How it’s used in SEO and marketing

  • Embedding keywords, titles or pages with an encoder (for example sBERT) to cluster them or map them to topics.
  • Finding internal linking opportunities by comparing page embeddings.
  • Understanding re-ranking in AI search: cross-encoders often decide which retrieved chunks make the final shortlist, which rewards passages that are clear and self-contained.
  • Understanding zero-click answers: knowledge graph question answering explains why factual entity-attribute queries are answered directly in the results.

Example

Given the question “Who is Barack Obama married to?”, the patent’s approach does not search for pages containing those words. The programmer model produces a program that finds the entity Barack Obama in the encoded knowledge store and follows the spouse relationship, returning Michelle Obama. The quality of the answer depends on how well the encoder captured the facts in the first place.

Limitations and things to watch

  • A patent shows what Google has explored, not what runs in Search today.
  • Encoders have input length limits, so long pages are usually split into chunks before embedding.
  • Embeddings from different encoder models are not comparable; use the same model for everything you compare.

Learn it on MLforSEO

Sources and further reading

Other terms from ML Models & Algorithms

See all 55 ML Models & Algorithms terms →