Chunking (Content Chunking)

Chunking is the step where a retrieval system cuts a document into smaller passages before indexing it. Each chunk gets its own Embedding and is scored on its own, so an AI answer engine usually retrieves and quotes chunks, not whole pages. MLforSEO sums it up as: the page is read by humans, the chunk is retrieved by machines.

How it works

RAG pipelines typically split text by a fixed number of tokens, by sentences or paragraphs, or by layout (headings and lists), often with some overlap between chunks. Google does not publish how Search chunks pages, but its Vertex AI Search product documents layout-aware chunking that detects headings and can add ancestor headings to each chunk.

MLforSEO’s AI Search Optimisation & Agentic SEO course works with three numbers: roughly 500 tokens per chunk (about 350 to 400 words), three to eight chunks passed to the generator per query, and zero context carried about what came before or after on the page. That last number is the problem: “as noted above” or an “it” with no antecedent cannot be resolved once the chunk is extracted.

How it’s used in SEO and marketing

• Writing sections that make sense on their own, with the subject named and the claim stated in full.
• One claim per headed section, so each chunk produces a sharp vector that matches one sub-query well. A paragraph covering problem, cause and solution blends three topics into one vague vector.
• Using headings that mirror the question a user would type (“What does a stockout cost a retailer?”) instead of “A closer look”.
• Keeping attribution inside the body: sources, dates and the brand name, since the byline and footer are not part of the chunk.
• Chunking your own content to build site search, internal-linking tools or a support assistant.

Example

A chunk reading “CRM vendor lock-in occurs when migration costs exceed the cost of staying on a suboptimal platform; for mid-market companies migration costs forty to one hundred and twenty thousand euros and takes six to nine months” is self-contained and quantified, so it survives. A competing chunk that opens “as discussed earlier” cannot be resolved and is dropped. The MLforSEO test: open your content at any random 400-word window and check you can still tell what is being claimed.

Limitations and things to watch

• Chunk sizes and methods differ between systems and change over time; treat the course figures as working guidelines.
• Writing for chunks does not mean writing thin content. Depth stays; it is the structure that changes.
• In general NLP, “chunking” can also mean grouping words into phrases (shallow parsing). This entry covers content chunking for retrieval.

Learn it on MLforSEO

• The chunk, not the page: content an AI can actually extract
• How modern search actually works: the five-stage retrieval funnel
• Blog and Site Architecture for SEO and Conversions in the AI Search Era
• AI Search Optimisation & Agentic SEO
• Extraction Readiness Audit