Retrieval-augmented generation (RAG) is a technique in which a language model first retrieves relevant documents from an outside source, such as a search index, and then writes its answer using those documents together with what it learned in training. AI search features use it to ground their answers in current web pages, so an answer engine can only cite pages that its retrieval step found.
Where RAG comes from
RAG models were introduced in a 2020 paper by Patrick Lewis and 11 co-authors, "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks", first posted to arXiv on 22 May 2020 and accepted at NeurIPS 2020.
The paper paired two kinds of memory. The parametric memory was a pre-trained sequence-to-sequence model, BART. The non-parametric memory was a dense vector index of Wikipedia (the December 2018 dump, split into 100-word chunks for 21 million documents) searched by a pre-trained neural retriever, Dense Passage Retriever. One version used the same retrieved document for the whole answer; the other could use a different document for each token.
RAG set the state of the art on three open-domain question-answering tasks, and its text was more specific, diverse and factual than that of a comparable model without retrieval.
How RAG works in an answer engine
- The engine turns the question into one or more searches.
- A retriever pulls candidate pages or passages from an index.
- The model writes an answer from the question plus the retrieved text.
- The engine shows some of the pages it used as citations.
Google's guide to its generative AI features uses the term directly. It defines RAG, also called grounding, as relying on Google's core Search ranking systems to retrieve relevant, up-to-date pages from the Search index, to improve the quality, accuracy and freshness of AI responses. OpenAI describes the first step for ChatGPT search, which typically rewrites a question into one or more targeted queries; see query fan-out.
Why RAG matters for GEO
Retrieval comes before writing. A page the retriever never finds cannot shape the answer or be cited, however good it is, so crawler access and indexing come first; see what an AI crawler is.
Retrieval also keeps answers current. The original paper showed that swapping the index changes what the model knows without retraining it, and Google names freshness as one of RAG's purposes. A new or updated page can reach answers through retrieval instead of waiting for a new model. How to write content that AI engines cite covers what to do with that.
An example: swapping the index
The authors built a second index from a December 2016 Wikipedia dump and asked the model who held each of 82 world-leader posts that changed hands between 2016 and 2018. With the 2016 index it answered 70 percent of the 2016 questions correctly, and with the 2018 index 68 percent of the 2018 questions. With the indexes mismatched, accuracy fell to 12 and 4 percent. The authors conclude that replacing the index is enough to update what the model knows.
Common misconceptions
- "RAG answers come only from retrieved text." The original design combines retrieved documents with the knowledge stored in the model.
- "RAG removes errors." The paper found that RAG hallucinated less than a model without retrieval, not that it stopped. OpenAI warns that search results and citations can be incomplete, outdated or incorrect.
- "Every AI answer involves retrieval." ChatGPT, for example, may search the web automatically when a question would benefit from current information, which means some answers involve no web search at all.