What is Retrieval-Augmented Generation (RAG)?
Retrieval-Augmented Generation: A pattern where the system first retrieves relevant documents from your data, then passes them to an LLM so its answer is grounded in your sources.
Retrieval-augmented generation combines search with generation. When a question comes in, the system searches your content (help articles, policies, contracts, tickets) for the most relevant passages, adds them to the prompt, and asks the model to answer using only that material, often with citations. The model supplies language and reasoning; your documents supply the facts.
A typical RAG pipeline splits documents into chunks, converts each chunk into an embedding, stores them in a vector database or search index, and at query time retrieves the best matches, often combining semantic search with keyword search and a re-ranking step. Quality depends mostly on retrieval: if the right passage is not retrieved, the model cannot use it, and it may guess instead.
Why it matters when you build or buy AI
RAG is the default way to build an assistant that knows your company, and it is usually a better first step than fine-tuning because you can update the knowledge by editing documents. Most RAG failures come from messy source content, poor chunking, and no evals on retrieval quality. Ask any vendor how they will measure whether the right documents were found, not just whether the answer sounds good.
Example
A staff assistant answers "what is our refund window for wholesale orders" by retrieving the wholesale policy page and quoting the relevant clause with a link.
Related terms
More in AI & LLMs
Book 20 minutes, or send a brief
A former CTO replies. You leave with a written plan and a fixed estimate.
Prefer LinkedIn? Connect with Harish.