RAG: Giving AI Access to External Knowledge
Put the pieces together: retrieve the right passages, add them to the prompt, and let the model answer from them.
You now have every part of the machine. Time to switch it on. RAG, short for retrieval-augmented generation, is exactly what the name says, in three steps:
- Retrieve. Embed the question and fetch the few most relevant passages from your vector database.
- Augment. Put those passages into the prompt, labelled, with instructions on how to use them.
- Generate. The model answers the question, reading from the passages rather than from memory.
It turns a closed-book exam into an open-book one. Instead of recalling a vague memory of “what companies usually do”, the model reads your actual handbook, which is current, private and exact. Better still, because each passage carries an id, the model can cite it: “27 days [§2.1]”. Users can check the source, and you can trace every answer back to a document.
The instructions matter as much as the retrieval. A typical RAG prompt says something like:
- Answer only from the sources below. Without this, the model happily mixes in its own general knowledge.
- Cite the source of each claim.
- If the sources don’t contain the answer, say so. “The handbook doesn’t say” beats a confident guess.
RAG doesn’t make hallucinations impossible: retrieval can fetch the wrong passage, and models occasionally misread a right one. But it moves the problem somewhere you can see, measure and fix, which is what the rest of this track is about.
Key takeaways
- RAG = retrieve relevant passages, augment the prompt with them, generate an answer from them.
- It gives the model current, private, exact information, like an open-book exam.
- Labelled passages let the model cite sources, so answers can be checked.
- Instructions matter: answer only from the sources, cite them, and admit when the answer isn’t there.
Check your understanding
Go deeper (more technical)
The term RAG comes from a 2020 paper by Lewis and colleagues at Facebook AI Research, which combined a retriever and a generator trained together. Today the word usually means the simpler recipe here: an off-the-shelf embedding model, a vector database and a chat model, joined by a prompt.
RAG is a form of the context engineering from Track 5: deciding exactly what goes into the window. Retrieving too little misses the answer; retrieving too much costs money and buries it. Typical systems retrieve a handful of passages, a few thousand tokens in total.
The simulation’s handbook and answers are made up.