Partner With Us

← All lessons / Building blocks / Lesson 05 of 12

Interactive lesson

Build a RAG system: give an AI the right facts at the right time.

Retrieval-augmented generation lets a model answer from your own documents instead of its memory. Build one step by step, and see what happens when retrieval goes wrong.

You’ll learn to
  • Explore how passage size and retrieval choices change the available evidence.
  • Spot answers that are unsupported by their sources.

Why models need retrieval

A language model's knowledge stops at the date its training data was collected, and it has never read your charity's policies. Ask it about your expenses rules and it will produce a plausible guess. Retrieval-augmented generation (RAG) fixes this by finding the relevant passages first and handing them to the model along with the question.

DocumentsChunksRetrieveAugment promptGenerate + cite

Here is how it works. First, the documents are split into short passages called chunks. When a question arrives, the system fetches the few chunks that match it best (how many it fetches is called top-k) and adds them to the prompt. The model then answers from those passages and cites them. An answer based on the sources like this is called grounded.

About this lab: the five documents belong to a fictional charity, the Riverside Community Health Trust. Splitting and searching run for real in your browser, using a standard keyword-scoring method called BM25. The final answers were written by our team to illustrate grounded answers, refusals and unsupported guesses. This is a demonstration, not an experiment showing that RAG always succeeds or that a model without retrieval always guesses.

Interactive

Build the pipeline, one stage at a time

1. Choose a volunteer's question.

Try this: ask about mileage with chunk size 1, then 2. Then ask who to report lost data to: which chunk size lets the answer name the data protection lead?

Key ideas to take away

  • RAG grounds answers in your documents. The model answers from retrieved text, and can cite it, instead of guessing.
  • Check the citations, not just the answer. A model can cite a source that doesn't actually support its claim. Spot-check that each cited passage says what the answer says it does.
  • Retrieval quality limits answer quality. If the right passage isn't retrieved, even the best model can't use it.
  • Chunking is a design decision. Chunks that are too small lose context; chunks that are too large bury the useful sentence among irrelevant ones and fill up the context window.
  • Tell the model what to do when the answer isn't there. “Say you can't find it” is far safer than a confident guess.
  • Mind what goes in. Only add documents to the search that people using the tool are allowed to see.

Check your understanding

3 quick questions. Try as often as you like. Your progress stays on this device; answers clear when you reload.

Want to go deeper?Equations, code and research for technical learners. Optional.

BM25, the scoring used in the lab

score(chunk, query) = Σ over query terms t:
    IDF(t) · tf(t) · (k1 + 1) / ( tf(t) + k1 · (1 − b + b · len / avg_len) )

IDF(t) = ln( 1 + (N − n_t + 0.5) / (n_t + 0.5) )        k1 = 1.2, b = 0.75

Rare words that appear in a chunk score highly; very common words score little; longer chunks are slightly penalised. It's fast and easy to inspect, but it only matches words, which is why “driving” doesn't find “car journeys”. Production systems usually add embedding search (Lesson 04) and a re-ranker that re-scores the top candidates.

A minimal RAG pipeline in Python

import chromadb

client = chromadb.Client()
docs = client.create_collection("policies")          # uses a default embedding model
docs.add(ids=["exp-1", "exp-2"],
         documents=["Mileage is paid at 45p per mile for car journeys.",
                    "Claims must be submitted within 30 days."])

question = "How much can I claim for driving to my shift?"
hits = docs.query(query_texts=[question], n_results=2)["documents"][0]

prompt = ("Answer using only the sources. Cite them like [1]. "
          "If the answer is not in the sources, say so.\n\n"
          + "\n".join(f"[{i+1}] {h}" for i, h in enumerate(hits))
          + f"\n\nQuestion: {question}")
# send `prompt` to the language model of your choice

How to measure a RAG system

  • Retrieval: recall@k measures the fraction of all relevant passages retrieved in the top k, averaged across test questions. If each question has one relevant passage, it becomes the fraction of questions where that passage is retrieved.
  • Generation: faithfulness asks: is every claim supported by the retrieved text? Answer relevance asks: does it address the question?

Lesson 07 covers evaluation in depth. The original RAG paper: Lewis et al. (2020), “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks” (opens in a new tab).

Continue exploring

We've picked resources that explain these ideas from different angles. They are independent of Code With Impact.

Sources for this lesson

The Riverside Community Health Trust is fictional, and all documents, questions and answers in this lesson are original. BM25 retrieval runs in your browser using the standard formula.

How we reference: we link and credit, we never copy other educators' text, diagrams or code. See our full Sources & credits register.