Search and Retrieval, lesson 14
Capstone: a search-backed assistant
Build and measure an assistant that answers from your documents.
This walkthrough uses the earlier lessons in this module, from "Keyword search with TF-IDF and BM25" to "Why grounded answers still go wrong". Nothing new is introduced; it shows how the parts meet in one system.
Part 1. The system and its handbook
A company with offices in the UK, the US and Germany runs a Q&A assistant over its staff handbook: about 400 pages of policies, some shared, some per office, a few readable only by HR. Here is how that handbook became an index, and what each choice was for.
Example 1.1
Indexing the handbook
- 400 pages→recursive split→about 2,000 chunks
Cut on headings, then paragraphs, then sentences, to about 300 tokens each with a little overlap. Tables are kept whole. ("Chunk size and overlap")
- "Leave > Parental leave (Germany):..."office DEupdated 2025-03group all-staff
Each chunk starts with its document and section heading, and carries metadata: office, last updated, who may read it. ("Chunk size and overlap", "Filters and metadata")
- each chunk→BM25 indexeach chunk→embedding→vector index
Two indexes from the same chunks: one by words, one by meaning. ("Keyword search with TF-IDF and BM25", "Vector search")
- 2,000 vectors→exact search
2,000 vectors is small, so the vector side checks every one. No ANN index needed yet. ("Indexes at scale with HNSW")
Your turn
HR replaces the 2023 parental leave page with a 2025 version. What has to happen in the index?
Part 2. One question, all the way through
Keep going with Pro
You've done the first part. Pro opens the rest of this lesson, and every other one.
- One question, all the way through
- Measuring it
- Where it fails, and how you'd see it
Search and Retrieval overviewSpotted a mistake? Tell us.