insideout Get Pro

09 · Answering from documents

Pro

Search and Retrieval, lesson 9

Answering from documents

Retrieve passages, then have the model answer from them.

"Keyword search with TF-IDF and BM25" to "Measuring search with recall, MRR and NDCG": finding the right passages, and checking that you did. This lesson hands them to a language model to answer from.

Part 1. The model hasn't read your documents

Say you work at a software company and a customer asks the support bot: "How long do enterprise customers have to ask for a refund?" The bot replies, confidently, "Most companies offer a refund window of 14 to 30 days."

Your company's handbook says 60 days, pro-rated. That page sits on an internal wiki. The answer was polite, fluent and wrong.

Your turn

Why did the bot get it wrong?

Nothing from your handbook ever reached this model. The page wasn't on the public web it was trained on, and the request carried only the question. So it filled the gap with what refund policies usually say. A bigger model or a lower temperature would give a smoother guess, but still a guess about other companies.

There are two ways to fix this. You could train the model further on your handbook, so the policy ends up inside its weights. Or, when the question arrives, you could search the handbook with everything the earlier lessons built, and paste the best passages into the prompt next to the question. The second is an open-book exam: the model doesn't need to have memorised the policy, only to read the page it's handed.

Part 2. Retrieve, then generate

Keep going with Pro

You've done the first part. Pro opens the rest of this lesson, and every other one.

Still to come
  1. Retrieve, then generate
  2. Building the prompt
  3. Keeping it current, and when RAG fits
See Pro plansI have Pro: sign in
Next lessonCiting sourcesMake every claim point to the passage it came from. Start lesson 10

Search and Retrieval overviewSpotted a mistake? Tell us.