insideout Get Pro

08 · Order effects in long prompts

Pro

Prompting, lesson 8

Order effects in long prompts

Where you put information changes whether the model uses it.

You've read "A budget for the context window", so you know what to put in a prompt. This lesson is about where it goes.

Part 1. Same passage, different place

In 2023 Nelson Liu and colleagues ran a simple test. Each prompt held a question and 20 Wikipedia passages. Exactly one passage contained the answer; the other 19 were on related topics but didn't. They moved the answer passage from first place, through the middle, to last, and measured how often models answered right.

Your turn

Where did the models do worst?

Accuracy made a U shape: best with the answer passage first, good with it last, worst in the middle. With 20 passages, GPT-3.5-Turbo did worse with the answer in the middle than when it was given no passages at all and answered from memory (about 56%). The answer was right there in the prompt, and it hurt.

Lost in the middle

The drop in accuracy when the needed information sits in the middle of a long prompt instead of near the start or end. Named after that 2023 paper. The pull toward the start is called primacy bias, and the pull toward the end recency bias.

The same paper found that versions of a model sold with longer windows weren't better at using a prompt that fit in both. A bigger window means more fits in, not that more gets used well.

Part 2. Newer models, smaller dip

Keep going with Pro

You've done the first part. Pro opens the rest of this lesson, and every other one.

Still to come
  1. Newer models, smaller dip
  2. Put the important parts at the edges
See Pro plansI have Pro: sign in
Next lessonFitting more into the windowSummarise, extract or retrieve instead of pasting everything. Start lesson 9

Prompting overviewSpotted a mistake? Tell us.