Keeping an agent's window clean

Agents, lesson 5

Summarise, trim and offload so long runs stay on track.

You know from "What an agent remembers" that the history is the agent's working memory, resent on every turn.

Part 1. How fast it fills

Put some example numbers on a coding agent. The system prompt, tool definitions and task take 6,000 tokens. Each step then adds about 300 tokens of reasoning, 100 for the tool call, and 4,000 for what the tool returns (a file, a log, some search results). That's 4,400 tokens a step, and every one of them is resent on every later turn.

Example 1.1

A 200,000-token window filling up

start
6,000of 200,000 (3%)
Before the first step: system prompt, tools and task.
after step 10
50,000(25%)
6,000 + 10 × 4,400. Already a quarter full, and the task has barely started.
after step 30
138,000(69%)
Most of this is old tool output the agent read once and won't need again.
after step 44
199,600
step 45→204,000: doesn't fit
Step 45 won't fit. Real agents often need more steps than that.

Your turn

The window isn't full until step 45. Is the agent fine until then?

Fitting isn't the same as being used well. Models recall facts from the middle of a long context less reliably than from the start or end (people call this "lost in the middle"). And because the whole history is resent each turn, the model call that picks step 40 reads about 178,000 tokens, against about 19,000 for step 4.

Show the answer

C. Fitting isn't the same as being used well. Models recall facts from the middle of a long context less reliably than from the start or end (people call this "lost in the middle"). And because the whole history is resent each turn, the model call that picks step 40 reads about 178,000 tokens, against about 19,000 for step 4.

Context window

The most tokens a model can read in one call, counting the system prompt, tools, history and the reply it's writing. For an agent, it's a budget that every step spends from.

Part 2. Trim at the source

Keep going with Pro

You've done the first part. Pro opens the rest of this lesson, and every other one.

Still to come

Trim at the source
Summarize old turns
Forget on purpose
Cleaning without losing the cache

See Pro plans I have Pro: sign in

Next lessonWhen a tool call failsRetries with backoff, and errors the model can recover from. Start lesson 6

Agents overviewSpotted a mistake? Tell us.

05 · Keeping an agent's window clean