Agents, lesson 1
Observe, decide, act
The loop an agent runs, and what the model sees on each turn.
"Letting the model call functions" in Prompting showed one tool call. An agent strings them into a loop that runs until the model can answer. This lesson follows one run turn by turn, with its messages and its token counts; you need a little Python and JSON.
By the end, you can:
Part 1. The model asks, your code runs
Follow one turn of the loop through its messages: the tool call and the tool result.
Ask a chat model "What would 3 nights at the cheapest hotel in Kyoto cost in US dollars?" and it can only guess. It can't search hotels or look up an exchange rate. So you offer it two tools your code can run, search_hotels(city) and convert(amount, from, to), and follow the first turn.
Example 1.1
The first turn, message by message
Your code sends the question and the two tool definitions to Anthropic's Messages API. Follow the turn until the model has the result it asked for. How many messages does the next request carry?
- Send the question1 message + 2 tool definitions
The user's message goes in
messages, the definitions in a separatetoolsfield. - Read the replystop_reason: "tool_use"
A
tool_useblock: the idtoolu_01, the namesearch_hotelsand the input{"city": "Kyoto"}. It is text in an agreed shape. - Run the toolsearch_hotels("Kyoto")
Your code looks the name up in its own table of functions. The model takes no part.
- Send the result back1 + 1 + 1 = 3 messages
Append the model's message as it came, then a user message with a
tool_resultcarrying the same id.
Answer3 messages: the question, the model's tool call and your tool result.
[{"role": "user",
"content": "3 nights, cheapest hotel in Kyoto, in USD?"},
{"role": "assistant",
"content": [{"type": "tool_use",
"id": "toolu_01",
"name": "search_hotels",
"input": {"city": "Kyoto"}}]},
{"role": "user",
"content": [{"type": "tool_result",
"tool_use_id": "toolu_01",
"content": "Hana Inn 12000 JPY; Kamo Stay 15500 JPY"}]}]The tool result travels as a user message, but no person wrote it: your code did. The id ties each result to the call that asked for it. Other APIs change the names, not the idea. OpenAI's Responses API sends a function_call with a call_id, and you answer with a function_call_output.
Tool call and tool result
A tool call is the model's request to run a tool: a name, JSON input and an id. A tool result is your code's answer, sent back with the same id. The model only writes the call. Your code decides whether to run it.
one turn = the model's tool call + your tool result
One reply can hold several tool calls, such as a hotel search and an exchange rate. Run them in any order, then answer them all in one user message, each result with its own id. Anthropic's API rejects the next request if any tool call is left without its result.
Your turn
A reply comes back with stop_reason "tool_use" and two tool calls: get_rate with id toolu_08 and search_hotels with id toolu_09. What should your next request add to the history?
Part 2. Round and round until it answers
Run the loop to the end, and tell a finished reply from a cut-off one by its stop reason.
One tool call isn't enough for the Kyoto question. The model needs the prices first, and only after reading them does it know what to convert. So your code repeats the turn: call the model, run what it asks for, append both messages, and call again.
Example 2.1
The whole Kyoto run
Run the loop on the Kyoto question to the end, with an example rate of 0.0065 US dollars to the yen. How many requests does your code send, and how many tools does it run?
- Request 1→ search_hotels("Kyoto")
Stop reason
tool_use. The search returns Hana Inn at ¥12,000 a night and Kamo Stay at ¥15,500. - Request 2→ convert(36000, "JPY", "USD")
Stop reason
tool_use. The model read the list: 3 nights at ¥12,000 is ¥36,000. The tool returns 234. - Request 3→ "About $234 for 3 nights."
Stop reason
end_turn. There's no tool call, so the loop stops and your code shows the answer.
Answer3 requests and 2 tools run. The run ends on the first reply with no tool call.
The model never sees the loop. Each request is a fresh call that reads the history so far, and the history is the only place the hotel prices live. ReAct, a 2022 paper, wrote each turn as Thought, Action, Observation. Agents still run that loop, and a tool result is often called an observation.
Agent loop and stop reason
The agent loop calls the model, runs any tool it asks for, appends the call and the result, and calls again. A model running in it with tools is an agent. Each reply's stop reason says why it stopped: tool_use, end_turn or max_tokens.
loop while the stop reason is tool_use, up to a step limit
Two traps. A reply can stop at max_tokens, the output limit, partway through a sentence or a call: no tool call, but not finished either. And a confused model can ask for tools for ever, so the loop needs a step limit. OpenAI's Chat Completions name the same three reasons tool_calls, stop and length.
Your turn
An agent's replies come back with these stop reasons, in order: tool_use, tool_use, tool_use, end_turn. Each tool_use reply held one tool call. How many requests did your code send, and how many tool results did it append?
tool_use reply held one call, and each call gets one result: 3. The end_turn reply is the answer, so nothing runs after it.Part 3. Every turn resends the history
Count the tokens each request sends as the history grows.
The provider keeps nothing between requests. So each one carries the lot: your system prompt, the tool definitions, the question, and every call and result so far. Measure the Kyoto run in tokens, with example sizes.
Example 3.1
The Kyoto run in tokens
The system prompt is 300 tokens, the two tool definitions 200 and the question 20. Each tool call is 30 tokens. The hotel list comes back as 400 tokens and the converted price as 10. How many input tokens does each request send?
- Request 1300 + 200 + 20 = 520
The fixed part, 500 tokens, and the question.
- Request 2520 + 30 + 400 = 950
Everything again, plus the first tool call and the hotel list.
- Request 3950 + 30 + 10 = 990
Plus the second call and its result.
- Add them up520 + 950 + 990 = 2,460
Input is billed on every request. The 500 fixed tokens were paid for three times, and the hotel list twice.
Answer520, 950 and 990 tokens: 2,460 input tokens to answer one question.
Each turn grows the next request by exactly what it added: the model's call and your result. A short call barely moves it. A long result moves it for the rest of the run, which is why later lessons trim both the results and the history.
Context
Everything the model reads in one request: the system prompt, the tool definitions and the history so far. In an agent it grows each turn by that turn's tool call and tool result, and it has to fit in the model's context window.
next request = this request + tool call + tool result
The fixed part opens every request unchanged, so providers can cache it and charge less for it on later turns ("Prefix caching" in Inference and Serving). Keep the start of each request stable, and put whatever changes after it.
Your turn
An agent's system prompt and tool definitions take 800 tokens, and the question 50. Every tool call is 40 tokens and every result 160. How many input tokens does the fourth request send?
Where you'll meet it
Asked how an agent works? Draw the loop: the model writes a tool call, your code runs it, appends the call and its result, and calls the model again with the whole history. Then name the two exits, a reply with no tool call or the step limit, and the cost: every turn resends everything so far.
Tools cost tokens before they're used. Whenever you send tools, Anthropic's API adds a tool-use system prompt of its own, 286 tokens for Claude Opus 5.5, and bills it with your definitions as input on every request. OpenAI's docs say function definitions count against the context limit and are billed as input tokens too.
def search_hotels(city):
return [("Hana Inn", 12000), ("Kamo Stay", 15500)]
def convert(amount, frm, to):
return round(amount * 0.0065) # example JPY to USD rate
TOOLS = {"search_hotels": search_hotels, "convert": convert}
def fake_model(history): # stands in for the API call
results = [m["content"] for m in history if m["role"] == "tool"]
if not results:
return {"tool": "search_hotels", "input": {"city": "Kyoto"}}
if len(results) == 1:
nightly = min(price for _, price in results[0])
return {"tool": "convert",
"input": {"amount": 3 * nightly, "frm": "JPY", "to": "USD"}}
return {"answer": f"About ${results[1]} for 3 nights."}
history = [{"role": "user", "content": "3 nights, cheapest hotel in Kyoto, in USD?"}]
for step in range(1, 9): # the step limit
reply = fake_model(history)
if "answer" in reply:
print("final:", reply["answer"])
break
result = TOOLS[reply["tool"]](**reply["input"])
print(f"step {step}: {reply['tool']} -> {result}")
history += [{"role": "assistant", "content": reply},
{"role": "tool", "content": result}]
else:
print("stopped at the step limit")
print(len(history), "messages")
# step 1: search_hotels -> [('Hana Inn', 12000), ('Kamo Stay', 15500)]
# step 2: convert -> 234
# final: About $234 for 3 nights.
# 5 messagesThe else on the for runs only if the loop uses up its steps without an answer. Swap fake_model for a real API call and the loop stays the same: the model decides the moves by reading the same history.
Check yourself
Four questions from across the lesson. Try each one before you look back.
1
When a model "uses a tool", what does the model itself actually produce?
tool_use block with the name search_hotels and the input {"city": "Kyoto"}. Your code decides whether to run it, runs it, and hands back the result. That split matters later: your code is where every safety check lives.2
A reply comes back with no tool request and the stop reason max_tokens. What should the loop do?
3
A model is called fresh on every pass of an agent loop. How does it know what a tool returned two turns ago?
4
A tool raises an exception halfway through a run. What should the loop do?
What to remember
1The model asks, your code runs
One turn adds two messages to the history: the model's tool call (a name, JSON input and an id) and your tool result with the same id. Only your code runs anything.
2Round and round until it answers
The loop calls the model until a reply has no tool call (
end_turn);max_tokensmeans cut off, not done, and a step limit stops runs that never end. The Kyoto run took 3 requests and 2 tools.3Every turn resends the history
Every request resends the whole history, so each turn grows the next request by its call and result: 520, 950, then 990 tokens, 2,460 in all.
Recap and going further
The lesson on one page
- The model writes a tool call: a name, JSON input and an id
- Your code runs it and sends back a tool result with the same id
- Both stay in the history
- Call the model, run the tool, append both, call again
- It ends on a reply with no tool call: end_turn
- max_tokens means cut off, not finished
- A step limit stops runs that never end
- Every request resends the whole history
- The next request grows by this turn's call and result
- A 400-token result is paid for on every later turn
- Your code runs every tool, never the model
- Tool errors go back as results, not crashes
Try this
Run the Kyoto loop against a real model
Take the scripted loop from this lesson and swap fake_model for a real API call. Describe search_hotels and convert as tools, keep returning the made-up hotel list from your own function, and log every request's messages, the stop reason of each reply and the input token count the API reports.
You need: Python, an API key for any model with tool calling, and a few cents of credit.
Look for
- The exact shape of the tool call in the raw reply, and the stop reason on each turn.
- How much the input token count grows from one turn to the next, and how much of it the tool definitions take.
- What the model does when
convertfails and you return the error message as the result. - Whether it ever answers before calling both tools, and how sure it sounds when it does.
The words, in plain language
- Tool call and tool result
- A tool call is the model's request to run a tool: a name, JSON input and an id. A tool result is your code's answer, sent back with the same id. The model only writes the call. Your code decides whether to run it.
- Agent loop and stop reason
- The agent loop calls the model, runs any tool it asks for, appends the call and the result, and calls again. A model running in it with tools is an agent. Each reply's stop reason says why it stopped:
tool_use,end_turnormax_tokens. - Context
- Everything the model reads in one request: the system prompt, the tool definitions and the history so far. In an agent it grows each turn by that turn's tool call and tool result, and it has to fit in the model's context window.
Read next
- ReAct: Synergizing Reasoning and Acting in Language ModelsShunyu Yao and others, 2022The paper that put reasoning and tool calls in one loop, written as Thought, Action, Observation.
- Tool use with ClaudeAnthropic docsThe tool_use and tool_result round trip in full, where tools run, and how tool definitions are billed as input tokens.
- Handling stop reasonsAnthropic docsEvery stop reason a loop has to handle, including a tool call cut off by max_tokens.
- Function callingOpenAI API docsThe same round trip in OpenAI's API: function_call items with a call_id, answered by function_call_output.
Agents overviewSpotted a mistake? Tell us.