insideout Get Pro

01 · Observe, decide, act

Free

Agents, lesson 1

Observe, decide, act

The loop an agent runs, and what the model sees on each turn.

"Letting the model call functions" in Prompting showed one tool call. An agent strings them into a loop that runs until the model can answer. This lesson follows one run turn by turn, with its messages and its token counts; you need a little Python and JSON.

By the end, you can:

  1. Follow one turn of the loop through its messages: the tool call and the tool result.
  2. Run the loop to the end, and tell a finished reply from a cut-off one by its stop reason.
  3. Count the tokens each request sends as the history grows.

Part 1. The model asks, your code runs

Follow one turn of the loop through its messages: the tool call and the tool result.

Ask a chat model "What would 3 nights at the cheapest hotel in Kyoto cost in US dollars?" and it can only guess. It can't search hotels or look up an exchange rate. So you offer it two tools your code can run, search_hotels(city) and convert(amount, from, to), and follow the first turn.

Example 1.1

The first turn, message by message

Your code sends the question and the two tool definitions to Anthropic's Messages API. Follow the turn until the model has the result it asked for. How many messages does the next request carry?

  1. Send the question1 message + 2 tool definitions

    The user's message goes in messages, the definitions in a separate tools field.

  2. Read the replystop_reason: "tool_use"

    A tool_use block: the id toolu_01, the name search_hotels and the input {"city": "Kyoto"}. It is text in an agreed shape.

  3. Run the toolsearch_hotels("Kyoto")

    Your code looks the name up in its own table of functions. The model takes no part.

  4. Send the result back1 + 1 + 1 = 3 messages

    Append the model's message as it came, then a user message with a tool_result carrying the same id.

Answer3 messages: the question, the model's tool call and your tool result.

The history after one turn
[{"role": "user",
  "content": "3 nights, cheapest hotel in Kyoto, in USD?"},
 {"role": "assistant",
  "content": [{"type": "tool_use",
               "id": "toolu_01",
               "name": "search_hotels",
               "input": {"city": "Kyoto"}}]},
 {"role": "user",
  "content": [{"type": "tool_result",
               "tool_use_id": "toolu_01",
               "content": "Hana Inn 12000 JPY; Kamo Stay 15500 JPY"}]}]

The tool result travels as a user message, but no person wrote it: your code did. The id ties each result to the call that asked for it. Other APIs change the names, not the idea. OpenAI's Responses API sends a function_call with a call_id, and you answer with a function_call_output.

Tool call and tool result

A tool call is the model's request to run a tool: a name, JSON input and an id. A tool result is your code's answer, sent back with the same id. The model only writes the call. Your code decides whether to run it.

one turn = the model's tool call + your tool result

One reply can hold several tool calls, such as a hotel search and an exchange rate. Run them in any order, then answer them all in one user message, each result with its own id. Anthropic's API rejects the next request if any tool call is left without its result.

Your turn

A reply comes back with stop_reason "tool_use" and two tool calls: get_rate with id toolu_08 and search_hotels with id toolu_09. What should your next request add to the history?

Append the model's message unchanged, then answer every call in one user message, each result carrying its call's id. Leave one out and the API rejects the request. Plain text loses the ids, and the provider runs none of your tools: your code does.

Part 2. Round and round until it answers

Run the loop to the end, and tell a finished reply from a cut-off one by its stop reason.

One tool call isn't enough for the Kyoto question. The model needs the prices first, and only after reading them does it know what to convert. So your code repeats the turn: call the model, run what it asks for, append both messages, and call again.

Example 2.1

The whole Kyoto run

Run the loop on the Kyoto question to the end, with an example rate of 0.0065 US dollars to the yen. How many requests does your code send, and how many tools does it run?

  1. Request 1→ search_hotels("Kyoto")

    Stop reason tool_use. The search returns Hana Inn at ¥12,000 a night and Kamo Stay at ¥15,500.

  2. Request 2→ convert(36000, "JPY", "USD")

    Stop reason tool_use. The model read the list: 3 nights at ¥12,000 is ¥36,000. The tool returns 234.

  3. Request 3→ "About $234 for 3 nights."

    Stop reason end_turn. There's no tool call, so the loop stops and your code shows the answer.

Answer3 requests and 2 tools run. The run ends on the first reply with no tool call.

Call the modelwith the whole historyTool call in the reply?yesYour code runs ittool_useAppend the calland its resultnoShow the answerend_turnStep limit reached:stop and report it
Figure 1The loop your code runs. It stops on a reply with no tool call, or when it reaches its step limit.

The model never sees the loop. Each request is a fresh call that reads the history so far, and the history is the only place the hotel prices live. ReAct, a 2022 paper, wrote each turn as Thought, Action, Observation. Agents still run that loop, and a tool result is often called an observation.

Agent loop and stop reason

The agent loop calls the model, runs any tool it asks for, appends the call and the result, and calls again. A model running in it with tools is an agent. Each reply's stop reason says why it stopped: tool_use, end_turn or max_tokens.

loop while the stop reason is tool_use, up to a step limit

Two traps. A reply can stop at max_tokens, the output limit, partway through a sentence or a call: no tool call, but not finished either. And a confused model can ask for tools for ever, so the loop needs a step limit. OpenAI's Chat Completions name the same three reasons tool_calls, stop and length.

Your turn

An agent's replies come back with these stop reasons, in order: tool_use, tool_use, tool_use, end_turn. Each tool_use reply held one tool call. How many requests did your code send, and how many tool results did it append?

Every reply answers one request, so 4 replies mean 4 requests. Each tool_use reply held one call, and each call gets one result: 3. The end_turn reply is the answer, so nothing runs after it.

Part 3. Every turn resends the history

Count the tokens each request sends as the history grows.

The provider keeps nothing between requests. So each one carries the lot: your system prompt, the tool definitions, the question, and every call and result so far. Measure the Kyoto run in tokens, with example sizes.

Example 3.1

The Kyoto run in tokens

The system prompt is 300 tokens, the two tool definitions 200 and the question 20. Each tool call is 30 tokens. The hotel list comes back as 400 tokens and the converted price as 10. How many input tokens does each request send?

  1. Request 1300 + 200 + 20 = 520

    The fixed part, 500 tokens, and the question.

  2. Request 2520 + 30 + 400 = 950

    Everything again, plus the first tool call and the hotel list.

  3. Request 3950 + 30 + 10 = 990

    Plus the second call and its result.

  4. Add them up520 + 950 + 990 = 2,460

    Input is billed on every request. The 500 fixed tokens were paid for three times, and the hotel list twice.

Answer520, 950 and 990 tokens: 2,460 input tokens to answer one question.

Request 1520 tokenssystem + tools 500Request 2950 tokenssystem + tools 500hotel list 400Request 3990 tokenssystem + tools 500hotel list 400Orange: what the last turn added. Question 20,each tool call 30, the converted price 10.
Figure 2Each request, drawn to scale. The fixed part goes every time, and once the hotel list is back it rides along on every later turn.

Each turn grows the next request by exactly what it added: the model's call and your result. A short call barely moves it. A long result moves it for the rest of the run, which is why later lessons trim both the results and the history.

Context

Everything the model reads in one request: the system prompt, the tool definitions and the history so far. In an agent it grows each turn by that turn's tool call and tool result, and it has to fit in the model's context window.

next request = this request + tool call + tool result

The fixed part opens every request unchanged, so providers can cache it and charge less for it on later turns ("Prefix caching" in Inference and Serving). Keep the start of each request stable, and put whatever changes after it.

Your turn

An agent's system prompt and tool definitions take 800 tokens, and the question 50. Every tool call is 40 tokens and every result 160. How many input tokens does the fourth request send?

Request 1 is 800 + 50 = 850, and each turn adds a call and a result, 200 tokens. The fourth request follows three turns: 850 + 3 × 200 = 1,450. 1,650 counts a fourth turn that hasn't happened, and 4,600 is all four requests added together.

Where you'll meet it

In interviews

Asked how an agent works? Draw the loop: the model writes a tool call, your code runs it, appends the call and its result, and calls the model again with the whole history. Then name the two exits, a reply with no tool call or the step limit, and the cost: every turn resends everything so far.

On the job

Tools cost tokens before they're used. Whenever you send tools, Anthropic's API adds a tool-use system prompt of its own, 286 tokens for Claude Opus 5.5, and bills it with your definitions as input on every request. OpenAI's docs say function definitions count against the context limit and are billed as input tokens too.

The Kyoto loop with a scripted model, in Python
def search_hotels(city):
    return [("Hana Inn", 12000), ("Kamo Stay", 15500)]

def convert(amount, frm, to):
    return round(amount * 0.0065)  # example JPY to USD rate

TOOLS = {"search_hotels": search_hotels, "convert": convert}

def fake_model(history):  # stands in for the API call
    results = [m["content"] for m in history if m["role"] == "tool"]
    if not results:
        return {"tool": "search_hotels", "input": {"city": "Kyoto"}}
    if len(results) == 1:
        nightly = min(price for _, price in results[0])
        return {"tool": "convert",
                "input": {"amount": 3 * nightly, "frm": "JPY", "to": "USD"}}
    return {"answer": f"About ${results[1]} for 3 nights."}

history = [{"role": "user", "content": "3 nights, cheapest hotel in Kyoto, in USD?"}]
for step in range(1, 9):  # the step limit
    reply = fake_model(history)
    if "answer" in reply:
        print("final:", reply["answer"])
        break
    result = TOOLS[reply["tool"]](**reply["input"])
    print(f"step {step}: {reply['tool']} -> {result}")
    history += [{"role": "assistant", "content": reply},
                {"role": "tool", "content": result}]
else:
    print("stopped at the step limit")
print(len(history), "messages")
# step 1: search_hotels -> [('Hana Inn', 12000), ('Kamo Stay', 15500)]
# step 2: convert -> 234
# final: About $234 for 3 nights.
# 5 messages

The else on the for runs only if the loop uses up its steps without an answer. Swap fake_model for a real API call and the loop stays the same: the model decides the moves by reading the same history.

Check yourself

Four questions from across the lesson. Try each one before you look back.

1

When a model "uses a tool", what does the model itself actually produce?

The model only ever produces tokens. A tool call is tokens in an agreed shape, like a tool_use block with the name search_hotels and the input {"city": "Kyoto"}. Your code decides whether to run it, runs it, and hands back the result. That split matters later: your code is where every safety check lives.

2

A reply comes back with no tool request and the stop reason max_tokens. What should the loop do?

No tool request isn't the same as done. The model ran out of room partway through, so the answer is truncated, or a tool call was cut off before it was complete. Check the stop reason rather than only looking for a tool call. Resending the same request with the same limit will most likely stop in the same place.

3

A model is called fresh on every pass of an agent loop. How does it know what a tool returned two turns ago?

Each model call is independent. The history is the agent's whole memory of the task, and your code resends it on every turn. That's why it grows, and why later lessons spend so much time keeping it small.

4

A tool raises an exception halfway through a run. What should the loop do?

An error is information. "Unknown currency code 'YEN', use 'JPY'" lets the model fix its own call on the next turn. Crashing throws away the work so far, blind retries repeat the same mistake, and an empty result invites the model to make something up.

What to remember

  1. 1The model asks, your code runs

    One turn adds two messages to the history: the model's tool call (a name, JSON input and an id) and your tool result with the same id. Only your code runs anything.

  2. 2Round and round until it answers

    The loop calls the model until a reply has no tool call (end_turn); max_tokens means cut off, not done, and a step limit stops runs that never end. The Kyoto run took 3 requests and 2 tools.

  3. 3Every turn resends the history

    Every request resends the whole history, so each turn grows the next request by its call and result: 520, 950, then 990 tokens, 2,460 in all.

Recap and going further

The lesson on one page

One turn
  • The model writes a tool call: a name, JSON input and an id
  • Your code runs it and sends back a tool result with the same id
  • Both stay in the history
The loop
  • Call the model, run the tool, append both, call again
  • It ends on a reply with no tool call: end_turn
  • max_tokens means cut off, not finished
  • A step limit stops runs that never end
The agent loop
What each turn costs
  • Every request resends the whole history
  • The next request grows by this turn's call and result
  • A 400-token result is paid for on every later turn
Where the control lives
  • Your code runs every tool, never the model
  • Tool errors go back as results, not crashes

Try this

Run the Kyoto loop against a real model

Take the scripted loop from this lesson and swap fake_model for a real API call. Describe search_hotels and convert as tools, keep returning the made-up hotel list from your own function, and log every request's messages, the stop reason of each reply and the input token count the API reports.

You need: Python, an API key for any model with tool calling, and a few cents of credit.

Look for

  • The exact shape of the tool call in the raw reply, and the stop reason on each turn.
  • How much the input token count grows from one turn to the next, and how much of it the tool definitions take.
  • What the model does when convert fails and you return the error message as the result.
  • Whether it ever answers before calling both tools, and how sure it sounds when it does.

The words, in plain language

Tool call and tool result
A tool call is the model's request to run a tool: a name, JSON input and an id. A tool result is your code's answer, sent back with the same id. The model only writes the call. Your code decides whether to run it.
Agent loop and stop reason
The agent loop calls the model, runs any tool it asks for, appends the call and the result, and calls again. A model running in it with tools is an agent. Each reply's stop reason says why it stopped: tool_use, end_turn or max_tokens.
Context
Everything the model reads in one request: the system prompt, the tool definitions and the history so far. In an agent it grows each turn by that turn's tool call and tool result, and it has to fit in the model's context window.

Agents overviewSpotted a mistake? Tell us.