insideout Get Pro

06 · When a tool call fails

Pro

Agents, lesson 6

When a tool call fails

Retries with backoff, and errors the model can recover from.

You know from "Designing tools a model can use" that a tool's errors go back to the model as observations.

Part 1. Some failures are worth a second try

An agent calls a weather API and gets one of these back: 503 Service Unavailable, 429 Too Many Requests, 404 Not Found, 401 Unauthorized. Send the same request again in a few seconds and the first two may well work. The last two will fail the same way every time.

Your turn

Which of these should your code retry automatically?

503 means the server is busy, 429 means you're going too fast, and a timeout may be a network blip. All of these are temporary. 404 (doesn't exist) and 401 (bad credentials) won't change by asking again, so retrying only wastes time. Hand those straight back to the model.

Transient and permanent errors

A transient error may clear up by itself, so waiting and trying again can work: timeouts, 429, and the 5xx family such as 500, 502, 503 and 504. A permanent error comes back the same however often you ask: a malformed request (400), something that doesn't exist (404), or missing rights (401, 403). Retry the first kind automatically; hand the second straight to the model.

Model APIs follow the same split, with one extra code to know: Anthropic's API answers 529 when it's overloaded, which is transient. And the official Python SDKs from OpenAI and Anthropic already retry twice by default, with backoff, on connection errors, 408, 409, 429 and 5xx responses. If your tool wraps a model call, check that setting before adding retries of your own, or you'll be retrying retries.

Part 2. Wait longer each time, and not in step

Keep going with Pro

You've done the first part. Pro opens the rest of this lesson, and every other one.

Still to come
  1. Wait longer each time, and not in step
  2. What retries buy, and what they cost
  3. When retries run out
See Pro plansI have Pro: sign in
Next lessonIdempotency keysMake tool calls safe to repeat. Start lesson 7

Agents overviewSpotted a mistake? Tell us.