The problem #

A model call can hang for thirty seconds instead of three. A tool call can fail once because of a network blip or a rate limit, then succeed on the very next try. An agent loop can keep calling tools forever, not because anything crashed, but because the model never produces an output that satisfies the stop condition. None of the last ten chapters prevent any of this by default. A graph makes the control flow explicit, a checkpointer makes a run resumable, an eval or a trace makes a failure visible after the fact, but visibility and resumability aren’t the same as survivability. Left alone, a hanging call just hangs, a transient error just fails the run, and a loop with no exit just keeps going.

The concept #

Three separate mechanisms cover three separate failure modes, and none of them substitutes for the others.

Timeouts: turning “still running” into a decision #

A timeout doesn’t diagnose why a call is slow, it just measures wall-clock time against a budget you set and raises once that budget is spent. client.post(url, json=payload, timeout=5.0) tells httpx to give up waiting after five seconds and raise httpx.TimeoutException locally, whatever is actually happening on the other end of that connection. The remote server might still be working the request when your client times out, and it has no way of knowing your side already moved on. That’s the whole job of a timeout: it doesn’t fix slowness, it just refuses to let “still running” stay an open question forever, converting it into a typed exception at a moment you chose instead of one the dependency chose for you.

Retries: backoff, jitter, and what’s actually safe to retry #

A retry assumes the failure a timeout (or anything else) produced was transient: the same call, tried again a moment later, has a real chance of succeeding. wait_exponential(multiplier=1, min=1, max=20) computes each delay as roughly multiplier * 2^attempt, clamped between min and max, so the wait between attempts doubles each time up to a ceiling instead of staying fixed or growing without bound. The doubling matters, not just the spacing: a dependency that’s overloaded or rate-limiting you needs progressively more room as attempts accumulate, not a constant trickle of retries arriving at the same rate that got it into trouble in the first place.

Backoff alone still has a gap. If every client computes the exact same delay sequence from the exact same failure, a burst of clients that all failed at once will all retry at once, arriving back at the dependency in synchronized waves, the “thundering herd” problem that can turn a brief blip into a self-inflicted outage. Jitter closes that gap by adding randomness to the delay, either via wait_random_exponential or by composing wait_fixed(3) + wait_random(0, 2), so clients that failed together don’t retry together. The exponential curve controls the average delay; jitter controls the correlation between clients. You need both: a perfectly tuned backoff curve with no randomness still produces a stampede if enough callers hit it at the same instant.

None of that answers whether retrying is safe, which is a separate question from whether it’s likely to succeed. An operation is idempotent if running it twice has the same effect as running it once: a lookup, a read, a price check. A retry on an idempotent call costs at most some latency. A non-idempotent operation, charging a card, placing an order, appending to a ledger, is a different story: if the first attempt actually succeeded on the server but the response was lost to the timeout (the request landed, the reply didn’t), a blind retry can execute the same side effect twice. That’s the case a timeout genuinely can’t resolve on its own: it tells you the response didn’t arrive, not whether the underlying operation ran. Making a non-idempotent call safe to retry means changing the call, not the retry policy, typically an idempotency key the server can use to recognize and discard a duplicate, or a check-before-write step that confirms the prior attempt’s outcome before trying again.

retry_if_exception_type is how “transient” gets encoded as a rule instead of a judgment call made at read time. Every exception raised inside the decorated function is checked against the predicate: if it matches, tenacity swallows it, computes the next backoff delay, and schedules another attempt; if it doesn’t match, the exception propagates immediately, bypassing wait and stop entirely, as if the decorator weren’t there. TRANSIENT_ERRORS in the example are the exceptions worth that treatment: a timeout, a connection failure, a provider’s own rate-limit error. A 401 or a malformed payload isn’t in that tuple, so it raises past @retry on the very first attempt. Classifying errors this way, as a type-based predicate rather than a manual except block scattered through the function, keeps the transient/non-transient line declarative: adding a new retryable exception is a one-line change to a tuple, not a restructuring of control flow.

The decision path for a single call looks like this:

flowchart TD
    A[Call] --> B{Raised exception?}
    B -->|No| Z[Return result]
    B -->|Yes| C{Retryable?<br/>retry_if_exception_type}
    C -->|No| F[Raise / give up]
    C -->|Yes| D{Attempts exhausted?<br/>stop_after_attempt}
    D -->|Yes| F
    D -->|No| E[Backoff + jitter, then retry]
    E --> A

stop_after_attempt(4) is the cap on that loop: after four total attempts, tenacity stops retrying and re-raises (or wraps the failure in its own RetryError, unless reraise=True is set, in which case the original exception surfaces instead). It bounds the retries themselves, independent of how long any individual backoff delay is.

Recursion limits: the backstop that doesn’t care why #

Retries are keyed to exceptions: something has to fail for a retry policy to act. LangGraph’s recursion_limit isn’t keyed to anything failing at all. It counts super-steps, the discrete steps of the graph’s own Pregel-style execution loop (nodes scheduled to run in the same pass belong to one super-step; the counter only advances between passes), and it raises GraphRecursionError once that count crosses the limit, the default is 25, regardless of whether every single step along the way returned cleanly. A graph where every node executes without error, every tool call succeeds, and nothing ever times out can still hit GraphRecursionError, because the model never produced an output that satisfied the graph’s stop condition. The counter that trips has nothing to do with success or failure of any individual step, only with how many steps have run.

That’s what makes it a genuinely different mechanism rather than a third flavor of retry. A retry policy can only see the failures you told it to look for. An agent that keeps calling tools “successfully,” forever, never raises anything a retry_if_exception_type predicate could catch, because nothing is technically failing. recursion_limit is the only one of the three mechanisms that terminates a run purely on step count, which is exactly why it has to exist independently even when timeouts and retries are both implemented correctly everywhere else.

Minimal example #

The mechanisms above aren’t independent theory, they’re what’s actually running in this function: a tool call wrapped with a timeout and a tenacity retry that only fires on transient exceptions, plus a graph invocation with a recursion limit as the loop backstop.

 1import httpx
 2from tenacity import (
 3    retry,
 4    stop_after_attempt,
 5    wait_exponential,
 6    retry_if_exception_type,
 7)
 8from langgraph.errors import GraphRecursionError
 9
10TRANSIENT_ERRORS = (
11    httpx.TimeoutException,
12    httpx.ConnectError,
13    RateLimitError,  # provider-specific, e.g. openai.RateLimitError
14)
15
16@retry(
17    retry=retry_if_exception_type(TRANSIENT_ERRORS),
18    wait=wait_exponential(multiplier=1, min=1, max=20),
19    stop=stop_after_attempt(4),
20    reraise=True,
21)
22def call_tool_with_timeout(client: httpx.Client, url: str, payload: dict) -> dict:
23    # timeout bounds the single call; retry only wraps the transient failures above
24    response = client.post(url, json=payload, timeout=5.0)
25    response.raise_for_status()  # 4xx (bad request, auth) is not in TRANSIENT_ERRORS, so it isn't retried
26    return response.json()
27
28# The loop-limit backstop, independent of timeouts and retries entirely:
29try:
30    result = graph.invoke(
31        {"messages": [HumanMessage(content=query)]},
32        config={"recursion_limit": 50},
33    )
34except GraphRecursionError:
35    # the graph took more steps than allowed and was stopped, not because a call failed
36    result = {"status": "stopped_at_limit"}

Read the code against the three mechanisms above and every line maps onto something already named. timeout=5.0 inside client.post is the bound on the single call: the piece that decides “still running” becomes “failed” after five seconds rather than staying open indefinitely. retry_if_exception_type(TRANSIENT_ERRORS) is the predicate: it’s checked against whatever call_tool_with_timeout raises, and only a member of that tuple, a timeout, a connection failure, the provider’s own rate-limit error, gets a second attempt. response.raise_for_status() can raise plenty of other things, a 4xx for a bad request or an expired key among them, and none of those are in TRANSIENT_ERRORS, so they raise straight past the decorator on the very first attempt instead of burning three more tries first. wait_exponential(multiplier=1, min=1, max=20) spaces those retries out with a growing, capped delay, and stop_after_attempt(4) is the ceiling on how many times that’s allowed to happen before reraise=True lets the original exception through instead of tenacity’s own RetryError.

recursion_limit in the graph.invoke call is doing none of that. It isn’t watching for exceptions at all, it’s counting super-steps, and it will fire even if call_tool_with_timeout never raises anything, ever, because the model simply never produces an output that satisfies the graph’s stop condition. That’s the same gap chapter 02’s plain MAX_ITERATIONS for-loop closed for a hand-rolled agent loop, and it’s the same backstop a resumed chapter 06 approval flow still needs after a human says yes: approval unblocks the graph, it doesn’t guarantee the graph that follows actually terminates on its own.

Trade-offs and design rationale #

Every default in this chapter is a bet, and the bet cuts both ways.

Retrying too aggressively has a real cost beyond latency. Each extra attempt against a paid API is money spent on a call that may fail again, and it adds request volume onto a dependency that’s already struggling, exactly when it can least absorb the extra load, the failure mode backoff and jitter exist to prevent but that a stop_after_attempt set too high, or a max wait set too large, can quietly reintroduce. A backoff ceiling of a couple of minutes across eight attempts means a single failing call can occupy a worker for most of ten minutes before giving up, which is fine for a background job and a poor choice for anything a user is waiting on synchronously.

Retrying too conservatively has a cost too, and it’s easy to under-weight because it’s invisible: a call that would have succeeded on the second attempt now fails the whole run on the first, and whoever’s waiting on it sees a failure that a five-second wait and one more try would have avoided. stop_after_attempt(1) (effectively no retry) is the right choice for something genuinely non-idempotent with no idempotency key behind it, and the wrong choice for a rate-limited read that will almost certainly succeed a moment later. There’s no single right number of attempts, there’s a right number for a given operation’s cost of failing versus its cost of retrying, and that’s a per-call decision, not a global one.

A recursion limit is blunt in a way retries aren’t: it can’t tell “the model is looping pointlessly” apart from “the model is doing twenty-three legitimate steps of real work.” Both look identical to the counter, a step is a step. That’s a real cost: a limit tuned against a two-step demo will cut off a production run that legitimately needs a dozen tool calls and a couple of reflection passes, and the failure it produces, GraphRecursionError, looks exactly the same whether the graph was genuinely stuck or just doing a lot of honest work. It’s still worth having anyway, because the alternative, no ceiling at all, means a model that never converges runs until something else stops it: a process timeout, an infrastructure limit, or a bill. A reasonable default starts from the actual step count of your longest legitimate path (count the tool calls and reflection passes a real task needs, then add real headroom) rather than from whatever number happened to work in the example you first copied it from.

Summary #

A timeout, a retry, and a recursion limit each answer a different question: how long is too long for one call, is this particular failure worth trying again, and how many steps is too many regardless of whether anything actually failed. Getting all three right means setting them from the actual cost of getting each one wrong, an unnecessary retry’s latency and money against a possible one’s chance of turning a blip into a full failure, a recursion limit low enough to catch a genuine runaway against one that doesn’t quietly truncate real work, rather than from whatever default shipped with the library. That completes the arc this series has been building since chapter 02’s plain for loop with a MAX_ITERATIONS cap: the same loop, the same need for a hard stop, now wrapped in graphs, checkpoints, structured output, evals, and tracing, with this chapter’s timeouts, retries, and recursion limit as the resilience layer that has to sit underneath all of it before any of it is safe to run unattended.