The problem #

A single call to an LLM is a one-shot: prompt in, text out. It cannot look up a live price, run a calculation it shouldn’t be trusted to do in its head, or check whether a previous step actually succeeded. If you want a model to use a tool and then reason over what that tool returned, one API call cannot do both halves of that job. You need something that calls the model, notices when it wants a tool, actually runs that tool, and hands the result back for another round of reasoning.

The concept #

An agent loop is that “something.” It is a plain loop your own code owns and drives:

1. Call the model with the conversation so far.
2. Check whether the response asked to call a tool (the model attaches a “tool call” request to its message, it never executes anything itself).
3. If it did: run the requested tool(s) in your code, wrap each result in a message, append the model’s request and the tool’s result to the conversation, then go back to step 1.
4. If it didn’t: the model returned a plain answer with no tool calls attached. That absence is the loop’s exit signal, there is no separate “I’m done” flag to check.

graph TD
    A[Call the model] --> B{tool_calls present?}
    B -->|No| C[Return content, exit loop]
    B -->|Yes| D[Execute each requested tool]
    D --> E[Append AIMessage + ToolMessage results]
    E --> A

That four-step skeleton is the easy part, and it looks the same regardless of which model sits behind it. What actually decides whether the wiring works is two things underneath it: what a “tool call” really is on the wire, and why the order you feed messages back in is not a style choice.

What a tool call actually is on the wire #

The model never executes anything. What it returns is a structured request, and the shape of that request is where providers stop agreeing with each other.

OpenAI’s chat API puts it on the assistant message itself: a top-level tool_calls array, each entry carrying an id, a type of "function", and a function object whose arguments field is a JSON-encoded string, not a parsed object, your code (or your framework) has to json.loads it before use. The matching reply is a separate message with role: "tool" and a tool_call_id copied from that entry; the API rejects the request if that role-"tool" message doesn’t immediately follow the assistant turn that produced the matching tool_calls entry.

Anthropic’s Messages API represents the same idea differently: there is no separate tool_calls field at all. The assistant’s content is a list of blocks, and a tool_use block (with id, name, and an already-parsed input object) can sit right alongside a plain text block in that same list, since Claude is allowed to think out loud and call a tool in the same turn. The reply travels back not as a new role, but as a user-role message whose content is itself a list containing one or more tool_result blocks, each pointing at the tool_use_id it answers.

LangChain sits on top of both and normalizes them into one shape: AIMessage.tool_calls is always a plain list of dicts, each one {"name": ..., "args": ..., "id": ..., "type": "tool_call"}, whether the adapter underneath had to pull that out of OpenAI’s stringified arguments or lift it straight out of an Anthropic tool_use block. ToolMessage.tool_call_id gets translated the same way in reverse: back into a role: "tool" message for OpenAI, or into a tool_result content block for Anthropic, at serialization time. Ollama’s native tool-calling API mirrors OpenAI’s shape closely enough that the same LangChain code path handles it without a special case, which is why the example below can swap qwen3:1.7b for gpt-4o or claude-* and change nothing but the model string.

Why ordering and IDs aren’t optional #

Both of these APIs are stateless: every call resends the entire conversation, there is no server-side handle to “the previous turn” to resume. That has a direct consequence for the loop. The only thing that tells the model “this is the answer to the tool you asked for” is what’s physically present in that resent history, and both major providers reject a request where a tool result doesn’t immediately follow the assistant turn that requested it, and it must be provided for every single tool call that turn issued.

ID-based matching (rather than pure position) exists because a single assistant turn can request more than one tool at once, both providers support this, and once there are two or three pending calls in flight, “the next message answers the last request” stops being well-defined. tool_call_id (OpenAI) and tool_use_id (Anthropic) are how the model recovers which result belongs to which request even when several came back in the same round. But ID matching does not relax the ordering requirement, it just makes ordering unambiguous instead of accidentally correct: the tool results still have to be appended immediately after the request they answer, for every call issued, or the next call to the API fails outright.

Minimal example #

This is a trimmed version of a shopping-assistant agent loop, cut down from a working example built on LangChain’s bind_tools and ToolMessage abstractions (the full version, with a discount tool and error handling for unknown products, lives in ch03_agent_loop_langchain_tool_calling.py).

 1from langchain.chat_models import init_chat_model
 2from langchain_core.messages import HumanMessage, SystemMessage, ToolMessage
 3from langchain_core.tools import tool
 4
 5MAX_ITERATIONS = 10
 6
 7
 8@tool
 9def get_product_price(product: str) -> float:
10    """Look up the price of a product in the catalog."""
11    prices = {"laptop": 999.99, "tablet": 499.99}
12    return prices[product.strip().lower()]
13
14
15def run_agent_loop(query: str) -> str | None:
16    tools = [get_product_price]
17    tools_by_name = {t.name: t for t in tools}
18    # bind_tools() only announces the tool's schema to the model, it does not
19    # execute anything. Running it and feeding the result back is this loop's job.
20    llm = init_chat_model("qwen3:1.7b", model_provider="ollama").bind_tools(tools)
21
22    messages = [
23        SystemMessage(content="You are a shopping assistant. Always call get_product_price instead of guessing a price."),
24        HumanMessage(content=query),
25    ]
26
27    for _ in range(MAX_ITERATIONS):
28        ai_message = llm.invoke(messages)
29
30        if not ai_message.tool_calls:
31            return ai_message.content  # no tool calls requested: the model is done
32
33        messages.append(ai_message)  # the request must precede its result
34
35        for call in ai_message.tool_calls:
36            tool_fn = tools_by_name.get(call["name"])
37            try:
38                result = tool_fn.invoke(call["args"]) if tool_fn else f"Error: unknown tool {call['name']}"
39            except Exception as e:
40                result = f"Error: {e}"
41            messages.append(ToolMessage(content=str(result), tool_call_id=call["id"]))
42
43    return None  # bailed out: caller must treat this as a failure, not an answer

Every field in this loop maps onto the mechanism above. llm.bind_tools(tools) is what makes ai_message.tool_calls show up at all: it attaches each tool’s schema to the outgoing request so the model can reference get_product_price by name, but bind_tools itself never executes anything, it only makes the request possible. When the model decides to call it, ai_message.tool_calls is LangChain’s normalized list described above, one dict per requested call, with name, args, and id already in place regardless of whether qwen3, GPT, or Claude produced the response underneath.

The ordering discipline discussed earlier lives in three lines. messages.append(ai_message) runs before the for call in ai_message.tool_calls loop, so the request that asked for a tool always lands in the transcript strictly before its answer, satisfying the API-level requirement that a tool result immediately follow the turn that requested it. Inside that loop, tool_call_id=call["id"] is the piece doing the actual matching: it copies the ID LangChain already normalized off the request and stamps it onto the ToolMessage, which is what lets the next llm.invoke(messages) thread each result back to the exact call that asked for it, even when the model requests two or three tools in the same turn. And the try/except around tool_fn.invoke(call["args"]) is what turns a Python exception into something the model can actually read: a bare crash here would kill the loop instead of giving the model (or the user) a chance to see the tool actually failed.

Every pass through the loop is: invoke, check tool_calls, run what was asked, append the outcome, loop again. That is the whole mechanism; only the wire format changes underneath LangChain’s abstraction when the provider changes, not the shape of the loop.

Design trade-offs: why hand-roll this at all #

The agent loop is shaped the way it is because the underlying APIs are stateless and message-based: nothing persists on the provider’s side between calls, so the full history, every prior tool call and every prior result, has to be reconstructed and resent on every single round trip. That statelessness is also the entire point: because you own the array of messages, you can inspect it, redact a tool result before it goes back to the model, drop an old turn to save tokens, retry a single failed step, or fork the conversation into two branches, all without asking a framework’s permission. Nothing about that is available once a loop is running inside someone else’s abstraction.

The cost is that every provider quirk described above becomes your problem, by hand, every time: verbosity that scales with the number of tools and turns, and a small set of footguns that show up often enough to be worth naming precisely. Forgetting a hard iteration bound (MAX_ITERATIONS here) turns a confused model into an unbounded token bill: a model that keeps requesting the same tool with slightly different arguments will happily do that forever. Appending a ToolMessage without its matching AIMessage already in place, or reusing the wrong tool_call_id, is not a style nit; per the ordering rules above, most providers will reject the next call outright, and the ones that don’t will silently misattribute a result to the wrong request. Handling only the first entry in ai_message.tool_calls when a model requested several in parallel leaves later tool calls without a matching result, which the API then refuses, since every issued call needs an answer, not just the first one. And swallowing a tool’s exception instead of feeding it back as an “Error: …” message removes the one signal that lets the model recover: it will either repeat the same failing call or quietly fabricate an answer instead of admitting it doesn’t know.

Hand-rolling the loop is the right call when you need that level of control over what actually enters the context window, when you’re deliberately learning or debugging the mechanism itself, or when the real shape of the problem is small enough (a couple of tools, a low iteration bound) that fifteen lines you can read end to end beats a dependency you now have to trust. Reach for a higher-level abstraction once the loop stops being something you can eyeball: parallel-safe execution, streaming partial output, human-in-the-loop interrupts mid-loop, or several of these loops composed together turn “a loop” into “a state machine you’re maintaining by hand.” That is exactly the seam the next chapter picks up.

Summary #

Underneath the phrase “agent,” there is no new kind of intelligence, there is a plain loop wrapped around an ordinary model call: send the conversation, check whether the response asked for a tool, run it if so, append both the request and the result in the order the API demands, and repeat until the model answers with nothing left to call or the loop hits its bound. The part worth carrying forward isn’t the four steps themselves, it’s what’s underneath them: a tool_calls entry is a normalized view over genuinely different wire formats (OpenAI’s stringified arguments and top-level array, Anthropic’s tool_use content blocks mixed with text, Ollama’s OpenAI-shaped variant), and the strict request-then-result ordering exists because these APIs have no memory of their own, only the resent history proves what happened. The next chapter keeps this exact mechanism but swaps the hand-rolled for loop for LangChain’s create_agent and the @tool decorator: same call-check-execute-append cycle, the same tool_call_id matching happening under the hood, just with far less of it left for you to get wrong by hand.