The problem #

Free-form text output turns every downstream consumer into a parser: regex against a paragraph, hope the model kept a consistent format, or spend a second call asking the model to reformat its own answer. A response that’s slightly off, a missing field, a value wrapped in a sentence of prose instead of standing alone, a list where a single object was expected, rarely fails right where the mistake happened. It surfaces two functions downstream as a KeyError, or worse, it doesn’t surface at all and just feeds wrong data into whatever runs next.

The concept #

Define a Pydantic model describing the exact shape you’re willing to accept: field names, types, which ones are required. Hand that model to the framework instead of asking for text and hoping. On a plain chat model, with_structured_output(Schema) wraps the model so .invoke() returns a parsed and validated instance of that schema. On an agent built with create_agent, the equivalent is the response_format parameter: pass a Pydantic class, and once the run finishes, the agent’s state carries the validated instance under the structured_response key. Either way, the contract moves from “the model probably returned something like this” to “the model returned exactly this, or the call failed to produce it.”

That sentence hides a fair amount of machinery, and the details matter once something goes wrong, so it’s worth taking the mechanism apart.

From schema to constraint #

A Pydantic model isn’t handed to the model as a hint in the prompt. LangChain calls model_json_schema() on it, producing a JSON Schema document: field names, types, which ones are required, nested objects and lists spelled out structurally. That schema is then attached to the underlying API call in one of a few ways, controlled by with_structured_output’s method parameter. method="function_calling" (the default for most models) wraps the schema as a tool definition and asks the model to “call” it with matching arguments, the same tool-calling machinery from chapter 03, just aimed at a single always-invoked pseudo-tool instead of an optional one. method="json_schema" is newer and, where supported, hands the schema to a provider endpoint built specifically for structured output rather than emulating it through tool calls.

Either path can also take a strict flag. Without it, the schema is advisory: it shapes the tool signature or the response-format instructions the model is steered toward, but nothing at the token-sampling level stops the model from emitting a value that violates it, which is why a validation pass after the call still happens. With strict=True, on providers that support it (OpenAI, xAI), the guarantee moves earlier: the provider constrains decoding itself, restricting which tokens are even legal to sample next so the output can’t leave the schema’s shape in the first place. That’s the difference between “steered toward” and “structurally incapable of leaving.” Most of the time you’re in the first regime, and the validation step is doing real work, not just double-checking a formality.

Provider-native versus tool-call emulation #

create_agent formalizes the same choice as two named strategies. ProviderStrategy reaches for a model provider’s own structured-output API (OpenAI, Anthropic, xAI, and Gemini all expose one) and gets the strongest guarantee available, since the provider itself is enforcing the shape. ToolStrategy is the fallback: for models without that native support, or when one isn’t available, the schema becomes a tool the model has to call, and the result is extracted from the tool-call arguments instead of a native response field. create_agent picks between them automatically (this is often called AutoStrategy): reach for ProviderStrategy when the model exposes one, drop down to ToolStrategy otherwise. In the example below, ChatOpenAI is one of the models with native support, so response_format=JobSearchResponse resolves to ProviderStrategy without any of the strategy classes appearing in the code. You’d only write ToolStrategy(...) or ProviderStrategy(...) explicitly to override that default, say, to force tool-call emulation on a model that has a native path you don’t trust yet.

Two ways to fail: raise now, or retry with feedback #

This is where the two entry points genuinely diverge. A bare with_structured_output call that fails to parse into the schema raises there and then: .invoke() throws, no automatic retry, you catch it or you don’t. create_agent’s response_format, running under ToolStrategy, is more forgiving: a failed validation doesn’t propagate as an exception, it’s caught internally, turned into an error message (“failed to parse, please fix your mistakes” or similar), and fed back to the model as if it were a tool result, giving the model another turn to correct itself before the framework gives up. That behavior is tunable through handle_errors: True (the default) catches every validation failure with a generic message and retries; a custom string uses that message for every retry instead; one or more specific exception types (StructuredOutputValidationError for a plain schema mismatch, MultipleStructuredOutputsError for when the model calls more than one structured-output tool when exactly one was expected) narrows which failures get a second chance and lets everything else propagate; a callable lets you compute a bespoke message per failure; and False disables the retry entirely, so a bad attempt raises immediately, the same posture with_structured_output has by default. Same underlying guardrail, Pydantic validation against the same kind of schema, but a meaningfully different amount of self-correction happening before the caller ever sees a failure.

Shape is not truth #

None of this touches whether the value is correct. A schema is a promise about structure: this field exists, it’s a string, that one’s a list of objects with these four keys. It says nothing about whether the string is accurate or the list reflects anything real. Validation, whether it’s Pydantic checking types after the fact or a provider constraining decoding as it generates, is a syntactic guarantee: the output conforms to a shape. Nothing in a JSON Schema can express “and the URL actually resolves” or “and this company exists,” because those are semantic claims about the world, not structural claims about the response, and a schema has no vocabulary for the world. That gap between validated and true is exactly the seam this chapter’s example runs into next.

Minimal example #

Trimmed from a small search agent (introduced in chapter 03) that calls the Tavily search API and returns typed job postings instead of a paragraph to parse:

 1from langchain.agents import create_agent
 2from langchain_core.messages import HumanMessage
 3from langchain_openai import ChatOpenAI
 4from langchain_tavily import TavilySearch
 5from pydantic import BaseModel, Field
 6
 7
 8class JobPosting(BaseModel):
 9    """Schema for a single job posting found by the agent"""
10
11    title: str = Field(description="The job title")
12    company: str = Field(description="The hiring company")
13    location: str = Field(description="The job location, e.g. city/country or 'Remote'")
14    url: str = Field(description="URL to the job posting")
15
16
17class JobSearchResponse(BaseModel):
18    """Schema for the agent's structured response"""
19
20    jobs: list[JobPosting] = Field(
21        default_factory=list, description="Job postings matching the query"
22    )
23
24
25agent = create_agent(
26    model=ChatOpenAI(model="gpt-5-mini", temperature=0),
27    tools=[TavilySearch()],
28    response_format=JobSearchResponse,
29)
30
31result = agent.invoke(
32    {"messages": HumanMessage(content="search 3 jobs in europe for a senior .NET engineer, remote")}
33)
34
35response: JobSearchResponse = result["structured_response"]
36print(response.jobs[0].title, response.jobs[0].company)

Two schemas are doing the work here, not one. JobPosting describes a single result: four required string fields, nothing optional, nothing left for the model to improvise. JobSearchResponse wraps a list of those, with default_factory=list so an empty result set is a valid response rather than a missing field the model has to remember to include. That nesting, a schema whose field is itself a list of another schema, is exactly the shape model_json_schema() turns into a structured JSON Schema document with a jobs array of JobPosting objects, and it’s exactly the shape that gets attached to the call: as a ProviderStrategy-backed structured-output request here, since gpt-5-mini is an OpenAI model with native support, or as a tool call under ToolStrategy on a model without one, with the framework choosing automatically from the same response_format=JobSearchResponse line.

Once the run finishes, result["structured_response"] is a real JobSearchResponse instance, not a dict and not a string to regex. response.jobs is a list of JobPosting objects, each field already typed and present, because a response missing location or carrying title as a number instead of a string would have failed validation before ever reaching this line, either raising outright or, if a retry path was available, going back to the model as an error and giving it another attempt first. Nothing downstream needs to know the model called Tavily two or three times, chained a follow-up search, or reformulated the query to get here. It reads response.jobs[0].title the same way it would read a field off a database row.

What it can’t tell you, no matter how clean the schema, is whether that title genuinely belongs to a job that exists at that URL. That’s not a gap in this particular schema, it’s the boundary of what schema validation was ever built to check, as the previous section covered, and it’s the thread the next chapter picks up.

Trade-offs #

Forcing a fixed schema is a real design decision, not a free upgrade over free-form text, and it’s worth being honest about what it costs. The model loses room to hedge or add context outside the fields you defined: if it discovers a fifth relevant job, or wants to flag that two listings looked like duplicates, there’s nowhere to put that unless you specifically modeled a field for it. An unanticipated but perfectly reasonable answer doesn’t come back slightly off, it comes back rejected, because the schema has no slot for “close enough.” That pushes real design work onto the schema itself: guess wrong about which fields you’ll actually need, and you’re stuck choosing between loosening the schema (more optional fields, looser types) or eating failed calls that would have been fine as prose.

The retry-with-feedback behavior under ToolStrategy is worth building into your own tools when failures tend to be near misses: a stray string where an int belonged, one optional field left out, something a model reliably fixes on a second look once told what’s wrong. It stops paying for itself when failures are systematic instead, a schema nested more deeply than the model can reliably hit turns retries into repeating the same mistake at your expense, since each retry is another full model call. In that case, failing fast (handle_errors=False, or the plain with_structured_output posture of raising immediately) is more honest: you find out sooner and can decide whether to flatten the schema, rather than silently paying for several automatic attempts that were never going to succeed. include_raw=True on with_structured_output sits between the two: it doesn’t retry, but it also doesn’t raise, it hands you the raw message, the parsed result if there was one, and the parsing error if there wasn’t, and leaves the retry-or-not decision to your own code instead of the framework’s.

And sometimes the answer is: don’t. If the consumer of the output is a person reading a chat reply, structure is overhead they never asked for. If what you actually need is a single number or one word buried in a sentence, a light instruction (“answer with just the number”) and a one-line parse handle it without a schema, a tool call, or a validation step at all. Structured output earns its cost at the boundary where the next reader is a function, not a person, exactly the boundary response.jobs[0].title sits on in the example above.

Summary #

A schema tells you the response is well-formed. It doesn’t tell you the response is right, and no amount of nesting, strict mode, or retry logic changes that, because validation, whether Pydantic checking a parsed object or a provider constraining its own decoding, is a check on shape, not on truth. Treat “valid JobSearchResponse” and “correct JobSearchResponse” as two separate claims, and keep in mind they take different kinds of effort to guarantee. Chapter 06 guarded a risky action by pausing and asking a person before it happened. This chapter guards the output itself, without asking anyone, by constraining what the model is allowed to produce in the first place and, depending on the mechanism, giving it one more chance to fix a bad attempt before you ever see it. Neither one touches whether the content inside that guarded shape is actually correct. Closing that gap, checking correctness now that output has a fixed shape you can systematically check against, is exactly what a minimal eval harness is for, and that’s where we go next.