Producing structured objects: why it matters.
•
March 27, 2026
This is the first post in a three-part series on using LLMs to generate complex structured objects in production systems.
When most people think about Large Language Models in production, they picture chatbots generating free-form text or maybe writing some code. But here’s what I’ve learned building real systems: the most impactful application of LLMs is creating structured objects from unstructured data.
Let me explain why this matters and how to do it reliably.
Why Structured Objects?
In production, the key to adding value with LLMs is to use them as one tool among many. But for this, you often need them to produce very old-school, structured objects.
Think about it: your LLM output needs to be passed to databases, APIs, visualization libraries, and other deterministic systems. You can’t just hand these systems free-form text and hope for the best.
Real-World Examples
Let me give you some concrete scenarios where structured objects make all the difference:
Suspicious Activity Report Generation
When you generate a summary of transaction data that amounts to suspicious activity (for example as part of regulatory reporting in a fintech), you want to be sure that the names, amounts, account numbers and other such information contained in the output are exactly those contained in the raw data. Not approximately correct. Exactly right.
Guaranteed Valid Data Queries
Suppose you want your agent to query a database. SQL is a wonderfully expressive language, with a near-infinite space of possible valid queries (let alone invalid ones). How do you constrain your agent to only select from the (much smaller but still huge) space of queries that you know might come up in your particular task?
Extraction of Chart Configs from Images
Suppose you want to take an image of a chart and generate a config to reproduce that chart. Apart from the underlying data query, you will need to describe the properties of the chart itself. These might be interrelated in complex ways:
- The number of series allowed might depend on chart type
- Whether you want a legend will depend on the number of series
These interrelationships might or might not be easy to express through a JSON schema.
Manipulation of Domain Objects Through Chat
Once you’ve created a complex structured object, you might want to modify it, for example via an interactive chat.
A common failing of LLMs is that just as they fix one thing on your request, they tend to randomly modify (read: break) another.
Therefore you do not want to generate a new object each time, because you want to make the smallest possible change that satisfies the user query.
The Evolution of Approaches
So suppose you want to do the above with the help of an LLM, reliably. What would you do?
The Really Naive Approach
Put the description of what you’re looking for into a prompt, send the output to a parser, and hope.
Problem: The result might not even be valid JSON, let alone have the structure you need. You can of course retry if the parsing fails, but there is no way of telling how many retries you will need, and no guarantee of ultimate success.
The Somewhat Less Naive Approach
Set the JSON mode flag of the API to make sure the result is at least a valid JSON.
But will it have the right schema?
Maybe, if you’re lucky.
A Pretty Good Approach
Use function calling or equivalent, to get back a Pydantic object of the type you specified.
In the LLMs whose API allow enforcing a json schema (OpenAI for a while now, and since recently also Anthropic and Google) Langchain will use that, for others it’ll try parsing and retry upon fail.
In Langchain, for example, that is done as follows:
llm = [code to create a BaseLanguageModel for your provider]
struct_llm = llm.with_structured_output(MyPydanticOutputClass)
output = llm.invoke(...)
In a lot of applications, that is already enough.
This approach gives you:
- Guaranteed valid JSON structure
- Type checking at the Python level
- Field validation through Pydantic
- Clear contracts between your LLM and downstream code
But What If It’s Not Enough?
What if you need some validation of the output that is not neatly described by a JSON schema?
- “This field must contain valid markdown”
- “This query must return at least one row of data”
- “The output must contain every single URL and account number that the input contains”
- “Every name occurring in the output must also occur verbatim in the input, and vice versa”
In these cases, we need to enforce on the model output some kind of complex validation logic.
This is where things get interesting.
In the next post, I’ll show you how to handle complex validation requirements using an agentic feedback loop. We’ll look at concrete implementation patterns that ensure your LLM outputs meet arbitrary validation criteria, with real code examples.
Stay tuned for Part 2: Complex Validation with Agentic Feedback Loops.