An AI response can be easy for software to read while still containing the wrong information. Structured output helps organize an answer into expected fields, but a tidy object does not establish that its values match the source. Before sending those values into a report or another action, separate the formatting checks from the evidence checks.
This article uses a small, hand-written example to explain that distinction. The records are fictional fixtures, not responses obtained from an AI model. No model accuracy, product comparison or benchmark result is claimed.
Ask three different questions
First, can the response be parsed as JSON? This concerns syntax: whether the text follows the rules required by the parser. Second, does the parsed object contain the fields and types your application expects? This concerns structure. Third, do those values faithfully represent the source and the task? This concerns meaning and evidence.
These checks can fail independently. An object may omit a required field. It may put text where your application expects a number. Or it may contain the correct fields and types while changing a quantity. A successful structural check should not silently stand in for all three decisions.
A fictional notebook order
Imagine an approved source record describes three notebooks at a unit price of four US dollars. The task is to extract the item, quantity, unit price and currency without changing them. An object containing Notebook, 3, 4 and USD agrees with that fictional source.
Now change only the quantity to 30. The quantity is still a whole number, so a simple type check can pass. The value nevertheless disagrees with the source. Change only the currency to EUR, and the same distinction appears: a string can be structurally acceptable while expressing the wrong currency.
A fourth record leaves out the unit price entirely. It illustrates a different problem: a missing required field. The local demonstration checks required fields and simple types separately from matching the source values. It is intentionally small and is not a complete JSON Schema validator or an example of a real customer order.
Define what an unknown value should mean
Before extraction, decide how the workflow should represent information that the source does not contain. A missing price is not automatically zero. An unreadable date is not necessarily today’s date. Document the allowed treatment of uncertainty and keep it distinct from a verified value.
If the schema permits a null value or a separate review state, explain when it is appropriate. If the task cannot proceed without a value, route the record for clarification rather than inventing a plausible answer. The exact schema and handling rules should match the application and the current provider capabilities.
Keep value checks close to the source
For each field that matters, identify the source location or rule that could confirm it. Preserve units, identifiers and distinctions such as unit price versus total price. Check relationships where relevant, but do not assume that passing an arithmetic check proves every input is correct.
Assign a review path for conflicting, incomplete or unsupported records. Where downstream actions have meaningful consequences, establish what requires human review before the action occurs. Logging the decision and the source reference makes a later correction easier to understand than a bare pass or fail label.
Read provider guarantees narrowly
Google’s structured-output documentation describes schema-based response formatting and recommends validating returned values in application code. It also discusses handling outputs whose meaning is incorrect. Read such documentation for the specific feature and provider you use; do not extend a formatting guarantee into a claim that the underlying facts have been verified.
Evaluate an AI workflow with a small, repeatable test
Source and scope
The three-question worksheet and fictional notebook records are ShamsMag editorial examples. They illustrate validation logic rather than a completed AI evaluation. Cover: AI-generated editorial illustration.
