JSON mode is not structured output
Two features are often confused. JSON mode asks the model to produce syntactically valid JSON; it does not promise that the keys, types or allowed values match what your code expects. Structured output goes further: you supply a JSON Schema and the provider constrains or validates generation so the response conforms to it. When a provider enforces the schema during decoding, a missing field or an unknown enum value cannot appear, which removes a whole class of parsing bugs.
Neither feature says anything about whether the content is right. A schema-valid answer can still pick the wrong label with complete confidence. Treat the schema as a guarantee about shape, and keep evaluation and review for the decision itself.
One schema for a classification decision
The examples on this page use the same small schema: one label restricted by an enum, and a short reason. An enum is the most useful constraint for classification, because it turns an open text field into a closed list your code can switch on. Keep additionalProperties set to false and list every field as required; several providers need both for strict enforcement.
{
"type": "object",
"properties": {
"label": { "type": "string", "enum": ["billing", "technical support", "account access", "other"] },
"reason": { "type": "string" }
},
"required": ["label", "reason"],
"additionalProperties": false
}OpenAI and Gemini
OpenAI Structured Outputs is requested in Chat Completions with response_format set to type json_schema, a name, the schema, and strict set to true; the Responses API takes the same schema under text.format. Strict mode supports a documented subset of JSON Schema, requires every property to be listed as required, and requires additionalProperties false. The Python and JavaScript SDKs can build the schema from a Pydantic model or a Zod object and parse the result for you. When the model declines a request for safety reasons, the answer arrives as a refusal instead of schema output, so handle that branch explicitly. The older json_object mode only guarantees valid JSON.
Gemini takes response_mime_type application/json together with a response_schema in the generation config; the schema format is a subset of the OpenAPI schema object, and newer versions of the API also accept standard JSON Schema. For pure classification Gemini offers a shortcut: response_mime_type text/x.enum with an enum schema returns exactly one of the allowed strings, no JSON wrapper at all. Very large or deeply nested schemas can be rejected, so keep classification schemas flat.
OpenAI Structured Outputs guide · Gemini structured output guide
Anthropic, LangChain and Ollama
With Anthropic models the long-established pattern is tool use: define a tool whose input_schema is your JSON Schema and force it with tool_choice naming that tool, then read the tool input as your structured result. Check the current Anthropic documentation for the models you use, because newer structured output options have been added alongside tool use.
LangChain wraps these provider features behind one call: with_structured_output accepts a Pydantic class, a TypedDict or a JSON Schema and returns parsed objects. Its method argument chooses between function calling, JSON mode and native JSON Schema where the provider supports it. Passing include_raw=True returns the raw message and any parsing error next to the parsed value, which is what you want in production instead of an exception that loses the original response.
Ollama accepts a format field on its chat and generate endpoints. The value json requests any valid JSON; passing a full JSON Schema object constrains the local model to that schema. Ollama recommends also describing the expected structure in the prompt and using a low temperature, because small local models follow schemas less reliably than hosted frontier models.
# OpenAI Chat Completions
response_format = {
"type": "json_schema",
"json_schema": { "name": "ticket_label", "strict": True, "schema": SCHEMA }
}
# Gemini (google-genai SDK)
config = { "response_mime_type": "application/json", "response_schema": TicketLabel }
# LangChain (any supported chat model)
labeller = llm.with_structured_output(TicketLabel, include_raw=True)
# Ollama /api/chat
{ "model": "llama3.1", "messages": [...], "format": SCHEMA, "stream": false }LangChain structured output docs · Ollama structured outputs
What structured output does not solve
A schema cannot tell the model how to choose between two labels that overlap, and it cannot express that a message is too ambiguous to classify unless you add a label for that. It does not produce a calibrated confidence: a field called confidence filled in by the model is just another generated value unless the provider returns token probabilities or you measure agreement yourself.
It also does not carry your review policy. Something in your application still has to decide which results are applied automatically, which are shown to a person, and how corrections are recorded. The more of that logic you rebuild around a raw model call, the closer you are to writing your own decision service.
When a decision API is the simpler choice
If the task is choosing one label from a list, Jev API Pro takes the text, your instructions and the allowed labels, and returns one of those labels with a confidence value when the model supplies it, plus a review flag when the result falls under your threshold. You do not maintain per-provider schema code, refusal handling or a parser. If your task needs free-form extraction of many fields, a provider structured output API is the better tool, and the comparison above tells you which field to set.
Try a classification in the playground · Classify text from Python