---
title: "Get structured JSON out of an LLM — reliably"
description: "Asking a model to reply in JSON works until it does not. Learn why tool schemas beat prompt begging, and how to validate every response before it reaches your code."
url: https://articles.sythra.ai/articles/llm-structured-json-output-python
slug: llm-structured-json-output-python
author: "vaibhavkothari"
author_url: https://articles.sythra.ai/writers/vaibhavkothari
date_published: 2026-08-26T08:38:08.169Z
date_modified: 2026-08-29T10:22:15.966Z
topics: ["Python", "Ai", "Coding", "Learning"]
reading_time_minutes: 6
publisher: "Sythra"
publisher_url: https://sythra.ai
access: free
language: en
---

# Get structured JSON out of an LLM — reliably

> Asking a model to reply in JSON works until it does not. Learn why tool schemas beat prompt begging, and how to validate every response before it reaches your code.

Source: https://articles.sythra.ai/articles/llm-structured-json-output-python · Author: vaibhavkothari · Published: 2026-08-26 · Reading time: 6 min · Topics: Python, Ai, Coding, Learning

You ask a model for JSON. It replies:

```text
Sure! Here's the JSON you requested:

\`\`\`json
{"name": "Priya", "age": 29}
\`\`\`

Let me know if you need anything else!
```

`json.loads` chokes. So you write a regex to pull out the braces, and it works — until the day the model returns trailing commas, or a field you never asked for, or an `age` of `"twenty-nine"`.

There is a proper fix, and it is not a better prompt. It is a **schema**.

## Why prompting alone is fragile

Asking politely for JSON leaves three failure modes open:

1. **Wrapper text** — explanations before and after
2. **Wrong types** — `"29"` instead of `29`, `"yes"` instead of `true`
3. **Missing or extra fields** — no `age`, or a bonus `notes` key

Each is rare. Across thousands of calls, rare becomes constant. And every one of them lands as a crash in production rather than a warning in development.

## Step 1 — Define the shape you want

Write the schema before writing the prompt. JSON Schema is the vocabulary both you and the model understand:

```python
INVOICE_SCHEMA = {
    "type": "object",
    "properties": {
        "vendor": {"type": "string", "description": "Company that issued the invoice"},
        "invoice_number": {"type": "string"},
        "total": {"type": "number", "description": "Total amount, digits only"},
        "currency": {"type": "string", "enum": ["INR", "USD", "EUR", "GBP"]},
        "line_items": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "description": {"type": "string"},
                    "amount": {"type": "number"},
                },
                "required": ["description", "amount"],
            },
        },
    },
    "required": ["vendor", "invoice_number", "total", "currency"],
}
```

Two details do most of the work. The `description` on each field is read by the model — treat it as instruction, not documentation. And `enum` removes an entire class of variation: currency can only ever be one of four strings.

## Step 2 — Use tools instead of asking nicely

Every serious model API supports tool use, and it is the reliable path to structured output. You describe a "tool" whose input *is* your schema, then force the model to call it:

```python
import os
import json
import anthropic

client = anthropic.Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])

def extract_invoice(text):
    message = client.messages.create(
        model="claude-sonnet-5",
        max_tokens=1500,
        tools=[{
            "name": "record_invoice",
            "description": "Record the structured details of an invoice",
            "input_schema": INVOICE_SCHEMA,
        }],
        tool_choice={"type": "tool", "name": "record_invoice"},
        messages=[{"role": "user", "content": text}],
    )

    for block in message.content:
        if block.type == "tool_use":
            return block.input

    raise ValueError("Model did not call the tool")
```

`tool_choice` is the important line. It removes the model's option to write prose instead, so the response arrives as a parsed Python dict — no code fences, no "Sure!", no regex.

## Step 3 — Validate anyway

A schema guides the model; it does not physically prevent every mistake. Validate at the boundary, where a bad value can still be caught cheaply:

```bash
python -m pip install pydantic
```

```python
from typing import Literal
from pydantic import BaseModel, Field, ValidationError

class LineItem(BaseModel):
    description: str
    amount: float

class Invoice(BaseModel):
    vendor: str
    invoice_number: str
    total: float = Field(gt=0)
    currency: Literal["INR", "USD", "EUR", "GBP"]
    line_items: list[LineItem] = []


def parse_invoice(raw):
    try:
        return Invoice.model_validate(raw)
    except ValidationError as error:
        print("Validation failed:", error)
        return None
```

Pydantic gives you real Python objects with autocompletion, plus rules a JSON Schema cannot express as neatly — `Field(gt=0)` rejects a negative total outright.

Pydantic can also generate the schema, so the two never drift apart:

```python
INVOICE_SCHEMA = Invoice.model_json_schema()
```

Define the shape once, in one place. This is the version to use in real projects.

## Step 4 — Retry on failure, with the error attached

When validation fails, do not retry blindly. Tell the model what was wrong:

```python
def extract_with_retry(text, attempts=3):
    last_error = None

    for attempt in range(attempts):
        prompt = text
        if last_error:
            prompt = f"{text}\n\nYour previous attempt failed validation:\n{last_error}\nFix it."

        raw = extract_invoice(prompt)
        try:
            return Invoice.model_validate(raw)
        except ValidationError as error:
            last_error = str(error)
            print(f"Attempt {attempt + 1} failed, retrying")

    raise ValueError(f"Gave up after {attempts} attempts: {last_error}")
```

Models correct themselves well when shown the specific error. Two attempts fix the overwhelming majority of failures.

## Step 5 — Fall back to text parsing when you must

For an API without tool support, harden your parsing:

```python
import json
import re

def extract_json(text):
    fenced = re.search(r"```(?:json)?\s*(\{.*?\}|\[.*?\])\s*```", text, re.S)
    if fenced:
        candidate = fenced.group(1)
    else:
        start = min((i for i in (text.find("{"), text.find("[")) if i != -1), default=-1)
        if start == -1:
            raise ValueError("No JSON found")
        end = max(text.rfind("}"), text.rfind("]"))
        candidate = text[start:end + 1]

    candidate = re.sub(r",\s*([}\]])", r"\1", candidate)   # trailing commas
    return json.loads(candidate)
```

Prefer the fenced block when present, fall back to outermost braces, strip trailing commas. It is a workaround, not a solution — use tools when you can.

## Step 6 — Set temperature to 0 for extraction

```python
message = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=1500,
    temperature=0,
    tools=[...],
    tool_choice={"type": "tool", "name": "record_invoice"},
    messages=[{"role": "user", "content": text}],
)
```

Temperature should be 0 for extraction and classification. Creativity is a feature when writing and a bug when parsing an invoice.

## A complete extraction pipeline

```python
def process_documents(paths):
    results, failures = [], []

    for path in paths:
        text = open(path, encoding="utf-8").read()
        try:
            invoice = extract_with_retry(text)
            results.append(invoice.model_dump())
        except Exception as error:
            failures.append({"path": path, "error": str(error)})

    print(f"{len(results)} succeeded, {len(failures)} failed")
    return results, failures
```

Collect failures rather than crashing on the first one. Batch jobs should always finish and report.

## Common problems (and fixes)

| Problem | Fix |
|---|---|
| Model writes prose around the JSON | Use tools with `tool_choice` |
| Numbers arrive as strings | Let Pydantic coerce, or state "digits only" in the description |
| Enum values drift | Add `enum` to the schema and `Literal` in Pydantic |
| Fields missing | List them in `required` |
| Nulls where you expect values | Make the field optional with a default |
| Inconsistent across runs | Set `temperature=0` |

```remember
# Remember this
Asking nicely for JSON -> fails eventually, at scale
Schema-constrained output -> makes malformed JSON impossible
Validate anyway -> the shape can be right and the values wrong
---
Retry with the validation error attached. The model fixes its own output
far more reliably than a second attempt at the same prompt.
```


## FAQ

### How do I get reliable JSON output from an LLM?

Define a JSON Schema and pass it as a tool with `tool_choice` set to that tool. The response then arrives as a parsed object rather than text you have to scrape.

### Why does the model add text around my JSON?

Because a plain prompt allows it to. Forcing a tool call removes the option to produce prose, which is why tools are far more reliable than prompt instructions alone.

### Should I use Pydantic with LLM output?

Yes. Generate the schema from your Pydantic model so there is a single definition, then validate every response before it reaches the rest of your code.

### What temperature should I use for structured extraction?

Zero. Extraction and classification want determinism; randomness only helps when you are generating creative text.

### How should I handle a response that fails validation?

Retry with the validation error included in the prompt so the model can correct it. Two attempts resolve almost all failures; log the rest instead of crashing the batch.

## Next reading on Sythra Articles

- [Talk to an AI from Python in about 20 lines](https://articles.sythra.ai/articles/python-talk-to-llm-20-lines)
- [Build a chatbot that answers from your own PDFs](https://articles.sythra.ai/articles/rag-chatbot-pdf-python-claude)
- [Stop memorizing Python syntax](https://articles.sythra.ai/articles/seven-python-patterns-for-ml-notebooks)

## Glossary (terms defined in this article)
- **JSON Schema** (medium) — A standard way to describe the exact shape of a JSON object
- **Pydantic** (medium) — A Python library that checks data against a typed model and converts it
- **Temperature** (basic) — How much randomness the model uses when choosing words