What an extraction model returns that a JSON prompt does not

Rafael

• 2026-10-20

On October 19 Cloudflare released Clairic, a preview model that reads typed values out of a text. You name the fields you want and their types. It answers each one with a value of that type, or with null when the text does not contain it. It answers in one forward pass, writes no tokens, and is billed at $0.10 per million input tokens.

We added it to Curiosity Studio this week, as a second kind of model in StructuredAI next to the decision models. This post is about what its answer contains, because that is where it differs from asking an LLM for JSON.

What a JSON prompt gives you

The usual way to pull fields out of a document is to give an LLM a schema and ask for JSON. Structured output modes now make the shape reliable: the reply parses, and every key is there. They say nothing about the values.

Dan Nguyen's write-up of extracting financial disclosures and a police blotter with OpenAI's Structured Outputs shows what is left. On a clean disclosure page the results looked accurate. On a scanned page turned on its side, the model "got about half of the values right". On the blotter it left a time out of a field and often named the suspect as the victim. It gave "different results every time" he ran the script.

None of those errors shows up in the JSON. A wrong amount and a right one are both a number. A field the document does not have comes back as an empty string, a null, or a plausible invention. The reply does not say which. To find out, you compare against the source by hand.

What Clairic gives you instead

The example is the one on Cloudflare's model page: an invoice of two sentences and six fields. Each answer carries more than its value.

Three of these change what code can do with the answer:

• A span is checkable. A text value is a slice of the input. A reviewer can be shown where it came from, and code can check it is there.
• Null is an answer of its own. The purchase order number is null because the probability that the text has none is 1.00. No prompt told it to "leave it empty if unsure". The model's best guess is kept beside it, so you can see it would have picked the invoice number.
• An estimate has a width. A value the text does not state, such as when the invoice will be paid, comes back as a distribution. A wide interval says the evidence is thin; a narrow one that it is not.

What Cloudflare measured

Cloudflare's post includes a support-claims pipeline run two ways: every step by GLM-5.3, or Clairic extracting, Cloudflare's Clef decision model deciding and GLM-5.3 writing only the reply.

Both pipelines got the facts and the decision right in every run. Clairic gave the same repair estimate each time; GLM-5.3's changed between runs ($250, $250, $220). In a forecast of daily TSA passenger numbers, the average error over 21 days was 3.3%. 17 of the 21 actual values fell inside the 80% interval. Three of the four misses fell on the Columbus Day weekend, which the model does not account for.

These are Cloudflare's numbers on Cloudflare's examples. The post gives no accuracy benchmark across many documents. Read them as a description of the shape of the answer, not as a measure of how often it is right on yours.

Where it falls short

Cloudflare lists the limits of the preview:

• One and two step arithmetic, a total or a day count, worked in their tests. Longer chains of dependent arithmetic and sums over many table rows do not yet. Extract the parts and let code add them up.
• An estimate needs evidence in the input. Without it, the estimate says little.
• Calibration is measured, not guaranteed, and intervals can come out too narrow on data unlike what the model was trained on.
• The forecast follows the pattern of the series it is given. It is not a time-series model.

Two more points matter for an enterprise reader. Clairic runs on Cloudflare's Workers AI, so every input goes to Cloudflare; open weights are planned, with no date. And a typed answer is not a correct answer. The confidence and p_null are there so code can decide what to automate and what to send to a person.

How Studio calls it

In Studio, Clairic is a model of the Cloudflare Workers AI decision provider. An administrator sets it up on Manage > AI > Decision Providers with an API token and the account ID. Code reads fields through StructuredAI.ExtractAsync, with the same generated DecisionProviders constant it uses for decisions:

var invoice = await StructuredAI.ExtractAsync(mail.Body, f => f
    .Text("invoice_number")
    .Date("due_date", "Issue date plus the net payment days")
    .Number("total", "Amount due including shipping", unit: "USD", decimals: 2)
    .Text("po_number", "Purchase order number")
    .Estimate("days_to_pay", 0, 120, 1, "Days after issue until this invoice is paid"),
    StructuredAIOptions.Extraction(DecisionProviders.Cloudflare, userUID: CurrentUser));

bool needsPurchaseOrder = invoice["po_number"].IsNull;

Each input is one request to the account's Workers AI URL, recorded on the LLM usage page. Every field comes back as an ExtractedValue with its span, its confidence, its p_null and, for an estimate, its median and interval.

We checked the request against Cloudflare's published schema and parsed the documented answers in tests. The set-up form reached Cloudflare's API and showed its refusal of a test token. We have not yet run it on a volume of real documents, so we have no accuracy figure of our own to add to Cloudflare's. The Structured Extraction documentation has the field types and the full answer.

Further reading

• Introducing Clairic, a new System One class of extraction models, Cloudflare. The launch post, with the claims pipeline, the receipt example and the forecast.
• Clairic model reference, Cloudflare. The request and answer schemas, the 65,536-token context and the price.
• Clef: open-weight decision models, Cloudflare. The decision models Clairic is paired with in the pipeline above, released under Apache 2.0.
• OpenAI Structured Output demo, Dan Nguyen. A hands-on account of where JSON extraction with an LLM goes wrong.