What a diffusion LLM is, and what Mercury 3 is like to run in a workspace

Rafael

• 2026-10-19

Almost every language model you have used writes one token at a time. It reads the prompt, predicts the next token, appends it, and runs again. A 500-token answer is 500 passes through the model, one after the other, and no amount of hardware lets the 300th token start before the 299th is done.

Inception's Mercury 3, released today, does not work that way. It is a diffusion language model, and Inception reports 1,068 tokens per second for it on NVIDIA GPUs. As of this release, Curiosity can use it as an LLM provider for chat, AI tools and agents. This post explains what a diffusion LLM does differently, and what we saw when we ran it.

Writing by filling in blanks

Image diffusion models start from noise and remove a little of it at every step until a picture is left. Text has no natural equivalent of Gaussian noise, so the common text version uses masking instead. The Kuleshov group's How to build a diffusion language model describes the result as a "generative BERT".

Training takes a clean sentence, hides a random fraction of its tokens, and asks a transformer to predict the hidden ones. Unlike a next-token model, the transformer sees the whole sequence at once, left and right of every gap. The fraction hidden changes from example to example, so the model learns to work from a nearly complete sentence and from an almost empty one.

Generation runs the process backwards. It starts from a sequence that is all masks. In each round the model predicts every masked position, keeps some of the predictions and masks the rest again, so a few more positions are settled each round:

round 0   [M]  [M]  [M]  [M]  [M]  [M]  [M]
round 1   The  [M]  [M]  [M]  [M]  [M]  mat
round 2   The  cat  [M]  [M]  on   [M]  mat
round 3   The  cat  sat  down on   the  mat

The number of rounds is a setting, and it can be far smaller than the number of tokens. That is where the speed comes from: each round is one pass through the model, and each pass settles many tokens at once. The same post puts the gain at 5 to 10 times faster than autoregressive generation. Inception's Mercury paper reported 1,109 tokens per second for its smallest coding model on an H100, measured by Artificial Analysis.

Where the approach is weaker

Sean Goedecke's Strengths and limitations of diffusion language models lists the costs:

• A fixed output length. A plain masked model fills in a sequence of a set size, say 256 tokens. That is wasted work for a two-word answer.
• Long prompts. A token-by-token model computes the attention state for the prompt once and reuses it (the KV cache). A model that rewrites the whole sequence on every round cannot reuse it as easily, so it is slower at reading a long context.
• Changing course. A reasoning model that writes left to right can notice a mistake halfway and take it back. It is less clear how a model that settles a block at a time does the same.

The first two have a standard fix, block diffusion: generate the answer in blocks from left to right, run diffusion inside each block, and cache the blocks that are done the way an autoregressive model caches its prefix. The answer can then be any length, and the prompt is read once. Mercury 3 accepts a 1M-token context, up from 260K in Mercury 2.5, and Inception has not published how it reads one that long. The third point is open: one commenter in the Hacker News thread on Mercury 3 asks whether a model that writes out of order is harder to follow when you rely on its visible reasoning.

Connecting it to Curiosity

Inception's API is OpenAI-compatible, so the integration is small. In Manage → AI → LLM Providers there is now an Inception Labs entry that talks the OpenAI protocol to https://api.inceptionlabs.ai/v1, with mercury-3, mercury-2.5 and mercury-2 in its model list. Usage is attributed to Inception on the LLM usage page, and the reference prices come from our copy of Simon Willison's llm-prices, which now carries Inception's models.

We timed it from a development workspace:

These are measurements from one container on one day, not a benchmark.

When to try it

Mercury 3 suits work where the response time of each turn matters: an assistant that calls several tools per answer, a support reply drafted while the customer waits, an agent loop with many short steps. Inception says it matches or beats the speed-optimized models of the large labs on its benchmarks, and the new model has not yet been tested independently, so run your own prompts before you switch a production assistant. The Curiosity documentation has the setup steps.

Further reading

• How to build a diffusion language model, Kuleshov group. The training objective and sampling loop, with block diffusion.
• Strengths and limitations of diffusion language models, Sean Goedecke. A short, skeptical account of where diffusion loses to token-by-token generation.
• Mercury: Ultra-fast language models based on diffusion, Inception. The technical report on the first Mercury models, and its Hacker News discussion.
• Mercury 3: Diffusion at scale, Inception. The launch post, with the vendor's benchmark claims.