Diffusion LLMs vs Autoregressive Models: What's the Difference?
Diffusion language models generate text in parallel passes instead of one token at a time. Here's how that changes speed, cost, and design.
What’s the core difference between diffusion and autoregressive LLMs?
Autoregressive models write one token at a time, in order, and each token requires a full forward pass through the model. That’s how almost every mainstream LLM works today. Diffusion language models instead start with a masked, rough draft of the entire response and refine it over a handful of passes, resolving many tokens in parallel each time. The autoregressive model works like a typewriter. The diffusion model works like an editor revising a whole draft at once. Both can end up producing fluent, correct text, but the mechanics of getting there, and the speed that results, are very different.
TL;DR
- Autoregressive LLMs generate sequentially, committing to one token per forward pass, which makes decoding speed proportional to output length.
- Diffusion LLMs generate in parallel, starting from a masked draft and refining many tokens per pass, so each pass does more work and total generation time drops sharply.
- The speed advantage comes from memory bandwidth, not raw compute: since inference is typically bandwidth-bound, extracting more tokens per pass is where diffusion models gain ground.
- Diffusion text generation borrows directly from image and video diffusion models, the same family of techniques behind modern image generators, applied to language instead of pixels.
- For a long time diffusion LLMs traded intelligence for speed, useful for autocomplete-style tasks but not considered strong enough for serious agentic work.
- Inception’s Mercury model line is one concrete example of this gap closing, with Mercury 3 reportedly running at over 1,000 tokens per second while scoring in a similar intelligence range to other small-to-mid models like GPT-6 Luna and Claude Haiku 5.5 on Inception’s own benchmarks.
- Decision models are a related but separate idea: Mercury Decide is built to output a judgment and a confidence score in one fast call rather than generating free text, aimed at the high-volume “system one” checks that agent pipelines constantly make.
Everyone else built a construction worker.
We built the contractor.
One file at a time.
UI, API, database, deploy.
How does an autoregressive model actually generate text?
An autoregressive LLM predicts the next token based on everything that came before it, then feeds that token back in as context for predicting the next one. Repeat until the response is done. This is simple, well understood, and it’s the architecture behind the vast majority of production LLMs. But it has a structural cost: every single token costs a full pass through the model, and once a token is committed, the model generally can’t revise it. Some research has explored limited backtracking, but the default behavior is strictly left-to-right, one step at a time. That sequential dependency is the reason decoding speed scales with output length almost no matter how powerful the underlying hardware is.
How does a diffusion LLM generate text differently?
A diffusion LLM starts from a fully masked (blank or noisy) version of the entire response and iteratively denoises it, filling in and correcting tokens across the whole sequence in parallel over several passes. Instead of locking in word one, then word two, then word three, the model works on a rough skeleton of the full answer and sharpens it step by step. Watching this happen in slow motion (as a frame-by-frame replay shows) looks like paragraphs loosely taking shape and then being refined into their final form, rather than text being typed left to right.
Because each pass touches many tokens at once instead of one, the model needs fewer total passes to produce a finished response. And since LLM inference tends to be limited by memory bandwidth more than raw compute, cramming more useful token updates into each pass is exactly where the speed gain comes from. This is the same basic idea that underlies most modern image and video generators, just applied to text tokens instead of pixels. It’s not a coincidence: some of the people building diffusion language models today also worked on foundational diffusion techniques for image and video generation.
Why did diffusion LLMs used to lag behind on intelligence?
For most of their history, diffusion language models were treated as the fast option rather than the capable option, suitable for low-stakes jobs like autocomplete but not trusted with real agentic tasks. The open question in the field was whether diffusion could scale the way autoregressive transformers had, where bigger models and more training reliably bought more capability. As recently as mid-2024, that was still genuinely unresolved among researchers working on diffusion methods.
One coffee. One working app.
You bring the idea. Remy manages the project.
That changed as model builders iterated through multiple generations of diffusion LLMs, scaling both parameter count and training quality. In the case of Inception’s Mercury line, parameter counts reportedly grew more than 40x from an early “mini” version to the latest Mercury 3, with the company’s own intelligence average rising roughly 3x over that span. Notably, a mid-series jump (from Mercury 2 to Mercury 2.5) came from training improvements at roughly the same model size, while the move to Mercury 3 involved a genuine scale-up. That suggests both levers, better training and bigger models, independently improve diffusion LLMs, which is the same pattern that made scaling work for autoregressive transformers in the first place.
Is a diffusion LLM actually faster in practice?
Yes, and the gap can be large. Mercury 3 is reported to generate over 1,000 tokens per second on Nvidia Blackwell hardware, several times faster than comparable autoregressive models in its class. For latency-sensitive products, that difference is the whole point. One example cited by Inception involves a search product built for smart glasses, where a prior model’s generation time was the dominant bottleneck in the user experience; switching to a faster diffusion model cut response time to under a second. In an interface where anything taking several seconds reads as “broken,” that kind of speed matters more than squeezing out another benchmark point.
Pricing for Mercury 3 was set around 10 cents per million input tokens and 35 cents per million output tokens at launch, with cached inputs offered at a steep discount, which matters for agents that resend large chunks of repeated context on every step. The pitch isn’t that diffusion is cheaper per token than every rival. It’s that you’re paying for speed, and in agentic workflows that call a model repeatedly, speed compounds.
Are diffusion LLMs good enough for agentic or coding tasks?
They’re increasingly competitive for a meaningful slice of agentic work, though not uniformly at the frontier. On benchmarks covering reasoning, instruction following, and search tasks, Mercury 3 reportedly performs in a similar range to other compact models like GPT-6 Luna and Claude Haiku 5.5, including strong results on GPQA Diamond. On coding-specific benchmarks it trails top-tier models like Opus 5.5, but it’s positioned as well suited to medium-complexity coding tasks and as a default model for coding sub-agents, rather than as a replacement for a frontier reasoning model on the hardest problems.
Where the speed advantage becomes a design choice rather than just a convenience is in architectures that use many agents at once: recursive self-improvement loops, parsing large codebases, or running swarms of sub-agents across a big dataset. A model that responds in a fraction of a second changes what’s economically and practically feasible to run at scale, even if its raw intelligence ceiling is a notch below the biggest reasoning models.
What is a decision model and how does it relate to diffusion LLMs?
A decision model is a narrower tool than a general LLM. Instead of generating free-form text, it takes in some state (which can be text or, in newer versions, images) plus a specific typed question, and returns a direct answer along with a confidence score, in a single fast call. Mercury Decide is Inception’s diffusion-based take on this category. It’s meant for the high-frequency “system one” judgments that agent pipelines constantly need: is this support ticket urgent, is an agent stuck, did a page actually load, does this output pass a compliance check.
Remy doesn't write the code. It manages the agents who do.
Remy runs the project. The specialists do the work. You work with the PM, not the implementers.
Independent evaluations cited for Mercury Decide show it matching or beating comparable decision models on tasks like claim verification and legal benchmark scoring, at a fraction of the cost, with single-judgment latency reported around 0.16 seconds and batched judgments (64 at once) around 0.4 seconds. The hallucination rate on at least one third-party decision index puts it among the better-calibrated options tested, which matters specifically because a decision model’s value depends on it saying “not sure” instead of confidently guessing. The broader pattern this points to: pair cheap, fast decision calls for routine checks with a slower, more expensive reasoning model reserved for the parts of a task that actually need deep thought, escalating to a human when confidence is low.
Frequently Asked Questions
Do diffusion LLMs produce lower-quality text than autoregressive models?
Not inherently. The final output from a well-trained diffusion LLM can read just as naturally as autoregressive output. The historical gap was in overall benchmark intelligence at comparable model sizes, not textual coherence, and that gap has been narrowing as diffusion models scale.
Why are diffusion LLMs faster if they use more passes than “one token per pass”?
Because each pass resolves many tokens simultaneously rather than one. Inference is generally bottlenecked by memory bandwidth rather than compute, so a method that extracts more finished tokens per pass reduces the number of expensive round trips through the model, even though it is iterative rather than fully one-shot.
Can diffusion LLMs handle long outputs like long essays reliably?
Diffusion models generate the full draft structure up front and refine it in chunks, which in testing has shown fewer instances of stopping halfway through a long piece or describing what it “would” write instead of writing it, a known failure mode in some autoregressive models.
Are diffusion LLMs a replacement for autoregressive models everywhere?
Not currently. They’re strongest where speed and cost per call matter most, like high-volume agent loops, sub-agent swarms, and latency-sensitive interfaces. For the hardest reasoning and coding tasks, top autoregressive frontier models still lead on raw capability.
What is Mercury Decide used for if it doesn’t generate text?
It’s built for fast, structured judgments rather than open-ended generation, things like classifying, verifying, or scoring a given state against a specific question, returning an answer plus a confidence level in a fraction of a second, which suits automated checks inside larger agent systems.