What Is DFlash 2? Speculative Decoding Explained for Qwen3.8-27B
DFlash 2 is a block-diffusion draft model that speeds up Qwen3.8-27B inference up to 3.4x. Here's how it works and what it beats.
What is DFlash 2?
DFlash 2 is a draft model built for speculative decoding with Qwen3.8-27B. Instead of guessing one token at a time like older draft models, it predicts a whole block of tokens in a single forward pass, keeps multiple candidates at each position, and uses a small selector to pick one coherent sequence through them. Paired with Qwen3.8-27B in SGLang or vLLM, it pushes output throughput up to 3.43x over plain autoregressive decoding, and it does so losslessly: greedy output matches the target model exactly, and sampled output preserves the same distribution.
TL;DR
- DFlash 2 is a block-diffusion drafter, not a standalone chat model. It only works attached to a target model (here, Qwen3.8-27B) inside a speculative decoding server.
- It drafts a full block of tokens per pass (block size 8, 7 draft tokens) instead of guessing sequentially, then a lightweight selector traces the single best path through the candidates.
- On a single NVIDIA H200 with concurrency 1, DFlash 2 hits 236.1 tokens/sec on GSM8K, a 3.43x speedup over autoregressive decoding, versus 2.59x for Qwen3.8’s built-in MTP drafter.
- Its acceptance length (tokens accepted per verification step) beats both MTP and the community DSpark drafter across every benchmark tested, including GSM8K, MATH-500, HumanEval, MBPP, and MT-Bench.
- The speedup shrinks as concurrency rises: at concurrency 32, gains drop to roughly 1.0x to 1.45x depending on the task, because the GPU is already compute-bound with many parallel requests.
- Decoding stays lossless, meaning DFlash 2 changes speed, not output quality or correctness.
- It’s a drop-in draft model for SGLang and vLLM, requiring only a few extra launch flags rather than a custom serving stack.
Remy doesn't build the plumbing. It inherits it.
Other agents wire up auth, databases, models, and integrations from scratch every time you ask them to build something.
Remy ships with all of it from MindStudio — so every cycle goes into the app you actually want.
How does speculative decoding work in the first place?
Autoregressive decoding generates one token, feeds it back into the model, generates the next token, and repeats. Each step requires a full forward pass through the target model, which is why large language models are slow to stream out text: you pay the full compute cost of a 27-billion-parameter model for every single token.
Speculative decoding breaks that bottleneck by adding a second, much cheaper model, the “draft” model, that guesses several tokens ahead. The big target model then checks all of those guesses in one parallel forward pass instead of generating them one by one. If the draft’s guesses match what the target model would have produced anyway, they get accepted for free. If a guess is wrong, the target model falls back to its own token at that position and the draft continues from there. Because verification happens in parallel, a correct multi-token guess costs about the same as generating a single token normally, which is where the speed comes from. Crucially, the target model still decides every final token, so the output is identical to what plain autoregressive decoding would have produced.
The efficiency of this scheme hinges on “acceptance length”: the average number of drafted tokens that survive verification before the target model has to intervene. A better draft model produces longer accepted runs, which means fewer expensive verification steps per token generated.
What makes DFlash 2’s block-diffusion approach different?
Most draft models, including Qwen3.8’s built-in multi-token prediction (MTP) head, still generate their draft tokens sequentially: guess token one, then use that guess to help predict token two, and so on. Errors compound as the block gets longer, since a bad early guess drags down everything after it.
DFlash 2 takes a different approach borrowed from diffusion models. It predicts an entire block of tokens (eight positions in the benchmarked configuration) in one pass, and at each position it keeps several plausible candidates rather than committing to a single guess immediately. A lightweight selector then looks across the whole block and traces one coherent path through the candidates, choosing the sequence most likely to match what the target model would generate.
The model card also mentions “two-tap dynamic convolutions” built into the drafter’s backbone. Their stated job is to stop the draft’s quality from decaying toward the end of the block, which is a common failure mode when a model has to commit to guesses seven or eight tokens ahead without seeing how earlier guesses play out. By addressing that decay directly, DFlash 2 keeps its later token predictions almost as reliable as its early ones, which lengthens average acceptance runs.
How much faster is DFlash 2, in numbers?
The published benchmarks run Qwen3.8-27B on a single NVIDIA H200 GPU with FlashAttention 3, comparing four decoding setups: plain autoregressive decoding, Qwen3.8’s built-in seven-token MTP head, a community drafter called DSpark, and DFlash 2. All three speculative methods propose seven draft tokens per verification step.
One coffee. One working app.
You bring the idea. Remy manages the project.
At concurrency 1 (a single request at a time, the scenario where speculative decoding helps most):
- GSM8K: 68.9 tokens/sec autoregressive versus 236.1 tokens/sec with DFlash 2, a 3.43x speedup.
- MATH-500: 3.34x speedup.
- HumanEval: 3.11x speedup.
- MBPP: 3.29x speedup.
- MT-Bench: 2.67x speedup, the smallest gain among the five tasks tested.
For comparison, MTP’s speedups on the same tasks range from 1.96x to 2.59x, and DSpark’s range from 2.00x to 2.69x. DFlash 2 leads on every single benchmark.
At concurrency 8, the gap narrows but DFlash 2 still leads clearly, with speedups between 2.27x and 2.85x versus MTP’s 1.74x to 2.19x. At concurrency 32, gains compress further across all methods: DFlash 2 ranges from about 1.01x (MT-Bench) to 1.45x (GSM8K), while MTP and DSpark actually fall below 1.0x on some tasks, meaning they’re slower than plain autoregressive decoding once the GPU is saturated with 32 simultaneous requests. This pattern makes sense: speculative decoding’s advantage comes from using otherwise-idle compute to verify multiple tokens at once, and that idle compute disappears as concurrency climbs and the GPU becomes the bottleneck instead of sequential dependency.
Why does acceptance length matter more than raw speed here?
Acceptance length, the average number of tokens accepted per verification step, is the more fundamental metric because it isolates draft quality from hardware effects. DFlash 2 posts the highest acceptance length on every benchmark: 5.46 on GSM8K, 5.28 on MATH-500, 4.39 on HumanEval, 4.79 on MBPP, and 4.10 on MT-Bench, all measured against a maximum possible acceptance of 8 (the block size). MTP trails behind on all five, and DSpark trails further still.
Since every speculative method here proposes seven draft tokens per step, a higher acceptance length directly means fewer wasted guesses and fewer verification round trips per output token. That’s the mechanism, independent of GPU or batch size, that produces DFlash 2’s throughput advantage.
Is DFlash 2 worth adopting?
For anyone already serving Qwen3.8-27B and running single-request or lightly-batched workloads (concurrency in the 1 to 8 range), DFlash 2 offers a substantial free speedup with no quality tradeoff, since the decoding is lossless by construction. It’s also a straightforward swap: both SGLang and vLLM support it through a handful of command-line flags, with no custom infrastructure needed beyond installing the relevant development branches.
The caveat is that gains shrink sharply at high concurrency. If a deployment is already running large batches of simultaneous requests, the GPU is likely compute-bound already, and speculative decoding of any kind, DFlash 2 included, will deliver smaller returns. Teams should benchmark at their actual expected concurrency rather than assuming the concurrency-1 numbers will hold.
Frequently Asked Questions
What is speculative decoding?
It’s a technique where a small, fast draft model guesses several upcoming tokens, and a larger target model verifies those guesses in one parallel pass instead of generating tokens one at a time. Correct guesses are accepted for free; wrong ones fall back to the target model’s own output, so the final text is unchanged.
Does DFlash 2 work with any model, or just Qwen3.8-27B?
Other agents ship a demo. Remy ships an app.
Real backend. Real database. Real auth. Real plumbing. Remy has it all.
The published checkpoint is trained specifically as a draft model for Qwen3.8-27B. It is not a general-purpose language model and won’t run standalone; it must be paired with that specific target model inside a speculative decoding server like SGLang or vLLM.
How is DFlash 2 different from Qwen3.8’s built-in MTP head?
MTP (multi-token prediction) drafts tokens sequentially within its block. DFlash 2 instead predicts an entire block at once using a block-diffusion approach, keeping multiple candidates per position and selecting one coherent path afterward. Across every tested benchmark, DFlash 2 achieves longer accepted token runs and higher throughput than MTP.
Does using DFlash 2 change the quality of generated text?
No. Speculative decoding with DFlash 2 is lossless: greedy decoding produces output identical to running the target model alone, and sampled decoding preserves the same probability distribution. The only thing that changes is speed.
Why do speedups drop at higher concurrency?
Speculative decoding gains its speed by using spare GPU compute to verify several tokens per step in parallel. At low concurrency, that compute would otherwise sit idle. At high concurrency, the GPU is already busy processing many simultaneous requests, so there’s less idle capacity to exploit, and the relative speedup shrinks toward 1.0x or, in some cases for other drafters, below it.
