DSpine vs DFlash2: Which Speculative Decoding Draft Model Wins?
A hands-on H100 benchmark pits Tencent's DSpine against Z-Lab's DFlash2 on Qwen3.8-27B, comparing real tokens-per-second results.
What is DSpine, and how does it compare to DFlash2 for speculative decoding?
DSpine, built by Tencent, is a small draft model that speeds up Qwen3.8-27B by guessing multiple tokens ahead before the big model verifies them. In a direct test on the same H100 GPU with the same prompts, DSpine outperformed Z-Lab’s DFlash2, another popular draft model, by roughly 14 to 16 percent depending on prompt length. Both models rely on speculative decoding, but they draft tokens differently: DSpine passes predicted features between positions at every network layer, while DFlash2 blends neighboring guesses using small convolutions before picking a final path.
TL;DR
- DSpine beat DFlash2 by about 16 percent overall in a same-GPU, same-prompt benchmark on Qwen3.8-27B, hitting roughly 150 tokens per second versus DFlash2’s 129.
- Both models are draft helpers for speculative decoding, a technique where a small fast model guesses several tokens ahead and the large model verifies them in one pass instead of generating one token at a time.
- DSpine uses a block size of 16 draft tokens while DFlash2’s official setting uses 8, and DSpine’s architecture passes predicted features between adjacent positions across its five layers to keep guesses consistent.
- Tencent’s own benchmarks show DSpine accepting more tokens per round than DFlash2 across tasks like GSM8K (7.20 vs 5.69 at temperature 0) and MATH-500 (6.49 vs 5.49), which tracks with the real-world speed gap.
- DSpine uses more VRAM than DFlash2 in practice (about 78GB versus 73GB on an 80GB H100), a tradeoff worth factoring in before deploying either on memory-constrained hardware.
- Neither model changes the final output of the base model. Speculative decoding only speeds up generation; it does not alter what the large model would have produced anyway.
Production- Self-hosted
- SOC 2
- HIPAA
- Audit log
How does speculative decoding actually work?
Normally, a large language model generates one token at a time, and each token requires a full forward pass through the entire network. That’s slow, especially for long outputs. Speculative decoding fixes this bottleneck by pairing the big model with a much smaller “draft” model that predicts several tokens ahead in one shot.
The big model then checks all of those guessed tokens in a single pass, rather than generating them one by one. It keeps whichever guesses turn out to be correct and discards the rest, restarting generation from the first wrong token. Because verifying a block of tokens costs roughly the same as generating one token the normal way, every accepted guess is essentially free speed. The final text is identical to what the large model would have produced on its own: speculative decoding doesn’t change quality, it just skips redundant work.
The practical upshot: the more tokens the draft model gets right in a row, the bigger the speedup. That’s why “acceptance length” (how many draft tokens get confirmed per round) is the key metric separating draft models from each other.
What makes DSpine different from DFlash2?
Both models are compact helpers (DSpine’s checkpoint for Qwen3.8-27B uses five layers) that sit alongside the large model and generate draft tokens in parallel rather than one at a time. The difference is in how they keep those parallel guesses coherent with one another, since predicting multiple tokens at once risks the guesses contradicting each other.
DSpine handles this by passing predicted features between adjacent positions at every layer of its network. Each position’s guess is informed by its neighbor’s, carrying dependency information through the model’s depth. At the end, it resolves the final token sequence left to right using a lookup table. Tencent’s official release configures DSpine to draft in blocks of 16 tokens, with an 8,192-token sliding window, and recommends running it through SGLang with FlashAttention 3 and a dynamic verification method borrowed from DeepSeek’s DSpark project.
DFlash2, from the Z-Lab team, takes a related but distinct approach. It also drafts an entire block of tokens in parallel, but reconciles the guesses by blending neighboring predictions inside the model using small convolutions, then selecting the best overall path at the end. Its official configuration drafts 8 tokens at a time, half of DSpine’s block size.
What did the head-to-head benchmark actually measure?
The test ran both draft models against the same base model (Qwen3.8-27B), on the same Nvidia H100 GPU with 80GB of VRAM, using SGLang as the serving framework. The benchmark script sent 16 distinct prompts (8 short, 8 requiring longer or more reasoning-heavy answers), each run twice for 32 total requests, at temperature zero to keep results repeatable. Tokens-per-second was measured for every individual request plus overall averages.
DFlash2 was served using its official default of 8 draft tokens per block, consuming about 73GB of VRAM. DSpine was served with its default block size of 16 draft tokens, consuming closer to 78GB. Everything else (memory fraction, context length, request limits) was matched between runs to keep the comparison fair.
Results: DFlash2 generated about 129 tokens per second overall (133 on short prompts, 128 on long ones), with individual requests ranging from roughly 91 to 168 tokens per second. DSpine generated about 150 tokens per second overall (152 on short prompts, 150 on long ones), with individual requests ranging from about 88 to 227. That puts DSpine ahead by roughly 14 percent on short prompts, 16 percent on long prompts, and 16 percent overall in this particular test.
These numbers align directionally with Tencent’s own published acceptance-length figures, which show DSpine accepting more draft tokens per round than DFlash2 across benchmarks including GSM8K, MATH-500, HumanEval, MBPP, and MT-Bench, at both greedy decoding and temperature sampling.
Is DSpine worth switching to over DFlash2?
Based on this test, DSpine delivered a meaningful speed advantage, consistently around 15 percent faster across both short and long prompts on identical hardware. For teams running high-throughput inference where every percentage point of latency matters, that’s a real gain, particularly for agentic workloads with long tool-call chains where Tencent’s own benchmarks show even larger speedups (DSpine hit up to 3.84x over standard autoregressive decoding on WorkBuddy Office tasks in the “low” reasoning setting, versus DFlash2’s 3.17x).
The tradeoff is memory. DSpine’s larger default block size and additional runtime flags (like its ReplaySSM-based dynamic verification) push VRAM usage higher, around 78GB versus DFlash2’s 73GB in this setup. On an 80GB H100, that’s survivable, but it leaves very little headroom for larger batches or longer contexts. Teams running on smaller GPUs or needing to serve many concurrent requests may find DFlash2’s lighter footprint more practical even if it’s somewhat slower per request.
It’s also worth noting this was a single-GPU, single-request-at-a-time test. Results could shift under heavier concurrency, different batch sizes, or other hardware. Still, as a first real-world comparison beyond vendor-reported numbers, it’s a useful data point for anyone deciding which draft model to pair with Qwen3.8-27B.
Frequently Asked Questions
What is a draft model in speculative decoding?
A draft model is a small, fast neural network that predicts several upcoming tokens before the large model verifies them. Because verification happens in a single batched pass rather than token by token, correct guesses translate directly into faster generation without changing the final output.
Does using DSpine or DFlash2 change the quality of the model’s answers?
No. Speculative decoding is a decoding-time optimization, not a change to the model’s weights or reasoning. The large model still verifies every draft token, and the final output matches what it would have generated without any draft model. Only the speed changes.
Why does DSpine use more VRAM than DFlash2?
DSpine’s official configuration uses a larger draft block size (16 tokens versus DFlash2’s 8) and additional inference machinery, including a dynamic verification method and a sliding-window cache, which together increase memory overhead. In the benchmarked setup this added roughly 5GB of VRAM usage on an 80GB H100.
Can DSpine and DFlash2 be used with models other than Qwen3.8-27B?
Enterprise-ready- Self-hosted
- SOC 2
- HIPAA
- Audit log
- Role-based access
Both are released as model-specific checkpoints trained to pair with a particular base model’s architecture and tokenizer. The checkpoints discussed here target Qwen3.8-27B specifically; using them with a different base model would require a separately trained draft checkpoint.
Which framework do you need to run these speculative decoding setups?
Both DSpine and DFlash2 in this benchmark were served through SGLang, an open-source inference engine with built-in support for speculative decoding algorithms, FlashAttention 3, and the specific flags each draft model requires.
