How much faster can LLM generation actually get?
The short answer: expect 2 to 6 times faster generation in many practical scenarios, with the potential for nearly 8 times when combined with other tricks. The most direct evidence comes from DFlash, a block diffusion framework that achieved over 6x lossless acceleration across a range of models and tasks, outperforming the previous state-of-the-art EAGLE-3 by up to 2.5x [5]. A follow-up, DFlare, scaled the draft model and reported average wall-clock speedups of 5.52x on Qwen3-4B and 5.46x on Qwen3-8B, and 3.91x on a 20B model [2]. These numbers mean that a task that took 10 seconds could take under 2 seconds in the best cases.
Even more impressive, Spiffy showed that speculative decoding can multiply the benefits of other acceleration techniques, achieving total speedups of up to 7.9x when combined with KV-caching and multi-token unmasking [4]. That's a huge practical win for latency-sensitive applications like real-time chat or code completion. However, these are peak numbers; the actual speedup depends on the workload, as we'll see next.
What makes block diffusion drafting so much faster than older methods?
Traditional speculative decoding still generates draft tokens one by one, which creates a bottleneck. Block diffusion drafting breaks that by predicting an entire block of tokens in a single forward pass, so the drafting stage is no longer sequential [5][3]. This is possible because diffusion models can generate multiple positions in parallel, and the draft model is conditioned on the target model's internal features to improve accuracy [5][2].
The key to making this work well is how the draft model uses the target model's knowledge. DFlash initially used a single fused representation from a few target layers, but DFlare improved on that by letting each draft layer attend to its own combination of target layers, which boosted speedups by roughly 11% on 4B models and 8% on 8B models [2]. Another method, DART, uses parallel logit prediction to eliminate autoregressive rollouts entirely, achieving 2.03x–3.44x speedups and beating EAGLE-3 by 30% on average [3]. The takeaway: the more effectively the draft model can leverage the target model's hidden states, the higher the acceptance rate and the faster the generation.
When does block diffusion speculative decoding not deliver?
The biggest caveat is that gains are not uniform. AngelSpec found that no single drafting structure works best across all workloads: block diffusion excels on code and math (predictable, long continuations), while autoregressive multi-token prediction is better for high-entropy open-ended chat [1]. This means a system optimized for code generation might not speed up creative writing as much.
Another issue is stochastic sampling. DBLast showed that when the target model uses non-greedy decoding (which is common for creative tasks), the accepted draft length degrades as the entropy of the sampling distribution increases [6]. In plain terms, the more unpredictable the output, the harder it is for the draft model to guess correctly, reducing the speedup. FlexDraft also highlighted that parallel drafting and verification can suffer at large batch sizes due to uncertainty in the bonus token and accepted length, causing throughput gains to collapse [7]. So, while block diffusion is a powerful tool, it's not a one-size-fits-all solution—it works best for predictable tasks and moderate batch sizes.
Finally, the quality of the draft model matters. DFlare showed that scaling the draft model's depth and training data (from 800K to 2.4M samples) leads to consistent gains [2], but that requires extra training resources. For smaller teams, the overhead might not be worth it unless the workload is well-suited.
About These Sources
This answer is built on 7 studies (all preprints) — published from 2025 to 2026, 7 from 2024 or later — selected as the most relevant from 9 studies that passed quality screening, drawn from 25 papers retrieved from a database of over 500 million.
Sources used in this answer
AngelSpec: Towards Real-World High Performance Inference with Speculative Decoding
AngelSpec shows that block diffusion drafting is better for code/math while autoregressive MTP is better for chat, and their unified framework DFly achieves 1.98-2.40x speedup over autoregressive decoding and 10.5-11.8% higher throughput than DFlash across concurrency levels 4-64.
DFlare: Scaling Up Draft Capacity for Block Diffusion Speculative Decoding
DFlare scales draft capacity with layer-wise fusion, achieving average wall-clock speedups of 5.52x on Qwen3-4B, 5.46x on Qwen3-8B, and 3.91x on GPT-OSS-20B, improving over DFlash by 5-11%.
DART: Diffusion-Inspired Speculative Decoding for Fast LLM Inference
DART eliminates autoregressive drafting by predicting multiple masked positions in parallel, achieving 2.03x-3.44x wall-clock speedups and surpassing EAGLE-3 by 30% on average.
Spiffy: Multiplying Diffusion LLM Acceleration via Lossless Speculative Decoding
Spiffy is a lossless speculative decoding algorithm for diffusion LLMs that achieves 2.8-3.1x speedup alone and up to 7.9x when combined with KV-caching and multi-token unmasking.
DFlash: Block Diffusion for Flash Speculative Decoding
DFlash uses a block diffusion draft model to generate a full block in one forward pass, achieving over 6x lossless acceleration and up to 2.5x higher speedup than EAGLE-3.
DBLAST: Dependent Block Drafting for Stochastic Speculative Decoding
DBLast addresses the mismatch between independent block sampling and stochastic decoding, showing that accepted draft length degrades with higher entropy and proposing a dependent block drafter that improves accepted length on creative-writing benchmarks.
FlexDraft: Flexible Speculative Decoding via Attention Tuning and Bonus-Guided Calibration
FlexDraft introduces attention tuning and bonus-guided calibration to adapt to varying batch sizes, preventing throughput collapse at large batches by dynamically switching between parallel and sequential drafting.