Why does throughput collapse under high concurrency?
The biggest production risk is that block diffusion drafters generate long token sequences in one pass, but verifying those blocks indiscriminately can waste critical batch capacity on tokens that are likely to be rejected. DSpark [1] demonstrated this in a live serving system (DeepSeek-V4): when verification length was not adapted to each request, throughput degraded severely under strict interactivity constraints. By dynamically tailoring verification length based on prefix survival probabilities and engine-specific throughput profiles, DSpark improved per-user generation speeds by 60–85% at matched throughput levels, and enabled performance tiers that were previously unattainable. This shows that in production, the failure mode isn't just raw speed—it's the interaction between draft quality and batch scheduling.
How does draft quality decay with block length and sampling temperature?
Block diffusion drafters predict multiple future tokens in parallel, but because they sample from position-wise marginals rather than fully conditioned sequences, they often fail to capture the target model's preferred trajectory. This is especially problematic when decoding is stochastic (non-greedy), as DBLast [3] showed: accepted draft length degrades as the entropy of the target sampling distribution increases. In high-entropy regimes, independent block sampling becomes brittle, and DBLast's dependent block drafter (using a low-rank latent mixture) consistently improved accepted length over independent sampling. Similarly, DSpark [1] noted that parallel drafters suffer from rapid acceptance decay due to lack of inter-token dependencies, which is why they added a lightweight sequential module to model intra-block dependencies. The takeaway: if your production workload uses sampling with temperature > 0, you need a drafter that models dependencies, not just a parallel predictor.
Why is a fixed block size a failure mode in production?
Most block diffusion methods use a fixed inference block size, but BlockPilot [5] showed that the optimal block size varies across samples and is critical to performance. They found that optimal block sizes concentrate around the training block size, but still vary enough that a one-size-fits-all approach is suboptimal. BlockPilot predicts the optimal block size from the prefilling representation, achieving an acceptance length of 5.92 and a 4.20× speedup on Qwen3-4B at temperature T=1. This is a clear signal that production systems must adapt block size per request, not just per workload. The same theme appears in Bastion [2], which dynamically constructs query-dependent draft trees based on hardware constraints, and in D^2SD [4], which uses confidence scores to select the most likely rejection boundary and re-anchor alternative continuations. All these approaches point to the same conclusion: static configurations are a liability in production.
About These Sources
This answer is built on 5 studies (all preprints) — published in 2026, 5 from 2024 or later — selected as the most relevant from 8 studies that passed quality screening, drawn from 33 papers retrieved from a database of over 500 million.
Sources used in this answer
DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation
DSpark, deployed in DeepSeek-V4 serving, showed that indiscriminate verification of long blocks wastes batch capacity and degrades throughput; its confidence-scheduled verification improved per-user generation speeds by 60–85% at matched throughput.
Bastion: Budget-Aware Speculative Decoding with Tree-structured Block Diffusion Drafting
Bastion uses budget-aware tree-based diffusion drafting with an acceptance surrogate and online latency estimator, achieving up to 6.61× speedup over autoregressive decoding and outperforming state-of-the-art block-diffusion baselines by 39%.
DBLAST: Dependent Block Drafting for Stochastic Speculative Decoding
DBLast shows that independent block sampling degrades accepted length as target sampling entropy increases; its dependent block drafter with acceptance-oriented training improves accepted length, especially in high-entropy decoding.
D^2SD: Accelerating Speculative Decoding with Dual Diffusion Draft Models
D^2SD uses a dual diffusion draft framework with confidence-guided prefix trees and cascade attention, improving over both the underlying diffusion approach and strong autoregressive baselines.
BlockPilot: Instance-Adaptive Policy Learning for Diffusion-based Speculative Decoding
BlockPilot shows that optimal block size varies across samples and predicts it from prefilling representation, achieving an acceptance length of 5.92 and a 4.20× speedup on Qwen3-4B at T=1.