What should a fair evaluation actually measure?
The obvious metric is wall-clock speedup—how many times faster the target model generates tokens with the draft model than without. The papers here report impressive numbers: DFlash achieves over 6x lossless acceleration across a range of models and tasks [3], and DFlare reports average speedups of 5.52x on Qwen3-4B, 5.46x on Qwen3-8B, and 3.91x on GPT-OSS-20B [1]. But speedup alone is misleading because it depends on the hardware, batch size, and the target model's size. A fair evaluation must also report the acceptance rate—the fraction of drafted tokens the target model actually accepts—because that directly determines how many tokens are generated per verification step. For instance, the training interventions in [2] raised accepted draft length by 21–76% per benchmark, which is a more direct measure of draft quality than raw speedup.
Another critical metric is exactness: does the final output follow the same distribution as the target model's autoregressive sampling? All the papers here claim 'lossless' or 'exact' decoding, meaning the acceptance/rejection scheme guarantees the output distribution matches the target model [3][5]. A fair evaluation must verify this property, not just assume it. The optimal transport framework in [5] provides a principled way to design draft selection algorithms that guarantee exactness while maximizing acceptance probability—so a fair evaluation should check that the implementation actually adheres to that contract.
Why draft model quality and overhead are the real battleground
The core challenge in block diffusion speculative decoding is that the draft model must generate an entire block of tokens in one forward pass, but the target model verifies them left-to-right. This mismatch is the central trade-off: a draft model that is too weak produces low acceptance rates, while a draft model that is too heavy adds overhead that eats into the speedup. The papers show that improving draft quality is the key to scaling speedups. DFlash conditions the draft model on context features from the target model, which boosts acceptance rates and enables over 6x speedup [3]. DFlare goes further by giving each draft layer its own learnable combination of target layers, which increases per-layer expressiveness and allows the draft model to be deeper—yielding consistent gains over DFlash (roughly 11% on Qwen3-4B, 8% on Qwen3-8B, and 5% on GPT-OSS-20B) [1].
But a fair evaluation must also measure the overhead of the draft model itself—both the extra forward passes and the memory footprint. The papers emphasize that their methods add 'negligible overhead' [1] or 'without adding additional forward passes' [2], but these claims need to be verified in practice. For example, DDTree builds a draft tree from the draft model's per-position distributions and verifies it in a single target forward pass, which is efficient but requires a more complex attention mask [4]. A fair evaluation should report the total compute cost (draft + verification) and the memory usage, not just the end-to-end latency.
How should training objectives be evaluated?
A fair evaluation must also consider how the draft model is trained, because the training objective directly affects acceptance rate and speedup. The papers reveal a key insight: block diffusion drafters generate tokens bidirectionally, but the target model verifies left-to-right, creating a mismatch between training and inference. [2] empirically shows that three training-time interventions—token positional weighting, a first-error focal loss, and a chain loss—can raise accepted draft length by 21–76% per benchmark across four target models and six benchmarks, without changing the inference pipeline or the exactness contract. This suggests that a fair evaluation should report not only the final speedup but also the training objective and how well it aligns with the verification process.
The papers also disagree on the best way to scale draft capacity. DFlash uses a single fused representation from a few target layers, which limits per-layer expressiveness [3]. DFlare argues that this is a bottleneck and proposes a layer-wise fusion mechanism that injects richer target knowledge into each draft layer, enabling deeper draft models and consistent gains [1]. This is a direct conflict in approach, and a fair evaluation should test both architectures under the same conditions (same target model, same benchmarks, same hardware) to see which actually delivers higher acceptance rates and speedups. The fact that DFlare scales training data from 800K to 2.4M samples to exploit the larger capacity suggests that data scale is another variable that must be controlled in a fair comparison.
About These Sources
This answer is built on 5 studies (1 peer-reviewed, 4 preprints) — published from 2023 to 2026, 4 from 2024 or later — selected as the most relevant from 5 studies that passed quality screening, drawn from 31 papers retrieved from a database of over 500 million.
Sources used in this answer
DFlare: Scaling Up Draft Capacity for Block Diffusion Speculative Decoding
DFlare improves on DFlash by using a layer-wise fusion mechanism that gives each draft layer its own combination of target layers, achieving average wall-clock speedups of 5.52x on Qwen3-4B, 5.46x on Qwen3-8B, and 3.91x on GPT-OSS-20B, which are roughly 11%, 8%, and 5% higher than DFlash, respectively, across six benchmarks.
Teaching Diffusion to Speculate Left-to-Right
Three training-time interventions (token positional weighting, first-error focal loss, and chain loss) raise accepted draft length by 21-76% per benchmark across four target models and six benchmarks, without adding forward passes or changing the inference pipeline or exactness contract.
DFlash: Block Diffusion for Flash Speculative Decoding
DFlash, a block diffusion speculative decoding framework, achieves over 6x lossless acceleration across a range of models and tasks, delivering up to 2.5x higher speedup than the state-of-the-art autoregressive method EAGLE-3.
Accelerating Speculative Decoding with Block Diffusion Draft Trees
DDTree constructs a draft tree from a block diffusion drafter's per-position distributions and verifies it in a single target forward pass using an ancestor-only attention mask, aiming to improve acceptance length over single-trajectory verification.
SpecTr: Fast Speculative Decoding via Optimal Transport
SpecTr provides a principled optimal transport framework for speculative decoding with a set of k candidates per token, achieving a wall-clock speedup of 2.13x, a further 1.37x speedup over standard speculative decoding, while ensuring no quality degradation.