Weaver: Breaking the Independence Ceiling in Speculative Decoding

TL;DR

Researchers have introduced DFlash-TfM (Weaver), a new speculative decoding method that combines the speed of parallel "factorized" drafting with the accuracy of "autoregressive" dependency tracking. By solving the bottleneck of independence assumptions and introducing a novel verification kernel for Gated Delta Nets (GDN), they achieved a 4.37x speedup on Qwen3.6-27B models.

The "Independence Ceiling" in Modern Drafting

Speculative decoding traditionally uses a small "draft" model to guess tokens, which a large "verifier" model then checks in parallel. Recently, factorized drafters (like Medusa or DFlash) have become popular because they predict multiple future tokens in a single forward pass.

However, they have a fatal flaw: they assume that future tokens are independent of each other given the current prefix. In reality, language is highly dependent—the word at position $N+2$ depends heavily on the word chosen at $N+1$. As the draft length increases, the "marginal" guesses of a factorized drafter diverge from the true distribution, causing acceptance rates to plummet.

Methodology: Trees from Marginals (TfM)

The paper introduces Weaver, a lightweight autoregressive adapter (56.7M parameters) that acts as a bridge.

1. Top-K Prior: It takes the top-512 most likely tokens for each future position from the "fast but independent" DFlash model.
2. Conditional Refinement: Weaver then performs a tiny autoregressive pass only on these 512 candidates. Because it doesn't look at the entire 150k+ token vocabulary, it avoids the massive memory bandwidth cost usually associated with autoregressive generation.
3. Dynamic Tree Construction: Using a modified best-first search, it builds a "proposal tree" where each branch represents a highly probable sequence of tokens.

Figure 1: Overview of the DFlash-TfM drafting procedure, combining parallel marginals with autoregressive tree construction.

Systems Innovation: Fused GDN Verification

For models like the Qwen3.6 series, which use Gated Delta Net (GDN) layers, typical tree-verification (using attention masks) doesn't work because GDN is recurrent.

The authors derived a rollback-free tree-verification algorithm. Instead of updating the recurrent state and potentially having to "undo" it if a token is rejected, they treat the verification as a masked triangular solve. This allows the verifier to score the entire proposal tree in one fused GPU kernel pass, making it 7.1x faster than branch-by-branch verification at a tree size of 128 tokens.

Results & Performance

The performance gains are stark. While factorized drafters like DFlash saturate quickly, Weaver's acceptance rate stays high even for deep trees.

• Speed: 24.7% faster than the best DFlash baseline.
• Efficiency: Mean Acceptance Length (MAL) increased by 77%.
• Scaling: Unlike marginal or argmax-marginal drafters, Weaver surpasses the theoretical "marginal-oracle" ceiling by conditioning on actual drafted tokens.

Figure 2: Acceptance probability comparison showing DFlash-TfM surpassing the marginal-oracle ceiling at long draft depths.

Conclusion

DFlash-TfM (Weaver) proves that you don't need a huge autoregressive model to achieve high-quality speculation. By cleverly constraining the vocabulary to the top marginals and optimizing the underlying CUDA kernels for recurrent architectures, the authors have pushed the boundaries of LLM interactivity, making 27B+ models significantly more responsive for real-world applications.