MIGM-Shortcut: Bypassing the Heavy Lift of Masked Image Generation
Accelerating Masked Image Generation by Learning Latent Controlled Dynamics
The paper introduces MIGM-Shortcut, a lightweight acceleration framework for Masked Image Generation Models (MIGMs). By learning latent controlled dynamics through a small neural network, it achieves over 4x speedup on SOTA models like Lumina-DiMOO with negligible quality loss.
TL;DR
Masked Image Generation Models (MIGMs) are powerful but slow. MIGM-Shortcut accelerates these models by learning the "flight path" of internal features. By replacing the heavy base model with a 37x smaller "shortcut" network for most steps, it achieves a 4x to 5.8x speedup on text-to-image tasks without sacrificing the visual fidelity or semantic alignment of the original SOTA models.
Background: The Hidden Smoothness in Chaos
While it seems that predicting discrete tokens step-by-step is a jumpy, stochastic process, the authors found a surprising truth: the last hidden states of MIGMs evolve along an extremely smooth trajectory.
Figure 1: Heatmap and t-SNE visualization showing that features in consecutive steps share >95% cosine similarity.
However, current acceleration methods for continuous diffusion (like TaylorSeer) fail here. Why? Because MIGMs are controlled. Every time you sample a discrete token, you inject a choice that "forks" the future. If you only look at past features, you can't predict where the fork goes.
Methodology: The Latent Controlled Dynamics
The core insight of MIGM-Shortcut is treating the generation as a State-Space Model.
- State: The continuous feature .
- Control: The newly sampled discrete tokens .
Instead of running a multi-billion parameter Transformer to get , the Shortcut model (a tiny cross-attention block) takes and the new tokens to calculate the "velocity" or the delta .
The Architecture
The shortcut model is intentionally lightweight (e.g., 220M params vs 8B for Lumina-DiMOO). It uses a bottleneck design and adaptive layer normalization to ingest time embeddings, ensuring it knows exactly which stage of the "unmasking" process it is currently in.
Figure 2: The inference loop alternates between a "Full Step" (heavy base model) to recalibrate and multiple "Shortcut Steps" (light model) to accelerate.
Experiments: Breaking the Pareto Frontier
The authors tested MIGM-Shortcut on MaskGIT and Lumina-DiMOO.
1. Superior Quality-Speed Tradeoff
In text-to-image benchmarks, the method decimated training-free baselines like ML-Cache and TaylorSeer. While other methods saw their ImageReward scores tank as they pushed for 4x speedup, MIGM-Shortcut stayed nearly flat.
Figure 3: Pareto curve showing MIGM-Shortcut (red stars) dominating all existing acceleration methods.
2. Solving the Multi-Modality Problem
Single-step models (like Di[M]O) often suffer from artifacts or duplication because they can't model the joint distribution of all tokens at once. MIGM-Shortcut avoids this by maintaining the iterative refinement process—it just makes the iterations 30x cheaper to compute.
Critical Analysis & Conclusion
Why does it work? The "heavy" base model is essentially overthinking. In the later stages of generation, most of the work is just "refining" existing context rather than "discovering" new objects. The Shortcut model captures this refinement logic efficiently.
Limitations:
- It requires a small amount of training data (though the authors show 5-12 hours on 4 GPUs is enough).
- It still requires a "budget" of full steps to prevent error accumulation.
Takeaway: MIGM-Shortcut proves that the future of efficient AI isn't just about compression or pruning—it's about architectural offloading. By allowing a small model to handle the predictable "smooth" parts of a trajectory, we unlock massive throughput for large-scale generative systems.