[CVPR 2026] ELT: Elastic Looped Transformers — Slashing Generative Parameters by 4x via Recurrent Refinement

TL;DR

Elastic Looped Transformers (ELT) redefine how we build visual generative models. Instead of stacking hundreds of unique layers, ELT reuses a small set of high-quality transformer blocks iteratively. By using a new training trick called Intra-Loop Self Distillation (ILSD), a single ELT model can scale its quality and speed on-the-fly, achieving SOTA results (FID 2.0) with 75% fewer parameters and up to 3.5x higher throughput.

The Problem: The "Memory Wall" and Massive Models

The current "SOTA" in image and video generation (like DiT or MaskGIT) is synonymous with "Bigger is Better." To increase capacity, we stack more layers. This approach hits two walls:

1. Memory Wall: Loading billions of unique weights from VRAM to the processor is a massive bottleneck.
2. Inflexibility: Once a model is trained at a specific depth, you can't easily "shorten" it for a mobile device without a massive drop in quality.

Looped/Recurrent transformers are the logical solution—reuse weights to save memory—but historically, they have been "black boxes." Intermediate loops often produce garbage until the very last iteration, making them useless for elastic, real-time scaling.

Methodology: Depth through Recurrence & ILSD

The core "Aha!" moment of ELT is treating the recurrent loop not just as a deep stack, but as a progressive refiner.

1. Weight-Shared Looping

Instead of $D$ unique layers, ELT uses a block $g_{\Theta }$ of $N$ layers applied $L$ times. $F_{(N,L)}(\mathbf{x})=\underset{Lextloops}{\underbrace{g_{\Theta }(g_{\Theta }(\cdots g_{\Theta }(\mathbf{x})))}}$ This decouples the physical model size (determined by $N$) from the computational depth (determined by $L$).

2. Intra-Loop Self Distillation (ILSD)

ELT solves the "unstable intermediate state" problem by forcing the model to be useful at every loop. During training, it uses a Dual-Path system:

• Teacher Path: Runs the full $L_{max}$ loops to get the best possible prediction.
• Student Path: Exits early at a random loop $L_{int}$.
• The Loss: The student is trained to predict the ground truth and mimic the teacher’s mature representation.

Figure 1: The ELT Training Framework utilizing shared parameters and ILSD to enforce meaningful intermediate states.

Experiments: Superior Efficiency

The results are striking. ELT doesn't just "save parameters"—it often outperforms the massive baselines it replaces under similar compute budgets.

Performance Benchmarks

In class-conditional ImageNet 256x256 generation:

• MaskGIT-XL (Baseline): 446M params $\to$ FID 2.0.
• ELT-XL (8 layers x 4 loops): 111M params $\to$ FID 2.0.

By keeping the "working set" of weights small, the model can stay mostly on-chip, leading to a 3.5x throughput increase on hardware like TPUs because it avoids constant memory swapping.

Figure 2: Latent trajectories of Standard vs. Elastic Looped Transformers. Standard models only "converge" at the end, while ELT refines the image progressively.

Any-Time Inference: The Real-World Superpower

Because ILSD makes every loop count, ELT enables Any-Time Inference. A single model weights file can be deployed across:

• Mobile/Edge: Run 2 loops for instant, decent-quality previews.
• Cloud/Studio: Run 8 loops for high-fidelity, production-grade assets.

As shown in the paper's Pareto front analysis, you can traverse the quality/compute curve at runtime without ever retraining.

Figure 3: Quality (FID) vs. Compute (GFLOPs). ELT stays on the optimal Pareto front, allowing dynamic scaling.

Depth Insight: Scaling Laws for Looping

The paper reveals a crucial nuance: you cannot loop a model too much if the base block is too small. A $1imes32$ configuration (one layer looped 32 times) fails because a single layer lacks the representational capacity. The authors found that a "sweet spot" typically involves a block of 8-16 layers iterated 2-4 times.

Conclusion & Future Work

ELT is a paradigm shift for efficient generative AI. It proves that weight-tying is not a compromise—it’s an optimization.

Key Takeaways for Practitioners:

1. Parameter Efficiency: You can reduce your model footprint by 75% without losing generation quality.
2. Throughput: Recurrence is a hardware-friendly pattern that minimizes memory latency.
3. Elasticity: One model, infinite deployment tiers.

Future applications may include combining ELT with One-Step consistency models, where the number of loops becomes the only lever for users to dial in their desired "render quality" vs. "speed" in real-time.