[AMD 2026] DC-DiT: Beyond Static Patches — Learning to See What Matters in Diffusion

TL;DR

Diffusion Transformers (DiTs) have traditionally been "blind" to image content complexity, treating a clear blue sky and a complex textured fabric with the same computational weight. Dynamic Chunking Diffusion Transformer (DC-DiT) breaks this paradigm. By wrapping a DiT backbone in a learned encoder-router-decoder scaffold, the model learns to "chunk" tokens dynamically. The result? It spends more compute on intricate details and late-stage denoising, while breezing through backgrounds and noisy early steps, leading to better image quality (FID) with significantly higher efficiency.

The "Uniformity" Tax in Diffusion

The standard DiT architecture operates on a fixed grid of patches (e.g., 2x2 or 4x4). This creates two major inefficiencies:

1. Spatial Redundancy: A 256x256 image of a bird against a bokeh background has most of its "information" concentrated in the bird. Static patching treats every pixel equally.
2. Temporal Redundancy: In the early stages of diffusion, the image is mostly noise. High-resolution processing at $t=999$ is often overkill for capturing global structure.

DC-DiT’s research intuition is simple: Tokens should be where the information is.

Methodology: The Encoder-Router-Decoder Scaffold

DC-DiT doesn't just prune tokens; it learns an adaptive representation.

1. The Dynamic Routing

The model uses an isotropic encoder (convolutional residual blocks) to aggregate local context. A Router module then predicts a boundary probability $p_i$ for each token.

• High Similarity: Neighboring tokens are merged (compressed).
• Low Similarity: Tokens are marked as "boundaries" and retained.

2. Differentiable De-chunking

To overcome the non-differentiable nature of discrete token selection, DC-DiT employs Spatial Smoothing. It uses a confidence-weighted Gaussian kernel to blend boundary representations, ensuring that gradients can flow back to the router even when discrete "keep/drop" decisions shift.

Figure 1: The DC-DiT pipeline showing the flow from uncompressed input through the router-compressed DiT blocks and back via de-chunking.

Emergent Behaviors: Seeing Without Labels

One of the most striking results is that the model learns visual segmentation without any masks or labels. As shown in Figure 2, the boundary tokens (bright spots) naturally cluster around the subject of the image (e.g., the dog's face or the bird's feathers), while the background is represented by a handful of "chunked" tokens.

Furthermore, the model automatically learns a Temporal Schedule. It discovers that it can use high compression (fewer tokens) at $t\approx 1000$ and must increase token density as it approaches $t=0$ to refine fine details.

Figure 2: The router's boundary predictions. Notice how the model "focuses" its budget on high-entropy regions like textures and edges.

Experiments and Upcycling

The authors evaluated DC-DiT on ImageNet 256x256. The highlights include:

• SOTA Scalability: At XL scale with 16x compression, DC-DiT achieved an FID of 13.60, crushing the FLOP-matched static baseline's 16.35.
• Upcycling: You don't need to train from scratch. By taking a pretrained DiT and adding the scaffold with a brief "activation distillation" phase, they achieved better-than-baseline results in just 10-25% of the original training time.

Table 1: DC-DiT consistently outperforms "Isoflop" (FLOP-matched) baselines, signifying a more efficient use of the compute budget.

Critical Insight & Future Outlook

DC-DiT proves that adaptive compute is the next frontier for diffusion. While this paper focuses on ImageNet, the implications for Video Generation are massive—where temporal redundancy is even higher than spatial redundancy.

Limitations: The de-chunking mechanism (spatial smoothing) adds a slight overhead, and the optimal ratio loss weight $\lambda$ requires tuning. However, its composability with other methods like DyDiT makes it a highly practical tool for real-world deployment on hardware like the AMD Instinct GPUs used in the study.

The Takeaway: Stop wasting TFLOPs on background noise. Let the model decide where the detail lives.