[CVPR 2026] V-Co: Mastering Visual Co-Denoising for High-Fidelity Pixel Synthesis

TL;DR

Pixel-space diffusion is making a massive comeback, but it has always struggled with "semantic efficiency." V-Co provides a rigorous, 4-ingredient recipe for visual co-denoising—jointly generating pixels and pretrained DINOv2 features. The result? A mid-sized model (918M params) that crushes a 2B parameter baseline, setting a new benchmark for pixel-space ImageNet generation with an FID of 1.71.

Background: The Pixel-Space Re-emergence

While Latent Diffusion Models (LDMs) dominate the industry, they are inherently limited by the biases and artifacts of their pretrained VAEs. Pixel-space models like JiT (Just in Time) denoise directly in image space, offering a "cleaner" path to generation. However, learning high-level structure from raw pixels is hard.

Recent works tried "Representation Alignment" (e.g., REPA), essentially telling the model: "Make your internal features look like DINOv2." V-Co takes this further: "Don't just look like DINOv2; denoise DINOv2 features alongside the pixels."

The "Why": Solving the Alignment Bottleneck

Prior co-denoising attempts were often "black boxes" of complex architecture and heuristic guidance. The authors of V-Co identify that for co-denoising to work, you must solve three critical interaction problems:

1. Architecture: How do pixels and semantic tokens "talk"?
2. Guidance: How do you perform Classifier-Free Guidance (CFG) when you have two separate signal streams?
3. Calibration: How do you balance a pixel stream (low-level) with a semantic stream (high-level) that has completely different signal magnitudes?

Methodology: The V-Co Recipe

The authors systematically ablated every design choice to find the "Global Optimum" for co-denoising:

1. Dual-Stream Architecture

Instead of merging pixels and semantic features into a single backbone (Single-Stream), V-Co uses a Fully Dual-Stream design. Both streams have their own LayerNorms, MLPs, and Attention projections but interact through Joint Self-Attention. This allows the model to preserve the unique characteristics of semantic features while allowing pixels to "borrow" structural cues.

Figure 1: The V-Co recipe: Dual-stream interaction combined with structural masking and hybrid loss.

2. Structural Masking for CFG

In standard CFG, we drop the class label. In V-Co, we have class labels and DINO features. The breakthrough insight here is Semantic-to-Pixel Masking. To generate an "unconditional" image, the model doesn't just drop the DINO input; it explicitly masks out the attention weights from the semantic stream to the pixel stream. This "structural" drop is far more stable than input-level dropout.

3. Perceptual-Drifting Hybrid Loss

Simple MSE on the DINO features isn't enough. V-Co introduces a hybrid loss:

• Perceptual Field: Pulls the prediction toward the ground truth (Accuracy).
• Drifting Field: Pushes the prediction away from other generated samples of the same class (Diversity/Anti-Mode Collapse). A similarity-based gate ($s_i$) adaptively balances these two, ensuring the model focuses on diversity when it's lost and accuracy when it's close to the target.

4. SNR Calibration via RMS Scaling

Pixels and DINO features have different scales. If you simply add the same noise to both, one becomes "harder" to denoise than the other, leading to imbalanced training. V-Co uses RMS-based feature rescaling, which the authors prove is mathematically equivalent to "time-shifting" the diffusion schedule to match the Signal-to-Noise Ratio (SNR) of both streams.

Experimental Triumphs

The results on ImageNet-256 are striking. V-Co scales remarkably well:

• Efficiency: V-Co-B/16 reaches better FID than the JiT-B baseline in 1/3 of the training time.
• SOTA: V-Co-H/16 (1.9B params) achieves an FID of 1.71, outperforming the massive 2B parameter JiT-G and even closing the gap with some Latent Diffusion methods that use heavy VAE tricks.

Note: Notice how V-Co-L outperforms JiT-G despite having less than half the parameters.

Critical Insight & Conclusion

V-Co proves that semantic supervision is most effective when it is dynamic. By allowing the model to "generate" the semantic features as it generates the pixels, we create a feedback loop where high-level structure informs low-level texture in a way that static alignment losses (like REPA) cannot mimic.

The transition from "Alignment" to "Co-Denoising" represents a shift toward more unified multimodal generative models. While currently tested on ImageNet, the V-Co recipe provides a blueprint for any task where high-level knowledge (Text, Depth, Audio) needs to guide pixel synthesis.

Future Outlook: The next step for V-Co is clearly Text-to-Image scaling. If co-denoising works for class labels, doing it for CLIP/T5 embeddings could finally allow pixel-space models to challenge the dominance of Stable Diffusion and Midjourney.