GenCeption: Can Video Generation Models Become the 'Next-Token Predictors' of Vision?

Executive Summary

In the world of NLP, the "next-token prediction" objective acted as a catalyst, transforming specialized models into the generalist giants we know as LLMs. Computer Vision, however, has remained largely fragmented, relying on task-specific heads for depth, segmentation, and pose. GenCeption fundamentally challenges this status quo by positing that large-scale text-to-video generation is the missing universal pre-training paradigm.

By repurposing a video diffusion model (WAN 2.1) into a single-step feed-forward architecture, GenCeption masters a spectrum of vision tasks—from 3D geometry to expressive segmentation—using a unified head and a single loss function. It doesn't just synthesize pixels; it understands the physical "world model" underlying them.

The Core Motivation: Beyond Specialized Perception

Current vision foundation models like Segment Anything (SAM) or Depth Anything are impressive but remains "narrow." They require architectural modifications for new tasks. The authors argue that a true generalist vision model requires three ingredients:

1. Spatio-Temporal Evolution: Internalizing 4D causality and physics.
2. Vision-Language Alignment: Native instruction-following.
3. Scalability: A paradigm where data and compute lead to emergent intelligence.

Text-to-video generation satisfies all three. Generating a consistent video requires a model to "know" about gravity, object permanence, and 3D structure.

Methodology: Repurposing Diffusion for Speed and Precision

The technical brilliance of GenCeption lies in its transition from Iterative Generation to Feed-Forward Perception.

1. The Feed-Forward Shift

Instead of the slow 50-step denoising process typical of diffusion models, GenCeption uses a single forward pass. By setting the timestep $t=0$ (the noise-free limit in Rectified Flow) and negating the predicted velocity, the model acts as a powerful feature extractor that maps directly from input video to the target modality.

2. Unified Task Representation

GenCeption treats all visual tasks as sequence-to-sequence mapping.

• Dense Tasks: Depth, surface normals, and segmentation are all projected into a 3-channel RGB "ambient space." Even high-dimensional camera poses are mapped to a 3-channel "Rothko" Raymap.
• Sparse Tasks: For coordinates like 3D keypoints, the model appends learnable tokens to the Latent Diffusion Transformer (DiT), which are decoded via a simple MLP.

Figure 1: GenCeption Architecture. The input video and text prompt steer the unified DiT to output the desired modality in a single pass.

Experimental Breakthroughs: SOTA and Efficiency

GenCeption was trained primarily on 7,500 synthetic videos of humans, yet its performance on real-world benchmarks is staggering.

• SOTA Parity: It matches or exceeds specialized models like DepthAnything V3 and Lotus-2 across multiple benchmarks (Sintel, KITTI, ETH3D).
• Data Efficiency: Most impressively, GenCeption achieves SOTA results with 7x to 500x less training data than previous leaders like D4RT. This suggests the generative pre-training provides a "head start" that discriminative methods like VideoMAE lack.

Figure 2: Scaling and Efficiency. The generative backbone (WAN 2.1) significantly outperforms V-JEPA and VideoMAE V2 under identical fine-tuning conditions.

Emergent Behaviors: The World Model at Work

The most exciting aspect of GenCeption is its emergent intelligence. Despite being trained only on synthetic humans:

• Sim-to-Real: It generalizes perfectly to real-world footage.
• Out-of-Distribution (OOD): It accurately predicts depth and segmentation for cats, robots, and anthropomorphic characters—categories it never saw during post-training.
• Complex Instruction Following: It outperforms SAM3 in understanding complex linguistic queries for video segmentation.

Critical Analysis & Conclusion

Takeaways: GenCeption marks a paradigm shift. It proves that we don't need task-specific losses or complex encoders. If a model can generate a physically plausible world, it can perceive it.

Limitations: The authors noted that joint training with sparse tokens (for coordinates) can sometimes degrade dense task performance. This suggests that "pixel-space" remains the most natural domain for diffusion backbones, and forcing them into coordinate regression requires further architectural refinement.

Future Work: As video generation models scale (Sora, Kling, etc.), the perception capabilities "hidden" within them will likely become the foundation for autonomous robotics and general-purpose visual AI. GenCeption is the first robust step into this new era.