[CVPR 2025] HumanOrbit: Transforming Single Images into 3D Avatars via 360° Orbit Generation

TL;DR

HumanOrbit redefines 3D human reconstruction by shifting the paradigm from static multi-view synthesis to continuous video generation. By finetuning a large-scale video diffusion model (Wan 2.1) to perform a 360° camera orbit around an input subject, the researchers have achieved unprecedented view consistency and identity fidelity. The result is a robust, pose-free pipeline that turns a single photo into a high-quality textured mesh.

Perspective: From Images to Orbits

The field of 3D human reconstruction has long been haunted by the "consistency ghost"—the phenomenon where a person's face or clothing pattern changes as you rotate the camera. Previous SOTA methods like Zero-1-to-3 attempted to solve this using image-to-image diffusion, but they often lacked the temporal smoothness required for a truly coherent 3D structure.

HumanOrbit's Core Insight: A video diffusion model trained on billions of real-world frames already understands how objects should look from different angles. By constraining this model to generate an "orbit" rather than random motion, we can extract dense, consistent multi-view data that is far superior to independent image generations.

Methodology: The "Video-to-3D" Pipeline

1. Omni-View Generation with DiT

Instead of building a model from scratch, the authors leveraged Wan 2.1, a Diffusion Transformer (DiT) video model. They applied LoRA (Low-Rank Adaptation) to finetune it on a modest dataset of 3,000 sequences derived from 500 3D human scans.

The model takes a single image, pads it temporally, and denoises a sequence of 81 frames that represent a full 360° rotation. Crucially, this process is pose-free—it doesn't need to know the person's skeleton or the starting camera angle.

2. SfM and Mesh Carving

Once the orbit video is generated, the pipeline treats it like a real-world drone capture:

• Camera Estimation: It uses VGGT (Visual Geometry Grounded Transformer) to estimate camera parameters and an initial point cloud.
• Mesh Optimization: Using differentiable rendering, the system performs "mesh carving," optimizing a mesh to match the generated RGB frames and predicted normal maps.

Experiments: Superior Fidelity

The results are striking. In head-to-head comparisons with PSHuman and SV3D, HumanOrbit maintains the integrity of complex clothing patterns (like horizontal stripes) that usually become a "blurry soup" in other models.

Quantitative Edge

On the Clothing Co-Parsing (CCP) dataset, HumanOrbit achieved:

• MVReward: 0.8035 (vs. PSHuman's 0.6814)
• CLIP Score: 0.8317 (suggesting the highest fidelity to the original image identity).

As shown in the figure above, while competitors like SV3D distort the face or smudge textures, HumanOrbit generates crisp, realistic novel views.

Critical Analysis & Future Outlook

The Success Factors:

• LoRA Efficiency: The ability to achieve such high quality with only 500 scans proves that the "general knowledge" inside video diffusion models is doing most of the heavy lifting.
• Generalizability: Unlike many human-specific models, it handles partial bodies (headshots) and even exhibits zero-shot potential for non-human objects like dogs or chairs.

The Trade-offs:

• Inference Speed: At 17 minutes per orbit, this isn't yet ready for real-time mobile apps.
• Fixed Elevation: The current model orbits at a horizontal level, meaning the "top of the head" or "under the chin" might still suffer from lack of data.

Conclusion

HumanOrbit marks a significant step toward scalable 3D digital twin creation. By proving that video priors > image priors for 3D tasks, it opens the door for future models that could generate complex 4D (moving) avatars from a single snapshot.

Article by Senior Academic Tech Editor. Source: "HumanOrbit: 3D Human Reconstruction as 360° Orbit Generation".