[CVPR 2025] FaceCam: Solving Scale Ambiguity in Portrait Video Camera Control

TL;DR

FaceCam is a new framework designed for "re-capturing" portrait videos—taking an existing video and rendering it through a new, user-defined camera path. By ditching traditional 3D camera coordinates in favor of Scale-Aware Facial Landmarks, it eliminates the common "drifting" and "distortion" artifacts found in previous SOTA models.

Background: The Scale Ambiguity Trap

In the world of 3D computer vision, monocular videos (shot with a single lens) are inherently ambiguous. Because we don't know the exact distance of the subject from the lens, we cannot determine the "true" metric scale of the scene.

Current models usually use Camera Extrinsics ( $[R|t]$ matrices) to tell the AI where to move. However, a "5cm move" looks massive if the face is close to the camera but tiny if the person is far away. This mismatch leads to poor control and visual "hallucinations" where the face warps or shifts unnaturally.

Methodology: Why Landmarks Over Matrices?

The core insight of FaceCam is that image-space correspondences are sufficient to define camera motion. Instead of feeding the model abstract numbers, the authors use rendered facial landmarks from a proxy 3D head as the conditioning signal.

1. Scale-Aware Conditioning

By using 2D projections of 3D landmarks, the representation becomes naturally scale-invariant. If you scale the 3D scene and the camera translation by the same factor, the 2D landmarks remain identical. This forces the Diffusion model to focus on the relative perspective change rather than guessing the absolute scene scale.

Figure: The FaceCam pipeline. (A) Training uses a VAE-based Diffusion Transformer (Wan) conditioned on source video and target landmark maps. (B) Inference uses a proxy 3D head to generate landmarks for any desired trajectory.

2. Bridging Static Data to Dynamic Motion

Training a dynamic camera model is hard because high-quality datasets like NeRSemble only have static cameras. The authors solved this with two clever tricks:

• Multi-shot Stitching: Randomly cutting and joining clips from different static angles to teach the model how to handle perspective shifts.
• Synthetic Motion: Artificially zooming and panning on in-the-wild videos to improve generalization outside the studio.

Performance & Results

FaceCam was tested against heavyweights like TrajectoryCrafter and ReCamMaster.

• Identity Preservation: FaceCam achieved an ArcFace score of 0.85, significantly higher than TrajectoryCrafter (0.52), which often suffered from "melted" facial features due to 3D reconstruction errors.
• Robustness: The model maintains consistency even with challenging props like microphones, glasses, and complex hairstyles (see Figure below).

Figure: Comparison of Arc Left, Pan Right, and Zoom In motions. FaceCam (bottom row) maintains higher texture detail and geometric accuracy compared to baselines.

Critical Insight: The "Proxy Head" Magic

A fascinating finding in this paper is the Identity Disentanglement. During inference, you can use any 3D head (even a generic one) to generate the landmark trajectory. The model is smart enough to realize that the landmarks define the camera, while the source video latent defines the person. This makes the system incredibly flexible for creators who want to "draw" a camera path without a complex 3D setup.

Conclusion & Limitations

FaceCam sets a new bar for controllable portrait generation. By leaning into the specific geometry of the human face, it bypasses the "black box" issues of general-purpose video models.

Limitations:

• Occlusion: It cannot yet handle cases where the camera rotates to the back of the head (as landmarks disappear).
• Speed: Built on top of the Wan-Video backbone, inference is still relatively slow for real-time use.

Future iterations using more general "pixel-correspondence" trackers (like OmniGlue) could potentially expand this high-precision control to general scenes beyond human portraits.