TL;DR

Estimating "Metric Depth" (actual physical distance) has long been a challenge for AI. While monocular models struggle with scale ambiguity, Depth from Defocus (DfD) offers a physical solution by analyzing how blur changes across a focus stack. This paper introduces FOSSA, a Transformer-based model that achieves state-of-the-art zero-shot performance, and ZEDD, a high-fidelity real-world benchmark that finally provides the data density needed to advance this field.

The "Scale Ambiguity" Wall and the DfD Insight

In monocular depth estimation, a model might correctly identify that a chair is in front of a table, but it often struggles to tell if the chair is 2 meters or 2.5 meters away—this is scale ambiguity.

Depth from Defocus (DfD) circumvents this by using the laws of optics. When a camera's focal plane moves, objects at different depths go in and out of focus. By analyzing a "focus stack" (a series of images at different focus distances), we can mathematically derive the exact metric depth. However, previous DfD models were "brittle"—they worked on specific lab datasets but failed in the real world.

Methodology: The FOSSA Architecture

The core innovation of FOSSA (FOcuS Stack Attention) is how it treats the focus stack. Instead of processing images in isolation, it treats the stack as a temporal-like sequence.

1. Shared ViT Encoder: Each image in the stack passes through a Vision Transformer (ViT) to extract features.
2. Stack Attention Layer: This is the "secret sauce." Between ViT blocks, the model performs attention across the images in the stack at each pixel location.
3. Focus Distance Embedding: Much like positional encoding in NLP, FOSSA injects the physical focus distance ($d$) into the features, allowing the model to correlate visual blur with specific metric distances.

The FOSSA pipeline: Feature extraction with cross-stack attention followed by global feature refinement.

Bridging the Gap: Synthetic Training & ZEDD Benchmark

Since there are no massive datasets of real-world focus stacks, the authors built a sophisticated Data Pipeline. They took existing RGBD datasets and "simulated" focus stacks by applying a Generalized Point Spread Function (PSF). By randomizing the lens aperture ($N$) and the blur shape, they forced the model to learn robust optical features rather than overfitting to one specific lens.

To prove this works, they released ZEDD (ZEro-shot Depth from Defocus):

• 100 real-world scenes (8.3x more than previous benchmarks).
• 4K Resolution DSLR imagery.
• High-end Lidar Ground Truth with sub-centimeter accuracy.

Experimental Performance

FOSSA's zero-shot capability is its most impressive feat. Without seeing a single image from the ZEDD dataset during training, it outperformed models specifically designed for those environments.

• ZEDD Benchmark: FOSSA reduced the Absolute Relative error by 55.7% compared to DepthPro (a top-tier monocular baseline).
• Traditional DfD: On the classic DDFF dataset, FOSSA (after fine-tuning) crushed previous SOTA models, reducing MSE by 40.4%.

Qualitative comparison: FOSSA (far right) recovers sharp, metrically accurate boundaries compared to competing methods.

Deep Insight & Conclusion

The success of FOSSA reveals a major trend in computer vision: Foundational models benefit from physical constraints. By wrapping a powerful, pre-trained ViT (DepthAnything v2) in an architecture that understands optical blur, the authors created a tool that is both "smart" (semantic understanding) and "precise" (optical measurement).

Future Outlook: While FOSSA currently targets static scenes, the logical next step is Dynamic DfD—using these same focus cues to estimate depth from handheld video or moving subjects, potentially replacing expensive LiDAR sensors in consumer devices.