Imaging Hidden Objects with Consumer LiDAR: From Lab Curio to Smartphone Reality

TL;DR

Researchers from MIT and Dartmouth have developed a way to see "around corners" using the cheap, low-resolution LiDAR sensors found in modern smartphones. By treating handheld motion as a benefit rather than a bug, their Motion-Induced Aperture Sampling (MAS) model uses multi-frame fusion and particle filtering to track hidden objects and localize cameras with sub-10cm accuracy.

The Problem: The "Hardware Gap" in Hidden Vision

Non-Line-of-Sight (NLOS) imaging has long been a "holy grail" of computational photography. The idea is simple: use a wall as a "virtual mirror" to catch faint, multi-bounce reflections from hidden objects.

However, until now, this required research-grade LiDAR costing tens of thousands of dollars. Consumer LiDARs (like those in iPhones or robots) were considered "unfit" for NLOS due to:

• Weak Signals: Laser power is capped for eye safety.
• Low Resolution: Sensors often have as few as 64-100 pixels.
• Motion Blur: Handheld cameras and moving targets destroy the precise timing needed for reconstruction.

The Insight: Motion as a Sampling Tool

The authors suggest a paradigm shift: instead of fighting motion, use it. Inspired by Synthetic Aperture Radar (SAR), they realize that a moving handheld LiDAR effectively creates a "larger" virtual sensor over time.

The MAS Model

The technical heart of the paper is the Motion-Induced Aperture Sampling (MAS) model. It leverages the Light-Cone Transform (LCT), which transforms time-of-flight measurements into a space where the relationship between an object's shape and its reflection is a simple 3D convolution.

Figure: The MAS model decouples Object Shape (canonical response), Object Motion (shift in space-time), and Camera Pose (sampling points).

By pre-computing a "canonical" reflection for a known object (like a mannequin or a hand), the system can render what the LiDAR should see at any given position almost instantly.

Methodology: Particle Filtering for Real-Time Tracking

To handle the extreme noise of consumer sensors, the researchers implemented a Particle Filter.

1. Propagation: 1,000 "particles" (guesses of where the hidden object is) are moved based on a motion prior.
2. Evaluation: Each guess is compared against the actual noisy LiDAR frame using a dot-product score.
3. Resampling: Guesses that don't match the data are killed off; successful ones are cloned.

This allows the system to maintain a "probability cloud" of the hidden object's location, making it robust against transient noise and low resolution.

Key Results & Applications

The team demonstrated three breakthrough capabilities on a $100 STMicroelectronics sensor:

1. 3D Tracking around Corners

They successfully tracked hidden objects with a 4.7 cm average error. They even demonstrated NLOS Hand Tracking, showing that the system could distinguish between left and right hands even when they were completely hidden from view.

2. Camera Localization (NLOS-VO)

In a "white room" scenario where walls are featureless and standard cameras fail to navigate, the system used a hidden object (a "landmark") to determine the camera's position. By looking at the reflection of a hidden object, the robot knows where it is relative to the "virtual mirror."

Figure: Multi-object tracking results showing predicted distributions (blue/green) vs. ground truth.

3. Diffuse Object Imaging

While most NLOS work uses shiny retroreflective tape, this paper shows that the MAS model is robust enough to handle diffuse (matte) objects, though the SNR is predictably lower.

Critical Analysis & Conclusion

This work marks a transition from Physical Sophistication to Algorithmic Intelligence.

Limitations:

• Known Shapes: Currently, tracking works best when the object's shape is known beforehand. "Blind" NLOS tracking of unknown objects remains a challenge.
• Occlusions: Complex scenes with multiple occluding hidden objects aren't yet modeled.

Future Outlook: The authors envision a "plug-and-play" NLOS future. As LiDAR becomes standard in wearables and IoT, your Roomba could "see" a child running around a corner before they enter the room, or AR glasses could track your body pose by looking at the floor's reflections. The democratization of NLOS has officially begun.