Imaging Hidden Objects with Consumer LiDAR: Seeing the Unseen via Motion Induction

TL;DR

Non-Line-of-Sight (NLOS) imaging—the ability to see around corners—has long been the domain of $100klabsetups.Thispaperbreaksthatbarrierbyenabling\ast \ast plug-and-playNLOS\ast \ast onsmartphone-gradeLiDAR($100). By modeling how camera and object motion "sample" the hidden scene, the authors demonstrate real-time 3D tracking and localization using nothing but the faint reflections off a common wall.

The "Loud" Silence: Why Consumer NLOS is Hard

Standard LiDAR builds 3D maps by measuring direct "Line-of-Sight" (LOS) reflections. However, light also bounces off walls to hidden objects and back—these are "third-bounce" signals. In consumer devices, these signals are nearly impossible to use because:

• Weak Signals: Eye-safety limits mean very few photons return from a hidden object.
• Sparse Resolution: Smartphone LiDARs often have only ~100 pixels, compared to millions in research setups.
• Motion Blur: In the real world, both the camera (handheld) and the hidden object are moving, smearing the already faint signal.

Inside the Solution: Motion-Induced Aperture Sampling (MAS)

The core insight is that motion is not a bug; it's a feature. Much like "Synthetic Aperture Radar" (SAR), moving a low-resolution sensor creates a "virtual" larger and denser sensor.

The authors use the Light-Cone Transform (LCT), which simplifies the complex physics of light bouncing into a 3D convolution. They derived the MAS model to prove that an object's motion results in a simple shift of its "canonical" space-time response.

The MAS Model Framework

The model breaks measurements into three components:

1. Object Shape: A static template in space-time.
2. Object Motion: A time-dependent shift of that template.
3. Camera Pose: The spatial sampling pattern determined by where the phone is pointed.

Real-Time Inference: Particle Filtering

To handle the high noise and ambiguity (many hidden positions can explain one noisy measurement), the paper employs Particle Filtering. Instead of guessing one position, it maintains 1,000 "particles" (guesses).

• Propagation: Moves particles based on where the object likely went.
• Evaluation: Compares the real LiDAR data to a "rendered" version of what each particle would see.
• Resampling: Kills off unlikely particles and duplicates successful ones.

Experimental Breakthroughs

The team tested their algorithms on a smartphone-grade sensor and even a cheap ST VL53L8CX sensor ($10-20 part).

1. 3D Tracking

They successfully tracked hidden objects (including human hands) with an average error of just 4.7 cm. This allows a robot to "track" a person walking toward it from around a corner before they are even visible.

2. NLOS Camera Localization

In a "textureless" white room where normal cameras fail to navigate, the system used a hidden object (which the camera couldn't see directly) as a fixed landmark to calculate exactly where the phone was.

Academic Insight & Future Impact

This work represents a shift from hardware-heavy to algorithm-heavy NLOS imaging. By accepting the "Inductive Bias" that we often know the general shape of objects or how they move, we can trade off raw sensor power for smart multi-frame fusion.

Limitations: The model currently assumes rigid-body motion. If an object deforms significantly (like a person's limbs moving), the "canonical template" becomes less accurate. However, the introduction of a "Learnable Score Function" or neural transients could solve this in the near future.

Conclusion

This paper effectively "democratizes" a superpower. What was once the realm of high-end physics labs is now a software update away for robots and mobile devices. The era of Plug-and-Play NLOS has arrived.

For more details, visit the project page: sidsoma.com/consumer-nlos/