TL;DR

Standard drones are "half-blind," focusing only on what's in front of them. Fly360 changes this by introducing a 360° panoramic perception-to-control pipeline. By decoupling the drone’s heading from its flight direction, it enables safe "strafing" and backward maneuvers in cluttered environments—achieving a near-perfect success rate in scenarios where traditional drones end up in pieces.

Perspective vs. Panoramas: The "Why"

Most autonomous UAVs today suffer from "tunnel vision." They use forward-facing cameras, assuming that the drone always moves in the direction it faces. However, real-world tasks like dynamic filming, inspection, or search-and-rescue often require a drone to track a target while moving laterally or backward.

Current SOTA methods fall short because:

1. Limited FoV: They cannot see obstacles approaching from the side or rear.
2. Modular Cascading: Traditional "Map-Plan-Control" loops are too slow for high-speed dynamic avoidance.
3. Multi-View Fragmentation: Stitching 4-6 separate cameras often leads to "blind seams" and geometric inconsistencies.

Methodology: The Fly360 Architecture

Fly360 treats perception as a unified global problem rather than a collection of directional ones.

1. Two-Stage Pipeline

Instead of end-to-end RGB-to-Action (which suffers from a massive sim-to-real gap), the authors use Depth as a "Universal Translator."

• Perception Stage: Converts panoramic RGB into a 64x128 equirectangular depth map.
• Decision Stage: A lightweight recurrent policy predicts the 3D velocity vector.

2. Spherical Convolutional Recurrent Network

Standard 2D convolutions struggle with the distortion at the poles of a panoramic image. Fly360 utilizes SphereConv, which accounts for the spherical geometry, ensuring the drone perceives a tree at the edge of the image the same way it perceives one in the center.

Figure: The Fly360 pipeline integrates panoramic depth estimation with a recurrent policy network.

3. Fixed Random-Yaw Training

This is the "secret sauce." During simulation training, the drone is assigned a random, fixed yaw. It must learn to reach a goal while avoiding obstacles regardless of which way it is facing. This forces the neural network to develop a true 360° spatial understanding rather than just learning to "move away from pixels in the front."

Experiments: Performance Beyond the Frontal View

The researchers tested Fly360 across three grueling tasks: Hovering Maintenance, Dynamic Target Following, and Fixed-Trajectory Filming.

Hovering in the Storm (Park Environment)

When obstacles were thrown at the drone from all angles, Fly360 maintained its position with minimal collision time. Forward-view models, blocked by their own blind spots, were consistently knocked out of the air.

Table: Success Rate (SR) and Collision Time (CT) comparison. Fly360 dominates in high-speed, high-density obstacle scenarios.

Sim-to-Real: Human Chasing

In real-world tests, the drone was subjected to "Human Chasing"—a scenario where a person unpredictably pursues the UAV. Fly360 demonstrated fluid, omnidirectional evasion, proving that the simulation-trained policy scales perfectly to physical hardware.

Critical Analysis & Conclusion

Fly360 represents a shift from "Directional AI" to "Spatial AI." By embracing the full 360° sphere, it solves a fundamental limitation of aerial robotics.

Limitations:

• The system relies on a pretrained depth model; if the depth estimation fails in extremely low-light or featureless environments (like a white-walled room), the policy may fail.
• Current latency is low (22ms), but further optimization is needed for ultra-low-power edge chips on micro-drones.

Future Work: The integration of temporal multi-frame depth or "Panoramic Optical Flow" could further enhance agility in supersonic flight scenarios.

Fly360 isn't just a better way to fly; it's a new way for machines to see the world.