HDFlow: Hybrid Generative Planning—The "Best of Both Worlds" for Long-Horizon Robotics

TL;DR

Robotic assembly of a table isn't just one task; it’s a strategic sequence of subgoals followed by millisecond-level precision control. HDFlow (Hierarchical Diffusion-Flow) is a new framework that treats these two levels differently: it uses Diffusion for high-level "strategic thinking" and Rectified Flow for low-level "reflexive execution." The result? A system that is more accurate, faster, and more robust in the real world than previous single-model approaches.

The "Generative" Bottleneck in Robotics

Generative models, particularly Diffusion, have changed how we think about robot planning. By treating a trajectory as a sample from a distribution, robots can "imagine" paths to a goal. However, Diffusion has a fatal flaw for robotics: the Iterative Denoising Process.

Running 100 steps of denoising for every low-level motor command is computationally impossible for real-time interaction. Conversely, simpler models often fail to capture the multi-modal distribution of "strategies" needed for complex tasks like furniture assembly.

Authors Nandiraju et al. identified a core insight: High-level planning needs exploration (Diffusion's strength), whereas low-level execution needs speed and determinism (Rectified Flow's strength).

Methodology: The Architecture of Hierarchy

HDFlow operates in two distinct stages:

1. The Structured Latent Workspace

Before planning begins, the agent learns a Contrastive World Model. Using a DINOv2 visual backbone, it maps high-dimensional camera feeds into a latent space $Z$ where states progressing toward the goal are pulled closer together. This ensures the "map" the robot uses for planning is semantically organized.

2. The Hybrid Planner

The framework splits the labor:

• High-Level (Diffusion + EBM): Generates a sequence of $K$ subgoals. To ensure these subgoals aren't just "plausible" but actually "successful," a manifold-aware Energy-Based Model (EBM) guides the process.
• Low-Level (Rectified Flow): Takes the next subgoal and generates a dense trajectory of actions. Unlike Diffusion, Rectified Flow uses an ODE solver that can produce results in significantly fewer steps.

Figure 1: The dual-stage pipeline showing the World Model (left) and the HDFlow High/Low-level planning split (right).

Solving the "Manifold Deviation" Problem

A common problem when guiding Diffusion models with external signals (like an EBM) is that the guidance pushes the sample into "impossible" regions of latent space—mathematically known as moving off the data manifold.

HDFlow introduces Manifold Projection. After the EBM guidance shifts the sample toward success, the system projects it back onto a local approximation of the "feasible" latent space using K-Nearest Neighbors and PCA. This keeps the robot's "imagination" grounded in physical reality.

Figure 2: Visualizing the guidance step. Red arrows show EBM push, purple/green shows the projection back to the feasible manifold $M$.

Experimental Results: Dominating the Benchmarks

HDFlow was tested against heavyweights like Diffusion Policy and Simple Hierarchical Diffuser (SHD) across three major benchmarks: FurnitureBench, RLBench, and OGBench.

• Performance: In the "lamp" assembly task (Medium randomness), HDFlow achieved a 49% success rate, while the closest hierarchical baseline (SHD) only hit 22%.
• Efficiency: Compared to using Diffusion at all levels (HD), HDFlow reduced planning time from 142ms to 88ms, a critical leap for real-world deployment.
• Real-World Robustness: The system was successfully deployed on a Franka Research 3 arm, managing complex assembly despite the noisy observations of the physical world.

Figure 3: Side-by-side comparison of successful task completion in simulation and the real-world setup.

Critical Insight & Conclusion

HDFlow proves that the future of robotic planning isn't about finding the "one best model," but about architectural symmetry. By matching the mathematical properties of a generative model (Diffusion's stochasticity vs. Flow's efficiency) to the specific needs of a planning layer, we can finally tackle long-horizon tasks that were previously too complex or too slow for generative AI.

Takeaway: If you are building a hierarchical controller, don't just stack Transformers. Ask whether the high level needs to explore and the low level needs to react—and pick your generative paradigms accordingly.