GMP: Learning When and What to Recall for Robust Robot Manipulation

The quest for "Generalist Robot Policies" often hits a surprising wall: Memory. While humans effortlessly switch between repetitive pick-and-place (Markovian) and complex trial-and-error adaptation (non-Markovian), current AI models struggle. Give them too little history, and they can't adapt; give them too much, and they overfit, jitter, and fail at the simplest tasks.

Researchers from Stanford University have introduced Gated Memory Policy (GMP), a framework that solves this dilemma by teaching robots not just how to remember, but when to forget.

The "Curse of History" in Robotics

In robotic manipulation, memory requirements vary wildly. Some tasks require:

• Markovian: No memory (e.g., standard pick-and-place).
• In-trial Memory: Remembering a color seen 10 seconds ago within a single attempt.
• Cross-trial Memory: Iteratively adjusting pushing speed across multiple attempts to account for unknown friction.

Surprisingly, simply feeding more history into a Transformer backbone (like a standard Diffusion Policy) often degrades performance. The input space becomes too vast, the model overfits to training demonstrations, and the quadratic computational cost of self-attention kills the robot's reactivity.

Methodology: The Architecture of Selective Recall

GMP addresses these issues through a clever three-pronged architecture:

1. The Memory Gate (The "When")

Instead of always looking at history, GMP uses an MLP-based Memory Gate (${\mu }_t$). It acts as a binary switch. If the current visual and proprioceptive state suggests the robot is in a routine phase, the gate stays closed ($\mu =0$).

• Calibration: The gate is trained via a self-supervised process. The team compares the error of a "No-Memory" model vs. a "Memory-Enabled" model. If the error drops significantly with memory, the gate learns to open at that specific timestamp.

2. Cross-Attention with Cached Tokens (The "What")

Rather than global self-attention, GMP uses a Cross-Attention module. The policy only queries the history when needed. Since history is stored as a sliding window of cached tokens, computation scales linearly, not quadratically.

Figure 1: The GMP architecture featuring the Gated Attention module and Diffusion Transformer backbone.

3. Diffusion Action Noising

To prevent the model from becoming a "slave" to perfectly clean training history, GMP injects noise into historical actions during training. This mimics real-world execution errors, forcing the model to extract high-level intent from history rather than memorizing exact trajectories.

Experimental Breakthroughs: MemMimic

The authors introduced MemMimic, a benchmark specifically designed to test long-term memory.

• Iterative Pushing & Casting: The robot must push objects with unknown friction. In Trial 1, it might undershoot. GMP "remembers" this failure and adjusts its velocity in Trial 2 and 3—true in-context adaptation.
• RoboMimic Robustness: On standard Markovian tasks (Square, Tool Hang), where other long-history models crashed to 0% success, GMP maintained competitive performance because its gate remained closed.

Figure 2: GMP enables iterative adaptation in complex tasks like Iterative Flinging, where the robot learns cloth mass through trial and error.

Efficiency and Real-World Deployment

One of the most impressive feats is inference speed. By skipping the heavy history attention when the gate is off, GMP maintains a constant, minimal latency. In real-world "In-the-wild" tests using an ARX X5 arm, GMP successfully handled human perturbations (flipping a cup) by recalling the cup's original placement orientation from minutes prior.

Figure 3: Inference time comparison showing GMP's linear scaling vs. the quadratic growth of standard self-attention.

Critical Analysis & Conclusion

While GMP is a massive step forward, it still relies on a finite attention window. If a critical piece of information occurred 30 minutes ago, it might be purged from the cache. The authors suggest future work on "Selective Caching"—deciding which high-value tokens to keep forever.

The Takeaway: GMP proves that for a robot to be smart, it must be selective. By gating memory and noising history, we can finally build policies that adapt to the unknown without losing their grip on the simple.

For more details, visit the Project Page.