What would prove memory is real, not a benchmark trick?
The key is to look for evidence that memory improves performance in ways that generalize beyond the specific test set. That means showing gains on real robots, under perceptual ambiguity, and with data efficiency—not just on a benchmark where the memory module might be overfit. For instance, NativeMEM reports success rates jumping from 32.4% to 84.0% in simulation and up to 98.7% on real robots, and it achieves these results with only 20% of the training data used by prior methods [1]. That combination—large absolute gains plus data efficiency—is hard to explain as a benchmark artifact.
Another strong signal is when memory helps in conditions that mimic real-world messiness. Chameleon explicitly targets perceptual aliasing, where the same observation can arise from different histories, and shows consistent improvements in decision reliability and long-horizon control over strong baselines in those confusable settings [3]. Similarly, CraniMem demonstrates robustness to injected noise, with smaller performance drops under distraction compared to baselines [6]. These are exactly the conditions where a benchmark trick would fail.
How much does memory actually help, and under what conditions?
The gains are substantial but not universal. BATON, which adds transition-aware memory to a frozen vision-language-action model, improves task success by 11.6% and cumulative success by 14.9% over the state of the art on the RoboMemArena benchmark [2]. That's a meaningful jump, but it's on a specific benchmark. The real proof comes from the fact that BATON's memory is designed to handle subtask transitions—a common failure point in long-horizon tasks—and it does so without updating any parameters, suggesting the memory mechanism itself, not extra training, drives the improvement.
The conditions matter: memory helps most when tasks are long, partially observable, or perceptually ambiguous. Scene Memory Transformer (SMT) shows superior performance over reactive and memory-based policies on visual navigation tasks, which are inherently long-horizon and partially observable [7]. MIKASA-Robo provides a standardized benchmark of 32 memory-intensive tabletop manipulation tasks, explicitly designed to assess memory capabilities [5]. These studies converge on the same conclusion: memory is most valuable when the task requires integrating information from the past to make current decisions.
What are the honest caveats?
Not all memory systems are created equal, and some may be more like clever compression than true memory. For example, MM-Mem uses a pyramidal memory architecture inspired by fuzzy-trace theory, but its gains are on video understanding benchmarks, not robot manipulation [4]. That's a different domain, so its relevance to robot manipulation is indirect. Also, some memory designs rely on external databases or ad hoc read/write rules, which can be unstable under distraction [6]. The strongest evidence comes from systems that integrate memory directly into the policy, like NativeMEM, rather than bolting on an external memory module.
Another caveat: many of these results are from simulation or controlled lab settings. While NativeMEM reports real-robot success up to 98.7%, that's on a specific task setup [1]. The field lacks a universal benchmark for memory in robot manipulation, which is exactly why MIKASA-Robo was created [5]. So while the evidence is promising, it's not yet definitive across all real-world conditions. The most convincing proof would be independent replication across multiple labs and tasks, which is still emerging.
About These Sources
This answer is built on 7 studies (1 peer-reviewed, 6 preprints) — published from 2022 to 2026, 6 from 2024 or later — selected as the most relevant from 7 studies that passed quality screening, drawn from 59 papers retrieved from a database of over 500 million.
Sources used in this answer
NativeMEM: Native Memory Compression for Long-Horizon Robotic Manipulation
NativeMEM, a VLA policy with native memory compression, boosts success rates from 32.4% to 84.0% in simulation and up to 98.7% on real robots, while using only 20% of the training data of prior methods.
Don't Drop the BATON: Long-Horizon Robot Manipulation via Agentic Subtask Exploration and Transition-aware Memory
BATON, which adds transition-aware memory to a frozen VLA, improves task success by 11.6% and cumulative success by 14.9% over the state of the art on the RoboMemArena benchmark, without updating any parameters.
Chameleon: Episodic Memory for Long-Horizon Robotic Manipulation
Chameleon, using geometry-grounded episodic memory, consistently improves decision reliability and long-horizon control over strong baselines in perceptually confusable settings, as demonstrated on a real-robot UR5e dataset.
From Verbatim to Gist: Distilling Pyramidal Multimodal Memory via Semantic Information Bottleneck for Long-Horizon Video Agents
MM-Mem, a pyramidal multimodal memory architecture, achieves state-of-the-art performance on four video understanding benchmarks, but its focus is on video agents, not robot manipulation.
Memory, Benchmark & Robots: A Benchmark for Solving Complex Tasks with Reinforcement Learning
MIKASA-Robo introduces a benchmark of 32 memory-intensive tabletop manipulation tasks, providing a standardized way to evaluate memory capabilities in robotic manipulation.
CraniMem: Cranial Inspired Gated and Bounded Memory for Agentic Systems
CraniMem, a gated and bounded memory design, is more robust than Vanilla RAG and Mem0 baselines under injected noise, with smaller performance drops under distraction.
Scene Memory Transformer for Embodied Agents in Long-Horizon Tasks
Scene Memory Transformer (SMT) demonstrates superior performance over reactive and memory-based policies on a range of visual navigation tasks, which are long-horizon and partially observable.