Zero-Mem: Eliminating the Generative Tax in LLM Agent Memory
Zero-Mem: Zero-Token Memory Operations for LLM Agents
Zero-Mem is a novel agent-memory framework that achieves "zero-token memory operations" by eliminating all LLM calls and token consumption during memory construction, organization, and retrieval. It leverages a dual-view structure (entity-context graph and temporal hierarchy) to maintain competitive SOTA performance on benchmarks like LoCoMo and HotpotQA while relying on the LLM only for the final answer generation.
TL;DR
Current LLM agents are plagued by a "memory tax"—recurring token costs and latency added by using LLMs to summarize or manage their own past experiences. Zero-Mem breaks this cycle by introducing Zero-Token Memory Operations. It manages complex, multi-session memories using non-generative graph and temporal structures, invoking the LLM only once for the final answer. The result? A 57.6% speedup and zero operational token cost with higher accuracy than generative SOTA methods.
The Problem: The Generative Bottleneck
Memory is the backbone of consistent agent behavior. However, most current systems (like Mem0 or MemoryBank) use LLMs to "reflect" on or "summarize" history. This creates three critical failures:
- Traceability Loss: LLM-generated summaries often blur specific details or merge distinct entities, losing the "original evidence."
- Cost Explosion: Every memory update or retrieval step consumes tokens, making long-term deployments prohibitively expensive.
- Latency: Waiting for intermediate LLM calls to structure memory slows down agent responsiveness.
Zero-Mem asks a fundamental question: Does structured memory access require generation at all? The answer is a resounding no.
Methodology: Structured Evidence Selection
Zero-Mem replaces generative abstractions with a Provenance-preserving Token-Free Memory Substrate. Instead of "summaries," it maintains the raw interaction traces and organizes them through two specialized views:
1. The Relational View (Entity-Context Graph)
Using non-generative Named Entity Recognition (NER), Zero-Mem builds a graph where nodes represent entities and context units. It uses Personalized PageRank to propagate activations across the graph, allowing the agent to find "bridging" evidence that might not have a direct lexical match to the query.
2. The Temporal View (Hierarchical Trace Units)
To preserve the flow of conversation, Zero-Mem organizes traces into a hierarchy: Turns → Windows → Episodes. This ensures that if a specific turn is retrieved, its surrounding session context is also available, maintaining conversational locality.
Query-Conditioned Routing
Not all questions are the same. A question about "what happened yesterday" requires temporal priority, while "how are X and Y related" requires relational priority. Zero-Mem uses a deterministic routing mechanism to weight these two views dynamically.
Experiments: Superior Performance at Zero Cost
The researchers evaluated Zero-Mem against heavyweights like GAM and CompassMem on the LoCoMo (long-term conversation) and HotpotQA (multi-hop) benchmarks.
Key Results:
- Accuracy: On LoCoMo, Zero-Mem outperformed the best generative baseline by over 5 points in F1 score.
- Efficiency: While other methods consumed millions of tokens just to manage memory, Zero-Mem consumed zero.
- Speed: It is 57.6% faster than LightMem, the previous speed leader, because it avoids the overhead of intermediate model inference.
Deep Insight: Why "Zero-Token" Wins
The success of Zero-Mem suggests that the industry's reliance on "LLM-based reflection" might be an over-complication. By keeping the original traces as the authoritative source and using classical graph/hierarchical algorithms for organization, we avoid the "hallucinations" and "forgetting" that occur when an LLM tries to compress history into a summary.
Deterministic Evidence Calibration further ensures that the final LLM "reader" is grounded only in facts found within the traces, discarding conflicting or irrelevant noise.
Conclusion & Future Outlook
Zero-Mem marks a shift from generative memory to structured evidence selection. It proves that we can build agents that remember everything, trace every thought back to a specific interaction, and do so without the massive token bill. For the next generation of AI agents, the most efficient way to "remember" might just be to stop talking to themselves and start looking at the data.
Limitations: While the memory operations are zero-token, the system still relies on an external encoder (like BGE-M3) for dense embeddings. Future work could look at even more lightweight, local encoding methods to further reduce the compute footprint.