Why long-memory tests miss the bias toward recent facts
Most temporal knowledge graph reasoning models are built to prioritize recent history, and long-memory evaluations often don't catch how much this hurts performance on older but still relevant facts. In a 2023 study, researchers found that methods relying on recursive embedding learning—a way of updating representations over time—tend to focus on recent facts and overlook earlier ones, leading to imprecise knowledge propagation [3]. Similarly, a 2024 model called DyMemR was designed specifically because many existing models only consider a fixed time range, losing useful historical information [1]. If your evaluation only tests recent memory, it will miss this bias entirely.
The consequence is that an agent might ace a test that asks about last week's events but fail when asked about something from months ago that is still relevant. The papers suggest that memory should be dynamic—like human memory, with capacity limits and repetition effects—rather than a simple recency filter [1]. A good long-memory evaluation should include questions that require retrieving facts from varied time depths, not just the most recent sessions.
The problem of noise: when more history isn't better
Another blind spot is that long-memory evaluations often assume that having more history is always better, but in practice, too much history can introduce noise and invalid facts. The DyMemR paper explicitly notes that using all historical facts introduces noise, which hurts reasoning [1]. Their solution was a memory pool that selectively keeps potentially useful facts, mimicking human memory with loss and repetition. If your evaluation doesn't test whether the agent can filter out irrelevant or contradictory information, it won't reveal this failure mode.
A 2023 model called MTDM took a different approach: instead of using all history, it directly uses the facts most relevant to the missing knowledge, which improved prediction accuracy and efficiency [3]. This suggests that long-memory evaluations should include distractor facts or events that are temporally close but semantically unrelated, to see if the agent can ignore them. Without such tests, an agent might appear to have good memory when it's actually just memorizing everything without discrimination.
The hardest part: temporal and causal dynamics
Even when models have access to long context, they struggle with understanding the temporal and causal relationships between events—a gap that standard long-memory benchmarks often don't probe. In a 2024 study introducing LoCoMo, a dataset of very long-term conversations (up to 32 sessions, 600 turns, and 16K tokens on average), researchers found that LLMs exhibit challenges in comprehending long-range temporal and causal dynamics within dialogues [2]. Even with long-context LLMs or retrieval-augmented generation (RAG), performance still substantially lagged behind human performance [2].
This is a crucial insight: a model might recall individual facts correctly but fail to connect them in time or understand cause and effect. For agent memory, this means that an evaluation should include tasks like event summarization or questions that require reasoning across multiple time points, not just single-fact retrieval. The LoCoMo benchmark includes such tasks, but many existing evaluations don't, so they miss this fundamental limitation.
About These Sources
This answer is built on 5 peer-reviewed studies — published from 2023 to 2026, 3 from 2024 or later, 1 in Q1 journals, collectively cited 51 times — selected as the most relevant from 5 studies that passed quality screening, drawn from 35 papers retrieved from a database of over 500 million.
Sources used in this answer
Temporal Knowledge Graph Reasoning With Dynamic Memory Enhancement
Proposed DyMemR, a TKG reasoning model with a human-like memory pool (capacity, loss, repetition) to selectively retain useful historical facts, addressing the problem of noise from using all history and the loss of information from fixed time ranges.
Evaluating Very Long-Term Conversational Memory of LLM Agents
Introduced LoCoMo, a dataset of very long-term conversations (up to 32 sessions, 600 turns, 16K tokens on average), and found that LLMs struggle with long-range temporal and causal dynamics, even with long-context or RAG, still lagging behind human performance.
Temporal knowledge graph reasoning triggered by memories
Proposed MTDM, a memory-triggered decision-making network that uses earlier historical facts for initial representations and recent facts for updates, and directly uses the most relevant facts to improve prediction accuracy and efficiency, mitigating imprecise knowledge propagation.
Adaptive Path-Memory Network for Temporal Knowledge Graph Reasoning
Proposed DaeMon, an adaptive path-memory network that models temporal path information without depending on entity representations, which is useful for handling the large and growing number of entities in real-world TKGs, achieving up to 4.8% absolute improvement in MRR over state-of-the-art.
Temporal graph memory networks for knowledge tracing
Proposed a deep temporal graph memory network for knowledge tracing that jointly models relational and temporal dynamics, and introduced a generic technique for representing forgetting behavior via temporal decay constraints, showing superior performance on multiple benchmarks.