[AndroTMem] From Sequential Noise to Causal Anchors: Solving the "Memory Wall" in Long-Horizon GUI Agents

TL;DR

Execution of long-horizon tasks (30+ steps) remains the "final boss" for mobile GUI agents. AndroTMem reveals that the bottleneck isn't perception—it's memory representation. By replacing messy raw histories with Anchored State Memory (ASM), the researchers achieved performance boosts of up to 30% in task completion, proving that what an agent remembers is just as important as what it sees.

The "Memory Wall" in Mobile Interaction

Most current GUI benchmarks focus on "atomic" tasks: "Open Spotify" or "Search for a pizza." But real-world productivity is messy. Imagine comparing prices for AirPods across three different apps, checking your bank balance, and finally sending a recommendation to a friend.

In these long-horizon scenarios, agents hit a wall. If they keep the full history, the "noise" of previous UI transitions dilutes their attention. If they use a summary, they lose the exact price or contact name needed 20 steps later. This leads to State Loss and Context Drift, where the agent simply forgets why it is on a specific page.

Methodology: Anchored State Memory (ASM)

The core innovation of AndroTMem is the shift from linear history to relational anchors.

1. The Anatomy of an Anchor

Instead of a list of screenshots, ASM stores history as a set of $A=\{m_1,m_2,...,m_k\}$. Each anchor $m$ contains:

• Type: Is this a Subgoal, a Dependency, or an Exception?
• Content: The semantic information (e.g., "Price is $199").
• Evidence: The specific UI grounding for that state.
• Links: Causal pointers to previous anchors.

2. The Retrieve-Reason-Update Cycle

Unlike traditional models that ingest history like a text file, ASM-equipped agents act like a human with a notepad:

1. Retrieve: Fetch only the anchors relevant to the current screen.
2. Act: Decide the next step based on the current UI + retrieved anchors.
3. Update: Create or update anchors based on the outcome of the action.

Figure 1: Overview of AndroTMem and the ASM mechanism.

The Benchmark: AndroTMem-Bench

To test this, the authors created a massive dataset of 1,069 tasks across 50 apps. These aren't just "long" tasks; they are causally-dense.

• Avg Steps: 32.1 (Max 65).
• Intent Categories: 8 types (Compare & Decide, Purchase, Communicate, etc.).

Experiments: ASM vs. The World

The researchers tested 12 agents, including heavy-hitters like Gemini-3-Flash and GPT-4o. The results were startlingly consistent:

As shown in the graph below, as the number of interaction steps increases, the performance of "Raw History" and "Summary" agents plummets. However, ASM maintains a significantly higher success rate even at the 60-step mark.

Figure 2: Performance degradation curves. Note how ASM preserves capability as horizons lengthen.

Why It Works: A Structural Advantage

The study identified five critical failure modes that ASM mitigates:

1. State Loss: Forgetting previous values.
2. State Mis-binding: Associating a price with the wrong product.
3. Context Drift: Losing the "thread" of the task.
4. Unverified Progress: Assuming a click worked when it didn't.
5. Interruption Failure: Getting stuck on pop-up ads.

By treating these as specific "Anchors," the agent treats an ad pop-up as an [EXCEPTION] to be handled and dismissed, rather than a permanent change in the task context.

Critical Insight & Conclusion

The industry has spent years improving UI Grounding (can the agent click the right button?). AndroTMem suggests we have reached diminishing returns there. The next frontier is Interaction Memory.

Takeaway: Future GUI agents shouldn't just be "LLMs with eyes"; they must be "LLMs with a structured, causally-aware memory bank." ASM provides the blueprint for this transition, moving us closer to agents that can actually handle our complex, multi-app daily digital lives.

Limitations

While ASM is powerful, the authors note it doesn't yet solve cross-session memory (remembering a task from yesterday). Furthermore, the bench relies on stable app environments; real-world "UI drift" (apps changing layouts overnight) remains a challenge for future iterations.