AgeMem: Unifying Long- and Short-Term Memory Within the Agent Policy

1.1 Core work

Authors: Yi Yu, Liuyi Yao, Yuexiang Xie, Qingquan Tan, Jiaqi Feng, Yaliang Li, Libing Wu　|　Affiliations: School of Cyber Science and Engineering, Wuhan University; Alibaba Group

1.2 Five related papers revisited

Common benchmarks: ALFWorld · SciWorld · BabyAI · PDDL (AgentBoard) · HotpotQA · LOCOMO · LongMemEval · MSC

2.1 Abstract

Limited context windows constrain long-horizon reasoning in language-model agents, making memory management essential. Existing approaches usually treat long-term memory (LTM) and short-term memory (STM) as separate components governed by heuristics or additional controllers, limiting adaptability and end-to-end optimization. The paper introduces AgeMem, a unified framework embedding LTM and STM management directly into the agent policy. Memory operations are exposed as tool-like actions so agents decide when and what to store, retrieve, update, summarize, or discard. To train this unified behavior, the authors propose three-stage progressive reinforcement learning and design step-wise GRPO to address sparse, discontinuous rewards from memory operations. Across five long-horizon benchmarks and multiple backbones, AgeMem consistently outperforms strong memory-augmented baselines, improving task performance, long-term memory quality, and context utilization.

2.2 Motivation and narrative

AgeMem follows a clear three-part argument: identify fragmentation, explain why patches cannot resolve it, and propose unification within a policy.

Step 1 · Two parallel memory-research strands reach their limits

Short-term memory—primarily retrieval augmentation and context compression—relies on predefined schedules or heuristics. These can both overlook infrequent but critical details and introduce unnecessary noise. Long-term approaches fall into two categories, each with weaknesses: Trigger-based systems perform fixed operations at preset times and are rigid; Agent-based systems rely on handcrafted rules or extra expert models, limiting adaptability and increasing complexity.

Step 2 · Fragmentation itself is the root problem

LTM and STM are typically separate, loosely coupled modules, producing fragmented memory construction and suboptimal long-horizon reasoning. Deciding whether to preserve information in a long-term store and whether to compress it out of the current context are fundamentally the same information-value judgment at two timescales. Independent heuristics inevitably risk irreversible loss, such as discarding short-term information before saving it long term.

Step 3 · Three tractable technical challenges

• C1: Coordinating heterogeneous functions: LTM and STM pursue different objectives—persistence versus immediate availability—and need unified orchestration.
• C2: Mismatched training paradigms: existing RL frameworks use substantially different training strategies for the two memory types. Standard RL conflicts with fragmented, discontinuous memory experiences: writing a memory yields no immediately observable benefit, which may emerge only much later when answering a question.
• C3: Deployment costs: auxiliary expert LLMs for memory control substantially increase inference costs and training complexity.

2.3 Methodology

2.3.1 State representation

At time t , the state is defined as s_t = (C_t, M_t, T): C_t is the short-term context, a message list; M_t is the long-term store, with content, embedding, and metadata per entry; T is the task specification, comprising query q, contextual information I_q, and expected answer A_q).

2.3.2 Six memory tools: A unified action space

The paper's central abstraction is: LTM and STM operations share one tool table and are invoked by one policy.

Notice the direction of FILTER : it deletes messages with excessively high similarity, suppressing repetition rather than removing supposedly irrelevant content. This conservatively protects critical low-frequency information.

2.3.3 Three-stage progressive RL curriculum

Each trajectory τ = (τ⁽¹⁾, τ⁽²⁾, τ⁽³⁾) has total length T = T₁+T₂+T₃. Between stages, context resets combined with long-term memory retention are the key curriculum mechanism.

2.3.4 Reward design

2.3.5 Step-wise GRPO: Addressing C2

The terminal advantage is broadcast uniformly to all preceding timesteps for long-horizon credit assignment, allowing final outcomes to supervise every intermediate memory decision across all three stages. The tradeoff is explicit: broadcasting sacrifices step-level credit-assignment precision for trainability under extremely sparse rewards.

2.4 Experimental design

Five long-horizon benchmarks cover embodied tasks, science, symbolic planning, navigation, and multihop QA: ALFWorld (household tasks), SciWorld (interactive scientific experiments), PDDL (symbolic planning), BabyAI (compositional grid-navigation instructions), and HotpotQA (Wikipedia multihop questions).

Backbone: Qwen2.5-7B-Instruct, Qwen3-4B-Instruct　|　Baseline: LangMem, A-Mem, Mem0, Mem0ᵍ, AgeMem-noRL, No-Memory

Metric: success rate SR on ALFWorld/SciWorld/BabyAI, progress rate PR on PDDL, and LLM-as-a-Judge on HotpotQA; memory quality MQ (0–1); average prompt tokens and tool calls.

Main results · Qwen2.5-7B-Instruct

Main results · Qwen3-4B-Instruct

On Qwen2.5-7B, AgeMem improves 49.59% over No-Memory and exceeds strongest baseline Mem0 by 4.82 pp. On Qwen3-4B, relative improvement is 23.52%, exceeding strongest baseline A-Mem by 8.57 pp. RL training itself contributes 8.53 pp / 8.72 pp.

A transition in tool-use behavior (HotpotQA · Qwen2.5-7B)

This table is the paper's most convincing qualitative evidence: UPDATE/DELETE before training, is exactly zero and becomes active after training; meanwhile, RETRIEVE decreases. This marks a shift from reactive to proactive memory behavior.

Ablations

Reward ablation(HotpotQA · Qwen2.5-7B): Answer-Only yields J=0.509 / MQ=0.479 / 2,078 tokens / 3.93 calls; All-Returns yields J=0.544 / MQ=0.533 / 2,117 tokens / 4.92 calls. Full rewards converge faster and improve memory quality at a modest token cost.
FILTER threshold sensitivity: θ_f ∈ [0.4, 0.8] is stable within this range, with 0.5 best (J=0.551); 0.4 overfilters and 0.7 is too permissive. The method is relatively insensitive to this hyperparameter.

2.5 Value analysis

1. Conceptual contribution: Redefining memory from a system component into policy behavior. This distinguishes AgeMem from previous work. The MemGPT-to-Mem0 lineage treats memory as an external system around an LLM, with storage rules in prompts or code and the model passively receiving retrieved results. AgeMem treats memory management as an intrinsic part of task solving, optimized jointly with reasoning. In the tool-use table, the shift in UPDATE/DELETE from zero to nonzero directly supports this claim.
2. Methodological contribution: A three-stage curriculum makes memory behavior learnable. The value of writing memory is unobservable at write time. Context resets and LTM persistence construct a causal chain from storage to task outcome, turning delayed value into a learning signal. This curriculum has independent methodological value for the field.
3. Engineering contribution: Removing auxiliary controllers. A-Mem and Mem0 require extra LLM calls for extraction and update decisions; folding these into the main policy simplifies inference costs.
4. Substantial experimental scope. Five cross-domain benchmarks × two backbone generations × six baselines, plus component, reward, and hyperparameter ablations, form a comparatively strong evaluation for memory research.

2.6 Limitations

Acknowledged by the paper

1. Fixed tool set: a clear and effective abstraction, but extensible toward finer-grained control.
2. Controlled evaluation settings: the benchmarks remain controlled relative to open-ended deployment; persistent conversations and real-user interactions are important next steps.
3. Dependence on a single training source: three-stage trajectories currently originate from HotpotQA.

Additional critical observations

4. Advantage broadcasting sacrifices credit-assignment precision. Uniform terminal-advantage broadcasting in step-wise GRPO reinforces every memory action in a successful trajectory equally, including redundant, ineffective, or harmful actions later compensated for by other steps. ADD The paper does not analyze whether the rise from 0.92 to 1.64 consists entirely of useful storage.
5. Efficiency gains are modest relative to the narrative. Tokens decrease by only 3.1% / 5.1%, compared against AgeMem-RAG rather than all baselines. Meanwhile, All-Returns uses more tokens (2,117 versus 2,078) and calls than Answer-Only, creating tension with the compression reward's intent.
6. MQ has limited discriminatory power. The MQ advantage over the strongest baseline is only 0.006 / 0.018, versus task gains of 4.82 pp / 8.57 pp. MQ may fail to capture what drives performance: STM context management and retrieval timing may matter more than stored-content quality.
7. Missing direct RL competitors is the most obvious evidential gap. Related work cites Memory-R1 (RL+LTM) and MemAct (RL+STM), but neither appears among the baselines, which are all heuristic. Testing the central claim that unification beats separation requires comparisons with one-sided RL methods. Current experiments establish only RL superiority over heuristics.
8. A-Mem wins on PDDL. For Qwen2.5-7B, 17.31 versus 18.39 is AgeMem's sole loss, without detailed analysis. Symbolic planning needs structured memory, where A-Mem's explicit links may have an inherent advantage, suggesting a ceiling for flat entries plus semantic retrieval.
9. Scaling limits remain unknown. Backbones stop at 7B. Does three-stage RL still help at 30B/70B, or do native long-context capabilities absorb the gain? Supersede's recent finding of a 15 pp accuracy cliff in frontier models under bounded memory suggests the problem persists, but direct testing is needed.

R1 · MemGPT: Towards LLMs as Operating Systems (2023-10)

Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G. Patil, Ion Stoica, Joseph E. Gonzalez (UC Berkeley)

Problem addressed: systematically introduces the operating-system analogy of virtual-memory hierarchies to LLMs: paging exchanges information between main context (RAM) and external storage (disk), initiated through model function calls, while interrupts let the agent resume tasks after memory operations. It establishes that LLMs can manage their own memory, originating the memory-as-tools paradigm.

Remaining problems: memory policies are entirely driven by prompt engineering. Paging times and storage choices are specified in system prompts. Policies cannot improve from experience, are sensitive to wording, and largely fail on small models without GPT-4-level instruction following. MemGPT also focuses on overflowing context rather than independently asking what deserves long-term retention.

AgeMem's response: retains tool-based memory actions but moves decisions from prompts into trainable policy parameters. Its gains on open 4B/7B models arise because memory behavior no longer relies solely on innate instruction following.

R2 · A-MEM: Agentic Memory for LLM Agents (NeurIPS 2025)

Wujiang Xu, Zujie Liang, Kai Mei, Hang Gao, Juntao Tan, Yongfeng Zhang (Rutgers and collaborators)

Problem addressed: most post-MemGPT systems use flat entry collections with semantic retrieval but no relationships among memories. A-MEM introduces Zettelkasten principles: new memories become structured notes with context descriptions, keywords, and tags; the system automatically identifies existing-memory Link and supports memory evolution, where new entries trigger updates to old ones. Storage becomes a self-organizing knowledge network, exceeding contemporary SOTA across six foundation models.

Remaining problems: link creation and evolution remain LLM-prompt-driven without an optimization objective, so the system cannot know whether its links help downstream tasks. Incorrect updates can also cascade through the network, a risk later demonstrated as error propagation by ACL 2026's How Memory Management Impacts LLM Agents . A-MEM manages only long-term memory, leaving short-term context to external RAG.

AgeMem's response: uses A-MEM as a main baseline and beats it overall (45.74→54.31 on Qwen3-4B). Task rewards replace prompting heuristics, giving maintenance behavior a correctness signal. However, AgeMem loses to A-MEM on PDDL, suggesting explicit links retain advantages on highly structured tasks that AgeMem's flat representation lacks. These technical strands remain uncombined.

R3 · Mem0: Production-Ready AI Agents with Scalable Long-Term Memory (2025-04)

Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, Deshraj Yadav (Mem0 team)

Problem addressed: moves memory from research prototypes toward production readiness. A memory-centric extract–update pipeline dynamically extracts and retrieves conversational information, with graph variant Mem0ᵍ. It beats six baseline families on LOCOMO; relative to full context, engineering gains include 91% lower p95 latency and over 90% token-cost savings, alongside a 26% relative improvement over the OpenAI approach. It is currently a de facto open-source memory-layer standard in industry.

Remaining problems: each extract–update threshold and rule is manually tuned, requiring retuning across domains. More fundamentally, extraction optimizes the proxy objective of correctly extracting information rather than helping the agent complete tasks. These objectives need not align. Separate extraction/update LLM calls also create AgeMem's C3 cost.

AgeMem's response: outperforms Mem0 and Mem0ᵍ on both backbones, folds extraction decisions into the main policy, and replaces proxy objectives with task rewards. However, Mem0's LOCOMO conversational setting differs from AgeMem's agentic tasks; AgeMem does not validate conversational memory, precisely the concern in its second acknowledged limitation.

R4 · Memory-R1: Manage and Utilize Memories via RL (2025-08)

Sikuan Yan, Xiufeng Yang, Zuchao Huang, Ercong Nie, Zifeng Ding, and colleagues (LMU Munich / TU Darmstadt / Edinburgh; Hinrich Schütze, Volker Tresp)

Problem addressed: First introduces RL to memory management, using two specialized agents: Memory Manager, which chooses ADD/UPDATE/DELETE/NOOP, and Answer Agent, which distills retrieved memories before answering. PPO and GRPO train them using downstream QA correctness. Particularly striking data efficiency comes from only 152 training QA pairs, outperforming strong baselines and generalizing across LoCoMo, MSC, LongMemEval, and 3B–14B scales.

Remaining problems: The two-agent architecture is exactly the separation AgeMem aims to remove. Memory Manager and Answer Agent are optimized independently. The manager does not directly know the answerer's needs, and the answerer cannot directly alter storage decisions; rewards couple them indirectly. Memory-R1 also covers only LTM writing and does not address short-term context management such as compression, filtering, or overflow, while maintaining two parameter sets.

AgeMem's response: Memory-R1 is AgeMem's most direct predecessor and target for unification. One policy handles management and solving, with STM tools in the same action space. Unfortunately, AgeMem does not empirically compare against Memory-R1, leaving superiority over two-agent designs unproven.

R5 · MemAct — Memory as Action: Autonomous Context Curation (ACL 2026 Findings)

Yuxiang Zhang, Jiangming Shu, Ye Ma, Xueyuan Lin, Shangxi Wu, Jitao Sang (ADaM Lab, Beijing Jiaotong University)

Problem addressed: complements Memory-R1 by formalizing short-term context management as learnable policy actions. Even larger-context LLMs need careful working-memory management to mitigate attention dilution in long tasks. MemAct proactively curates context with RL-trained deletion and insertion, enabling substantially smaller models to achieve competitive accuracy while reducing context length by roughly half.

Remaining problems: MemAct only inserts and deletes within context, with no persistent external memory store. Deleted information is permanently lost and cannot be recovered in later sessions. This may suit single long tasks but undermines persistent interaction, and leaves cross-task retention unresolved.

AgeMem's response: FILTER with SUMMARY serve MemAct's role, but AgeMem's context cleanup and LTM ADD share one action space. Its policy can learn to save information long term before deleting it from context, whereas MemAct can only delete or retain. This is an architectural benefit of unification. Nevertheless, MemAct's roughly halved context far exceeds AgeMem's 3–5% savings, so AgeMem does not dominate specialized methods on efficiency.

The underlying trajectory is one of progressive internalization: decisions move from prompts to code rules to trained auxiliary models and finally into the agent's own policy. Each step reduces human priors and increases learning from experience. AgeMem is the current endpoint of this path, while its fixed tools and flat representation suggest what to internalize next.

1. Unification versus separation lacks a decisive experiment. Memory-R1 (RL+LTM), MemAct (RL+STM), and AgeMem (RL+unification) occupy three points in the design space, but no paper evaluates all three under the same setup. A rigorous controlled study could establish or falsify a central architectural claim at relatively low entry cost.
2. Structured representations and learnability remain unreconciled. AgeMem's PDDL loss to A-MEM suggests limits to flat entries and semantic retrieval. Incorporating explicit A-MEM links, MAGMA's multigraph architecture, or EverMemOS's self-organization into learnable actions—for example, adding LINK/MERGE tools—is a natural next step, but enlarges the action space and complicates credit assignment.
3. Credit-assignment granularity. Uniform step-wise GRPO broadcasting is a pragmatic workaround. Achieving genuine step-level credit under extremely sparse, cross-stage memory rewards—for example, asking counterfactually whether a task would succeed without a stored memory—is a central technical challenge.
4. Evaluation validity. The MQ/performance disconnect (0.006 versus 4.82 pp) suggests LLM-rated memory quality may miss what matters. LongMemEval-V2, AMA-Bench, and Supersede's supersession gap—continued use of stale values after factual updates—approach this problem from different angles.
5. Memory security and governance. Learnable memory is also vulnerable to contamination. Work in 2026 includes a long-term memory security survey (arXiv:2604.16548), SSGM governance (arXiv:2603.11768), and ACL 2026 error-propagation studies. How RL-trained rather than handcrafted write policies change attack surfaces is largely unexplored.
6. Real-world settings and scale. Current evaluations use controlled benchmarks. Persistent multisession interaction, memory evolution over weeks or months, and whether these methods remain necessary above 30B parameters are essential deployment questions.