Memory Is Not an Asset but a Self-Amplifying Loop

0. Search Disclosure

This issue searched ACL Anthology 2026 (acl-long / findings-acl / eacl-long), ICLR 2026 virtual and the MemAgents Workshop, ICML 2026 paper pages, OpenReview, and memory-related cs.CL / cs.AI entries in arXiv 2601–2608, screening 28 candidates, including 4 surveys and 6 new 2607 preprints, spanning memory architectures, retrieval, compression and forgetting, RL-trained memory policies, evaluation benchmarks, safety governance, and multi-agent shared memory.

Prioritizing “top-conference main-track long papers > Findings/Workshop > arXiv,” the final core selection was ACL 2026 main-conference long paper 2026.acl-long.27 . It is outside the initially covered list (MemGPT / A-MEM / Mem0 / Memory-R1 / MemAct / AgeMem) and is precisely the Source of the problem.

1.1 Bibliography and Positioning

The paper describes its contribution as a systematic quantitative analysis of agent memory dynamics through the two most basic operations—addition and deletion—revealing the experience-following property: “high input similarity between the current task query and the query in a retrieved record often produces high output similarity between their execution results”(p. 623). This yields two challenges—error propagation with misaligned experience replay.

1.2 Motivation: Using Data to Reject “More Experience ≥ Less Experience”

Since 2023, experiential memory—Generative Agents' memory stream, Voyager's skill library, ExpeL's experience pool, and AWM's workflow induction—has almost uniformly adopted additive designs: write back after execution, implicitly assuming more experience cannot hurt. The paper first dismantles this assumption.

1.3 The Paper's Narrative and Argument Structure

Rather than “propose a method → top a leaderboard,” the argument is a diagnostic chain:

1.4 Methodology and Mechanisms

The paper proposes no new architecture; it constructs a two-axis experimental design: addition policy (write-gate quality) × deletion policy (reclamation mechanism).

1.4.1 Formalizing Addition Policies

1.4.2 Formalizing Deletion Policies

1.4.3 Experimental Setup

Backbone model: GPT-4o-mini(most experiments); robustness checks use GPT-4o / DeepSeek-V3 / Qwen (Appendix B.2). RegAgent measures output similarity with an RBF kernel exp(−γ|x₁ − x₂|²), γ = 1.0 (Appendix A.1); input similarity is uniformly cosine similarity between text-encoder features (p. 625).

1.5 Experimental Design and Quantitative Results

Table 1 · Complete Addition-Policy Matrix (Paper Table 1)

Key Derived Quantities (Calculated in This Report)

Transition · The Paper's Most Persuasive Qualitative Evidence

On EHRAgent, performance follows evaluator quality in a strictly monotonic ladder, with clearly diminishing marginal gains:

But the real transition occurs on RegAgent :

Table 2 · Complete Deletion-Policy Matrix (Paper Table 2)

Key Derived Quantities (Calculated Here, Relative to No Deletion Under the Same Evaluator)

A Design Principle Supported by the Data but Not Explicitly Stated

Distribution Shift and Capacity Constraints (Textual Descriptions Only; Figure Values Not Extractable)

• Task distribution shift (Figure 7): for EHRAgent, strict addition / history-based / combined retain a small performance gap relative to no-shift variants; for AgentDriver, strict addition alone even exceeds the no-shift variant. Combined underperforms history-based deletion under shift for EHRAgent, while periodic deletion helps stabilize performance during shifts.
• Memory capacity constraints (Figures 17 / 18): EHRAgent's strict selective addition remains close to the unlimited-memory baseline under constrained capacity; combined deletion lets AgentDriver adapt to memory constraints and maintain performance. As constrained capacity grows, performance gradually converges, suggesting that naive unbounded memory growth is unnecessary.

1.6 Value Analysis

Its real contribution is not“proving selective addition useful” (already intuitive), but three stronger achievements:

1. Repositioning memory management from retrieval engineering to closed-loop control. Experience-following is the pivot: once agents follow memory, the store changes from passive data storage into a system state variable, and the write policy becomes a control law. Subsequent work on learned memory policies—Memory-R1, AgeMem, MemCon—makes sense only in this framework.
2. Providing a portable definition of value: a record's value = the average downstream score when it serves as a demonstration (Φ). Backend-independent, requiring neither LLM calls nor gradients, this is the lowest-cost current form of credit assignment for memory governance.
3. Pricing the evaluator, a component previously treated as a free resource. The C1/C2/C3/Strict ladder turns evaluator quality into a knob with directly readable performance consequences: if only a GPT-4o-mini-level judge is available, the data predict a 4.35 pp loss on RegAgent-like tasks.

Standalone Reusable Components:φ_hist 's utility-backfill mechanism; the experimental paradigm treating evaluator quality as a first-class variable; and the coupling principle derived here but unstated in the paper: deletion aggressiveness ∝ 1/evaluator quality.

1.7 Limitations

Author-Reported Limitations (Direct Quotations)

1. “One limitation of this work concerns the scope of memory management considered. … This study focuses on two basic, task-independent operations—memory addition and memory deletion—while omitting more complex mechanisms such as structural transformation, merging, summarization, and reflection.”
2. “Another limitation is the lack of theoretical guarantees. Our findings are primarily based on extensive empirical analysis, which, although comprehensive, does not provide formal theoretical proofs. Given the inherent complexity of agent systems, deriving such guarantees is often infeasible.”

Additional Critique (Independent Observations)

1. Evaluator quality is collapsed into a one-dimensional ladder, but Table 1 contradicts this. On CIC-IoT, C2 (GPT-4.1-mini, 68.80) is below C1 (GPT-4o-mini, 74.00) by 5.20 pp; on EHRAgent, C2 (32.21) exceeds C1 (26.19) by 6.02 pp. The same judges produce opposite orderings on two tasks, indicating C1 < C2 < C3 is not an intrinsic evaluator capability ranking but the product of judge–task fit. The naming and narrative imply a monotonic ladder inconsistent with the data.
2. The central “quality rather than scale” attribution lacks a controlled comparison. Add-all versus Fixed simultaneously changes record quality and memory Size(11.8×–41.0×). To establish quality as the main cause, a control must randomly downsample Add-all to Strict's size, such as 4100 → 2938. None is reported, so the competing explanation that an oversized retrieval pool reduces top-k precision cannot be ruled out.
3. The best genuinely deployable method is no better than doing nothing on RegAgent. C3 (67.35, 3139 records) versus Fixed (67.53, 100 records) gives −0.18 pp / 31.4× memory. The paper uses Strict (+3.42 pp) to support selective addition, but Strict = π_human, comparison with ground truth—precisely what real deployment lacks. It does not directly address whether the conclusion survives without ground truth.
4. No head-to-head with learned memory-management competitors. All Table 2 deletion policies are handcrafted. Memory-R1 at the same ACL 2026 (2026.acl-long.583) learns ADD/UPDATE/DELETE/NOOP via PPO/GRPO and, with only 152 training QA examples, beats heuristic baselines on LoCoMo / MSC / LongMemEval across 3B–14B. Without any learned manager as a control, the data cannot decide whether the bottleneck is evaluator signal or fixed policy rules. This is a gap in decisive experimentation, rather than merely supplementary experiments.
5. The main evidence for experience-following comes from the easiest setting in which to establish it.r ≈ 1 It comes mainly from RegAgent (Figure 3), a synthetic one-dimensional regression task whose output similarity is directly defined as exp(−γ|x₁−x₂|²). In a continuous, low-dimensional, smooth output space, similar inputs producing similar outputs is almost a property of the task itself, not of agent behavior. Figure 9's supplementary results are incomparable in metric terms: SR vs. ACC and discrete code vs. continuous trajectories.
6. Deletion-policy hyperparameter sensitivity is entirely absent.φ_per depends on a window and threshold α; φ_hist depends on β and minimum retrieval count n. Table 2 reports single points, yet the same Periodic policy on CIC-IoT swings across evaluator levels from +4.10 pp to −4.60 pp(8.70 pp), suggesting extreme sensitivity. Without α/β sweeps, one cannot tell whether Table 2 values lie at their respective optima or whether policy rankings are robust.
7. Deletion benefits are unevenly distributed; the abstract is optimistic. Strict + History-based relative to no deletion: EHRAgent +3.39 pp, CIC-IoT +4.20 pp, AgentDriver +0.81 pp, RegAgent −1.15 pp. Of four agents, one clearly worsens and one is essentially unchanged; “notable improvements” obscures the 2:1:1 distribution. The synthetic task is the sole deterioration, likely because every regression demonstration carries signal and no record is truly harmful. This valuable counterexample is not explored.
8. Statistical significance reporting is missing. CIC-IoT has only 1000 cases, so 1 pp ≈ 10 examples; Table 2's 0.10 pp difference between Combined (85.50) and No deletion (85.40) represents one example. No confidence intervals, repeated-run variance, or significance tests are reported. By comparison, contemporary Supersede (arXiv 2606.27472) explicitly reports p = 0.0033 for similarly sized experiments. When experiential-memory benefits are only a few percentage points, this weakens confidence in fine-grained rankings.
9. Limitations of This Report: Pointwise values in Figures 3–7 and 17–18 could not be extracted from the ACL Anthology PDF text layer. Thus error-propagation curves—including overtaking the ground-truth baseline after about 2000 executions—the KDE quality gap, distribution shift, and capacity constraints are described only through the paper's text, without independent quantitative verification. The official code repository was also not located, so exact α, β, and n settings could not be verified.

2.1 Generative Agents — Park et al., UIST 2023 (arXiv:2304.03442)

Problem addressed. First made long-term memory a first-class component of LLM agents. A memory stream plus recency × importance × relevance three-term weighted retrieval maintains coherent agent behavior across dozens of simulated social days. It established the still-default paradigm of memory as a retrievable experience stream.

Unresolved problem. All three scores are handcrafted heuristics describing what is worth retrieving, not what is worth storing. The memory stream is append-only, with neither a write gate nor deletion. It assumes records are trustworthy because observations in the simulated society do not go wrong. Transferred to real tasks, where observations are agents' own execution results, this premise collapses.

How the core paper responds. The core paper names it as a baseline and quantifies the cost of append-only behavior through Add-all: negative on all four agents, −16.2% to −22.1%. It retains the retrieval heuristic, still using cosine-similarity top-k, but inserts an upstream write gate π and a downstream reclamation operator φ.

Transfer of key designs.recency evolves into φ_per(delete if retrieval count in the window ≤ α): from ranking older records lower to removing cold records, and from soft scoring to hard reclamation.

2.2 Voyager — Wang et al., 2023 (arXiv:2305.16291 · TMLR)

Problem addressed. The skill-library paradigm stores and reuses executable programs that pass verification. It was the first system to get the write gate right: only skills passing environment self-verification are stored.

Unresolved problem. This gate is perfect because Minecraft provides a free, deterministic, zero-cost verifier. Most real task domains do not: EHR code generation lacks an oracle, and driving trajectories lack immediate ground truth. Voyager thus leaves the entire problem of obtaining a verifier where none exists to later work.

How the core paper responds. The core paper answers precisely this question. Its C1/C2/C3/Strict ladder asks how much is lost with approximate verification. Strict (π_human) is the idealized upper bound of Voyager-style perfect verification for general tasks; C1–C3 are practical substitutes one can obtain. The core contribution can be restated entirely as pricing Voyager's self-verification mechanism.

Transfer of key designs.“Verify before writing” is abstracted into the general indicator function π(q, e) and relaxed from binary execution success/failure to a decision from a potentially fallible evaluator.

2.3 ExpeL — Zhao et al., AAAI 2024 (arXiv:2308.10144)

Problem addressed. Includes both successful and failed trajectories in an experience pool and uses LLMs to extract natural-language insights from cross-task comparisons. It upgrades experience from reusable solutions to transferable rules and first explicitly argues that failure experiences have value.

Unresolved problem. Insight extraction is unidirectional and additive: insights can be added or upvoted, but are not reclaimed based on downstream outcomes. A direct tension exists between the value of failure experiences and their reproduction through experience-following. ExpeL assumes the LLM extracts correct lessons from failures; the core error-propagation experiments show that when extraction is unreliable, the stored object is the error itself, not a lesson.

How the core paper responds. This is An adversarial response. Figure 4 shows that both add-all and coarse selective addition widen the performance gap between an agent and its error-free variant (p. 628). Storing flawed executions is harmful on average unless the evaluator is sufficiently strict. This does not deny the value of failure experience; it quantifies the conditions needed to realize it.

Transfer of key designs. ExpeL's UPVOTE/DOWNVOTE is replaced by a harder quantity Φ(q_m, e_m)—the downstream task's actual score after retrieving the record, rather than an LLM's subjective assessment. This is the key move from self-assessment to external feedback.

2.4 Agent Workflow Memory — Wang et al., 2024 (arXiv:2409.07429 · ACL 2025)

Problem addressed. Rather than storing raw trajectories, it induces reusable workflows from them, substantially improving generalization on Mind2Web / WebArena. It raises memory granularity from instances to abstractions.

Unresolved problem. Abstraction does not eliminate errors; it promotes them to a higher level. Workflows induced from erroneous executions may be retrieved by more tasks and cause broader harm. AWM is also additive, growing without deletion or usage-based retirement.

How the core paper responds.Complementary rather than adversarial. The core paper explicitly excludes structural transformation, merging, summarization, and reflection in its Limitations, placing AWM's layer outside scope. Yet misaligned experience replay applies precisely to AWM: a workflow may be entirely correct but mismatched to the current task, and such records can only be identified by downstream utility (φ_hist), rather than correctness checks (π). The core paper supplies that identification mechanism—the missing half of AWM.

Transfer of key designs. Workflow reuse frequency is formalized as fr_t(q_i, e_i) and becomes a shared trigger for both deletion rules.

2.5 Memory-R1 — Yan et al., ACL 2026 Main-Track Long Paper (2026.acl-long.583, pp. 12805–12825)

Problem addressed. The paper states that most existing pipelines are static and heuristic, lacking a learning mechanism to decide what to store, update, and retrieve. Two agents—a Memory Manager learning ADD / UPDATE / DELETE / NOOP and an Answer Agent—are fine-tuned using outcome-driven RL via PPO and GRPO, only 152 training QA examples, already surpassing strong baselines on LoCoMo / MSC / LongMemEval across 3B–14B.

Unresolved problem. It learns the management policy, but its reward comes from QA correctness, a relatively clean signal. For open tasks without verifiable answers, such as code generation or driving, the reward source remains unresolved. This is precisely the problem characterized by C1/C2/C3—RL cannot manufacture reliable rewards; it merely moves evaluator-quality issues from the write gate into the reward function.

Relationship to the Core Work. Both are ACL 2026 main-track long papers, addressing two sides of the same problem: the core work says signal quality is the bottleneck, while Memory-R1 says policy form is the bottleneck. They never meet in a shared results table—the most conspicuous experimental gap left by this ACL.

Transfer of Key Designs (in Reverse). Memory-R1's {ADD, UPDATE, DELETE, NOOP} and the core paper's {π, φ} are two formulations of the same thing: π corresponds to ADD/NOOP, φ to DELETE, leaving only UPDATE absent. The core paper explicitly excludes UPDATE under structural transformation / merging, while Supersede subsequently demonstrates that UPDATE is precisely the true failure mode: on LongMemEval's knowledge-update subset, GPT-5.4 drops from 92% with full context to 77% with bounded self-maintained memory (p = 0.0033).

3.1 Evolution Tree

3.2 The Internal Logic of Evolution

The driving force is not better retrieval, but the progressive internalization of responsibility:

4.1 Theory

1. Predictability of the zero-net-gain threshold. Model the evaluator as a noisy channel with false-positive rate ε₊ and false-negative rate ε₋ ; given experience-following strength ρ(the slope regressing output similarity on input similarity), derive the dPerf/dt > 0 interval's (ε₊, ε₋, ρ) feasible region. Even the first-order question of whether the zero crossing depends only on ε₊ and not ε₋ remains unanswered—High value, low barrier(requiring only a controlled synthetic environment and a noise-injectable evaluator).
2. The compounding rate of error propagation. Figure 4 only describes a widening gap, without its functional form. Measure the expected amplification factor at round k on RegAgent, λ(k) = E[err_k]/E[err_{k−1}] and determine whether it diverges geometrically, sublinearly, or converges to a fixed point. This directly determines how often memory must be audited.
3. Φ 's credit-assignment bias.φ_hist A record's value is the average downstream score after retrieval, but top-k means k records jointly contribute to each task, so equal credit is an approximation. Use Shapley values or leave-one-out estimates for true marginal contributions and quantify ranking errors from equal credit. The paper's top-k ranges from 1 for AgentDriver to 6 for RegAgent, a natural gradient for testing whether bias grows with k.

4.2 Engineering and applications

4. Decisive experiment: evaluator quality or policy learning—which is the bottleneck? The core paper and Memory-R1 are both ACL 2026 main-track long papers but neither cite each other nor share a benchmark. Plug Memory-R1's RL manager into the four agents and run {C1, C2, C3, Strict} × {handcrafted rules, RL policy} as a 4×2 matrix. If RL at C1 matches handcrafted rules at C3, policy form is the main bottleneck; otherwise, signal quality is. The missing head-to-head needed to decide whether A beats B is the key next gap, rather than merely an interesting new direction.
5. The missing memory-size control. Randomly downsample Add-all to Strict's memory size, such as 4100 → 2938, and run identical retrieval. This single comparison decides quality-versus-scale attribution at very low cost: no agent rerun, only retrieval and inference.
6. Sensitivity curves for α / β. Sweep on EHRAgent β ∈ [0.2, 0.8], α ∈ {0, 1, 2, 5} and plot the memory-size–accuracy Pareto frontier. Combined achieves 42.34 ACC with 248 memories; the question is whether this point lies on the frontier and where that frontier bends.
7. Add UPDATE back. Supersede shows information replacement is the main bounded-memory failure mode (GPT-5.4 92% → 77%, p=0.0033; expanding conversations 24× yields 68% → 28%, with proportional memory expansion giving no recovery, 28% → 28%). Add a third operator ψ(in-place update) to form {π, ψ, φ}, testing whether updating offers a third path on RegAgent, where deletion harms performance.

4.3 Emerging directions

8. Bootstrapping the evaluator. C3 already results from fine-tuning on 300 correctly judged examples. Since Φ provides free downstream-utility labels, use Φ to train the evaluator in return, forming an evaluator → memory quality → downstream utility → evaluator bootstrap loop. Test its stability, since it too may self-poison. This is the most direct route from diagnosis to a self-healing system.
9. Reversible forgetting / tiered cold storage. Combined gives −4.37 pp on RegAgent but +3.67 pp on EHRAgent: its cost is highly task-dependent. Relax φ from deletion to demotion into a cold tier, excluded from top-k but retrievable in a second pass after failure. Test whether RegAgent's −4.37 pp can be recovered. This directly challenges irreversible deletion.
10. Propagation dynamics in multi-agent shared memory. Experience-following already compounds errors inside one agent; in shared-memory multi-agent systems (Collaborative Memory), a bad record can contaminate N agents at once. Transfer the error-propagation experiment to an N-agent shared pool, measure whether contamination speed scales linearly with N, and test whether access control can serve as a distributed π.
11. Adversarial perspective. If the write gate is the system's key control point, it is also an attack surface (Forged Reasoning Attacks). Treat C1/C2/C3 as defense-strength levels and measure the malicious-record fraction needed to push performance below Fixed at each level, yielding a comparable memory-system robustness budget.

Memory is Reconstructed, Not Retrieved: Graph Memory for LLM Agents (2026-06, arXiv)

Replaces one-shot retrieve-then-reason with iterative reconstruction over a Cue–Tag–Content graph, dynamically adapting retrieval paths to intermediate evidence discovered during reasoning. Up to +23% on LoCoMo with fewer tokens and lower latency. It improves an axis orthogonal to the core paper—retrieval versus writing—and is worth reading alongside it.
arXiv ｜ Code

Supersede: Diagnosing and Training the Memory-Update Gap in LLM Agents (2026-06, arXiv)

The only paper explicitly showing that increasing memory budgets cannot fix update failures (28%→28% at 24× conversation scale, n=25), and one of the few memory papers here reporting significance (p=0.0033). It fills the UPDATE gap excluded by the core paper.
arXiv ｜ Code ｜ Model

Memory as a Controlled Process: Learned Adaptive Memory Management (MemCon) (2026-07, arXiv)

UCLA and collaborators formalize memory operations as a Memory MDP and learn policies with a tabular online bandit using UCB exploration, warm-start priors, and backward-discounted credit assignment. No pretraining or additional LLM calls, and entirely backend-independent. Up to +15.2 points across 6 benchmarks × 3 frameworks, saving 5–20% of tokens. Among lightweight learned-management approaches, it is particularly engineering-friendly.
arXiv ｜ Code

Memory-R1 (2026-07, ACL 2026 Main-Track Long Paper)

At the same conference as the core paper. Only 152 training QA examples teach 3B–14B models ADD/UPDATE/DELETE/NOOP and outperform heuristics. Read them together to compare their different explanations of the same phenomenon.
ACL Anthology

CMI-Mem: Toward Generalizable Long-Term Memory Management via CMI-Augmented RL (2026-07, arXiv)

Uses conditional mutual information to enhance RL for memory management, directly aligned with §4.1's question of whether signal quality can be characterized information-theoretically. Note: only the title and link accessibility were verified here; no page-by-page close reading, so assessment is brief.
arXiv

Your Agent's Memories Are Not Its Own: Forged Reasoning Attacks on LLM Agent Memory and Defenses (2026-07, arXiv)

Memory security. If experience-following makes agents faithfully reproduce retrieved records, memory poisoning should theoretically be highly efficient. This paper supplies attacks and defenses. Note: only the title and links were verified here; not read closely.
arXiv

Collaborative Memory: Multi-User Memory Sharing with Dynamic Access Control (2025-05, arXiv)

Dynamic access control for multi-user shared memory. Together with the core paper's error propagation, it forms the two pieces of §4.3's contamination-dynamics question.
arXiv

Memory in the Age of AI Agents: A Survey (arXiv 2512.13564) and Its Companion Paper List

A survey by 46 authors and its continuously updated paper list, currently the broadest search entry point for this field.
Paper List

Items not found or not verified are explicitly marked in the table. The core paper's official code repository could not be located: neither ACL Anthology nor the arXiv abstract page provided a link. Implementation details beyond Tables 1 / 2, such as exact α, β, and n, therefore could not be independently verified.