Current agent 'memory' is mostly lookup, not learning—this matters: without slow weight-based consolidation, agents cannot develop expertise, face a provable ceiling on generalizing to novel compositions, and remain structurally vulnerable to persistent poisoning; pairing fast exemplar storage with slower consolidation is required.

Main Finding

Agentic memory systems used in deployed LLM agents (vector stores, RAG, scratchpads, scratch-streams, context-window engineering) are exemplar-based lookups — memos — not true memory. Treating retrieval-as-memo as equivalent to weight-based learning is a category error with provable consequences: retrieval-only agents cannot acquire rule-like, compositional expertise, their capability stagnates despite accumulating notes, and persistent external stores create a structural, growing security risk (persistent poisoning). The paper formalizes these limitations (a Compositional Generalization Gap theorem), grounds them in Complementary Learning Systems theory, surveys mechanistic and empirical support, and calls for hybrid architectures that consolidate experience into model weights.

Key Points

• Two distinct substrates and mechanisms
  • Change θ: parametric updates (pretrain/fine-tune/targeted edits) compress experience into weights and produce rule-like, generative generalization.
  • Change C: context / external store (RAG, memstreams, scratchpads) is exemplar-based lookup; it generalizes only by similarity to stored cases and is bounded by context and the base model’s frozen composition ability.
• Definitional claim: current agentic memory = exemplar lookup (episodic); the experiential/weight-based row (true, rule-like memory) is missing from deployed systems.
• Theorem (Compositional Sample Complexity Separation)
  • Formal setting: k base concepts F, composition operator ⊕ mapping pairs to outputs; both retrieval MR and parametric MP get the same n compositional examples.
  • Under a mild bounded in-context composition assumption (frozen model has accuracy ≤ ᾱ < 1 on unseen pairs given K exemplars), retrieval requires nR = Ω(k^2) stored compositional examples to reach a target compositional generalization, while parametric learning needs nP = O(d + log(1/δ))/δ (d = hypothesis class VC dimension).
  • Separation ratio nR/nP = Ω(k^2/d); for many structured operators this is large (Ω(k) to Ω(k^2)). The gap is independent of context-window size or retrieval engineering.
• Dynamic consequence — the "Frozen Novice" problem
  • Retrieval-only agents never change θ; every session starts from the same frozen model. Accumulating notes increases coverage but does not reorganize internal representations, so expertise (rule-like representations) does not emerge.
• Security consequence — persistent compromise
  • Prompt-injection or adversarial content written to an external store converts a transient attack into a persistent one: once injected, the content can be retrieved and reused in all future sessions. With repeated interactions, the probability of persistent compromise approaches 1.
• Empirical and mechanistic support
  • Mechanistic work (ROME, MEMIT, FFN fact memory studies) shows factual and rule-like knowledge can be localized and edited in weights.
  • Empirical studies: parametric storage/edits outperform retrieval for compositional and transfer tasks (cited Yao et al., Ovadia et al., Yang et al.). Benchmarks like SCAN and COGS show weight-based models outperform retrieval-only on systematic compositional splits.
• Practical tradeoffs acknowledged
  • Retrieval is reversible, auditable, cheap to deploy; parametric updates are compute- and design-intensive and carry different operational risks (forgetting, harder audit), but are necessary for true, generalizable learning.
• Recommendation: adopt co-existence architectures and invest in consolidation/fine-tuning, targeted editing, and benchmarks that measure compositional/generalization behavior over time.

Data & Methods

• Theoretical framework
  • Formalized compositional generalization problem: concepts F, composition operator ⊕, N = k choose 2 possible pairs, training dataset D of n compositional examples.
  • Defined Compositional Generalization Capacity CGC(M, D): expected accuracy on uniformly drawn pairs.
  • Assumption: frozen model’s in-context extrapolation accuracy on unseen pairs is bounded by ᾱ < 1 (holds for domain-specific or post-cutoff operators).
  • Proof sketch uses coverage vs. learning dichotomy: retrieval must store examples covering Ω(k^2) pairs for high CGC, parametric requires O(d) samples where d is hypothesis complexity; Fano’s inequality used in appendices to formalize limits on ᾱ.
• Security model
  • Simple probabilistic model: per-interaction injection success p0; cumulative interactions N(t). For persistent stored memos, P(compromise by t) = 1 − (1 − p0)^{N(t)} → 1 as N(t) grows.
• Empirical grounding (surveyed literature)
  • Mechanistic interpretability: ROME, MEMIT, studies of FFN units and fact memory localization (Meng et al., Yao et al., Geva et al., Ye et al.).
  • Empirical comparisons: Yao et al. (parametric storage vs external), Ovadia et al., Yang et al., SCAN and COGS compositional benchmarks.
  • Security experiments: MINJA, PoisonedRAG, InjecAgent showing high persistent injection success when writes to store are allowed.
• Scope and assumptions
  • The main results target domains where composition rules are domain-specific and underrepresented in pretraining (the regime where persistent agents are most useful).
  • Does not deny that retrieval helps many tasks (rare-entity recall, immediate context-sensitive tasks), but shows it cannot replace parametric consolidation for compositional generalization and long-term expertise.

Implications for AI Economics

• Cost-efficiency and scaling
  • Economic inefficiency of retrieval-only scaling: to match parametric generalization via retrieval requires storing and retrieving Ω(k^2) examples (storage, indexing, retrieval latency and compute), whereas parametric consolidation scales with hypothesis complexity d. For large k, retrieval becomes dramatically more expensive per unit of generalization.
  • Investment implication: firms relying purely on retrieval will face diminishing returns; capital should shift toward methods that consolidate important experience into weights (targeted fine-tuning, parameter-efficient tuning, editing).
• Product and market design
  • Market demand for continual learning, safe consolidation, and targeted editing tools will rise. Startups and incumbents that can deliver auditable, controllable consolidation primitives (e.g., targeted weight edits, safe offline fine-tuning, provenance-preserving consolidation) gain strategic advantage.
  • New service tiers/products: audited persistent agents that consolidate vetted experiences for improved generalization vs. “memo-only” agents prioritizing auditability and reversibility but with bounded capability.
• Risk, liability, and insurance
  • Persistent-memory agents create growing security externalities. Expected loss from attacks increases over time (compounding risk). This affects SLAs, pricing, and cyber-insurance premiums; enterprises should internalize higher operational costs for monitoring, validation, and remediation.
  • Procurement and regulation: buyers should require threat models and mitigation for persistent memory (write-controls, provenance, sanitization, kill-switches) and possibly mandate bounds on persistence for high-risk domains.
• Benchmarking and procurement incentives
  • Benchmarks and procurement specifications should measure compositional generalization and long-term learning, not just retrieval/seen-distribution performance. Current procurement that rewards only immediate retrieval accuracy misaligns incentives toward memo accumulation over genuine capability growth.
• Tradeoffs to manage
  • Retrieval benefits: cheaper, reversible, auditable, easy to patch; good for content where provenance matters or where changing θ is risky/expensive.
  • Parametric consolidation costs: compute, risk of catastrophic forgetting or opaque changes, and harder audit trails — but essential for scalable expertise.
  • Economic strategy: adopt hybrid/co-existence architectures that use retrieval for ephemeral data and parametric consolidation for distilled, validated rules/skills. Policies and tooling should balance auditability, update costs, and security.
• Actionable recommendations for economic actors
  • System builders: invest in consolidation pipelines (offline distillation, targeted edits) and design safe write/validation workflows for what gets consolidated.
  • Benchmark designers and procurement: include compositional-generalization and transfer splits; require evidence of consolidation and safe update procedures.
  • Investors and product managers: fund continual learning and targeted editing capabilities; evaluate tradeoffs between deployment speed (retrieval) and long-term capability growth (consolidation).
  • Insurers/regulators: model persistent-agent risk in underwriting and consider rules for minimization of persistent attack surfaces in high-stakes deployments.

Overall, from an AI-economics perspective, the paper implies a reallocation of technical and financial effort away from unbounded memo accumulation toward methods that safely and efficiently convert high-value, validated episodic experience into parametric knowledge — because that is the only scalable and economically sustainable path to lasting agent expertise and manageable security risk.