A simple external memory lets frozen AI agents learn from outcome verdicts and corrections, raising single-trial success on a banking task suite by up to 2.6× and converting many previously unsolved tasks; the accumulated rule store even transfers between different models.

Main Finding

Pairing a frozen-weight agent with an external, curated natural-language memory (Spark) and writing compact, retrievable “insights” after episodes enables meaningful continual learning without any weight updates. On a hard, knowledge-intensive banking benchmark (τ-bench), simple feedback signals—(a) a one-bit outcome verdict and (b) an after-the-fact verified correction—raise single-trial task success substantially on top of a strong static-RAG baseline, and the accumulated memory transfers between different models.

Key quantitative outcomes reported: - One-bit verdict (experience): single-trial (pass^1) success ≈ 1.6× the static-RAG baseline; banked solutions held across trials at rate ≈ 0.88. - After-the-fact corrections (instruction): pass^1 ≈ 2.6× the baseline; converted 22 of the 84 “floor” tasks the baseline never solved. - Replication on a stronger model (Claude Sonnet 5): corrections raised pass^1 from 0.248 → 0.397. - Cross-model frozen-store transfer: Mistral reading Sonnet-built store: pass^1 +0.224 vs its no-memory baseline; Sonnet reading Mistral store: +0.066. Conditional conversion rate on transferable tasks ≈ 31%.

Key Points

• Problem: deployed agents usually use frozen weights and miss nearly all learning opportunities produced during operation; retraining is costly and risky.
• Proposal: use an external shared memory service (Spark) that distils each episode into a compact natural-language insight (WHEN–THEN rule + retrieval query + metadata) during a post-episode reflection turn.
• Two feedback grades tested:
  • Experience: only a binary outcome verdict (success/failure); agent must infer lessons (only avoidance info on failures).
  • Instruction: verdict + the verified action sequence for failed episodes (the correct resolution provided once after failure).
• Baseline: static RAG agent retrieving over the full corpus of ~700 policy documents (BM25).
• Memory operations are agent-driven (search, create, enrich, feedback); writes only occur in a reflection turn after episode end and are curated automatically (filters, deduplication, validators).
• Protocol: τ-bench banking domain, 97 deterministic tasks, k=4 trials per task; trial-major scheduling; memory store starts empty and persists across a task’s trials so learning can accumulate and be measured.
• Metrics: pass^k (benchmark’s strict all-k-trials pass), per-trial pooled success curve, hold rate (probability that a pass at t holds at t+1), conversion of “floor” tasks.
• Statistical treatment: paired contrasts per task, cluster bootstrap for interval estimation.
• Transferability: frozen stores (read-only) built by one model can improve another model’s performance above its own no-memory baseline, indicating organizational knowledge capture.

Data & Methods

• Benchmark: τ-bench banking domain (τ-Knowledge extension). Hard, knowledge-intensive: agents must discover business rules via retrieval over ~700 policy documents; deterministic programmatic evaluation (97 tasks, deterministic except 1).
• Models:
  • Mistral Large (open-weights; main experiments run under all three conditions).
  • Claude Sonnet 5 (frontier model; replication run on baseline and instruction).
  • Customer simulator: GPT-5.2 (per leaderboard config).
• Memory system: Spark (standalone server, accessed via Model Context Protocol). Unit = insight (title, content rule, retrieval query, optional tags). Operations exposed to agent: memory_search, memory_create, memory_enrich, memory_feedback.
• Memory write constraints:
  • Only one rule per episode, in WHEN–THEN form, with decision-critical values verbatim and validated.
  • On successes: store decisive state-changing actions.
  • On failures (experience): only avoidance signals; on failures (instruction): the verified action sequence + diff provided to guide creation of a correct rule.
  • Writes mid-conversation are ignored (prevents poisoning).
• Experiment 1 (continual learning):
  • Conditions: no-memory static RAG baseline; experience (verdict-only learning); instruction (verdict + correction).
  • Each task: 4 trials; trial-major schedule across tasks; fresh store per condition; store starts empty and persists across the four trials of each task.
  • Primary measures: pass^k, per-trial success curve, hold rate; stratum analysis (floor tasks = tasks baseline never solves).
• Experiment 2 (cross-model transfer):
  • Take frozen stores produced by instruction runs of each model; have the other model read (search-only, read-only) that frozen store and measure performance vs its own no-memory baseline.
  • Define transfer stratum = tasks the producer solved at least once and the consumer’s no-memory baseline never solved.
• Statistical methods: paired per-task contrasts, cluster bootstrap for confidence intervals; full harness and data released.

Implications for AI Economics

• Cost-effective continual learning alternative: The approach yields large performance gains without any model retraining, avoiding capital and operational expenditures associated with iterative fine-tuning (GPU training costs, engineering effort, evaluation/safety cycles). Organizations can extract value from deployment interactions at much lower marginal cost.
• Reduced human remediation and escalation load: Higher single-trial success and conversion of previously unsolvable tasks imply fewer human interventions, fewer escalations, and lower handling time for edge cases. Even modest absolute improvements in pass rates can translate to outsized labor savings in customer-service and compliance-sensitive workflows.
• Faster feedback-to-value loop: Storing verified corrections and retrievaling them on demand creates near-immediate improvements across subsequent interactions. This shortens the learning cycle relative to batch retraining and enables continuous improvement in live systems.
• Data sovereignty and deployability: The method works on open-weights models (Mistral) and frontier models (Claude), and memory is a separable asset. Organizations with data-sovereignty constraints can self-host models yet still gain continual learning benefits by operating a shared memory service.
• Transferable organizational knowledge: Frozen, read-only stores built by one model improve other models’ performance meaningfully—this enables cross-system knowledge sharing without weight-sharing or model retraining. It supports modular organizational architectures where a central knowledge store augments multiple deployed agents.
• Product & market opportunities:
  • Memory-as-a-service (hosted or on-prem) offerings that provide curated, validated insight stores and lifecycle management.
  • Tooling for capturing verified corrections (ticketing integrations, expert-annotation pipelines) becomes high ROI because one correction can be banked and reused.
  • New commercial metrics: hold rate and conversion of floor tasks become directly monetizable KPIs (reduction in repeated tickets, compliance failures).
• Risks and limits affecting economic calculations:
  • This study assumes perfect, deterministic feedback (benchmark evaluator). Real-world feedback is noisy; value will depend on feedback quality and curation overhead needed to maintain precision (filtering, deduplication, human-in-the-loop validation).
  • Instruction-grade feedback (verified action sequences) yields far larger gains than binary verdicts, but those corrections are more expensive to obtain; organizations must weigh the marginal cost of supplying corrections versus expected benefit.
  • Memory transfer can encode producer-model stylistic shortcuts; naïve transfer may sometimes hurt—practical deployments need curation or contrastive distillation to maximize portability.
• Practical recommendations for adopters:
  • Start with a read/write shared memory layer for agents; instrument reflection turns to capture verified fixes and the minimal WHEN–THEN rule form used here.
  • Prioritize collecting instruction-grade corrections for high-impact or frequently failing tasks; use binary-outcome learning where correction cost is prohibitive.
  • Measure pass^k, per-trial success, and hold rate to quantify operational gains; monitor noise and design lightweight curation/validation.
  • Use frozen, read-only stores to transfer organizational knowledge across models and deployments to amortize the cost of corrections.
  • Model choice: open-weights models + on-prem memory can achieve much of the benefit, preserving data sovereignty and lowering licensing costs.

Limitations to keep in mind - Experiments run on a deterministic benchmark with perfect feedback; real deployment environments will face noise, partial feedback, and adversarial inputs. - Instruction signals (verified action sequences) require human/expert labor to produce in practice; the economic trade-offs depend on the cost of obtaining those corrections. - The memory lifecycle here had no human curation; larger-scale/longer-lived stores may need human oversight or stronger automated synthesis/deduplication to prevent degradation.

Overall, the paper documents a practical, low-cost route to continual improvement for deployed agents that shifts learning from weight-space updates to a sharable external memory asset—an approach with clear operational and economic upside, especially for organizations that prefer to avoid frequent retraining or must self-host models.