A shared, selective persistent memory for agentic LLMs substantially raises successful task completion while cutting token and runtime costs: selective memory hits 96% completion versus 79% (no memory) and 71% (full history), and a zero-token data-refresh mechanism yields a 14× reduction in task time and roughly 97× lower per-invocation token cost.

Shared Selective Persistent Memory for Agentic LLM Systems

Sanjana Pedada, Aditya Dhavala, Neelraj Patil · July 10, 2026

arxiv quasi_experimental medium evidence 7/10 relevance Full text usable extracted full text Source PDF

Selective shared persistent memory for agentic LLMs improves task completion (96% vs 79% without memory and 71% with full history), drastically reduces token costs via summary-driven generation (≈97x) and eliminates repeated model invocations with zero-token data refresh (14x task-time reduction).

Agentic LLM systems that generate code through multi-turn tool use face a fundamental context problem: each session starts from zero, discarding the configuration choices, domain constraints, data schemas, and tool-use patterns that made previous sessions productive. Naively persisting entire conversation histories is token-inefficient and counterproductive: irrelevant context degrades generation quality. We introduce shared selective persistent memory, an architecture that identifies and retains four categories of reusable context (task specifications, data schemas, tool configurations, and output constraints) while discarding session-specific reasoning traces. Crucially, this memory is shared: workspaces encapsulating selective memory can be transferred across users with role-based access control, enabling collaborative reuse without redundant specification. We implement it in a deployed collaborative workspace platform where LLM agents produce, edit, and maintain git-versioned artifacts (dashboards, reports, and data-driven documents) from heterogeneous sources (CSV, SQL, REST APIs, and MCP servers). A complementary zero-token data refresh mechanism decouples generated programs from runtime data, enabling artifact reuse without re-invocation. Across three enterprise scenarios, shared selective persistent memory achieves 96% task completion (vs. 79% without memory and 71% with full history). Zero-token refresh eliminates LLM re-invocation for recurring updates (14x task-time reduction), while summary-driven generation cuts per-invocation token cost by 97x versus raw data injection. A replication on four public datasets confirms generalizability, with zero-token refresh succeeding in 12/12 trials. Notably, naive full-history persistence actively degrades completion by biasing the agent with stale traces, while selective memory outperforms both extremes.

Summary

Main Finding

Persisting only structured, reusable context across agentic LLM sessions — specifically task specifications, data schemas, tool configurations, and output constraints — and discarding session-specific reasoning traces yields large gains in task success, token efficiency, latency, and collaboration. A deployed workspace implementing this "shared selective persistent memory" plus a zero‑token data refresh mechanism outperforms both (a) no memory and (b) naive full-history persistence, which can actively harm performance.

Key Points

Memory decomposition: persist four compact, stable categories:
- Task specifications (custom system prompts / domain rules).
- Data schemas (column types, distributions, value catalogs, samples).
- Tool configurations (connectors, parameter schemas, auth).
- Output constraints (runtime contracts, e.g., data-injection rule).
Selective forgetting: never persist multi-turn reasoning traces, intermediate files, tool invocation logs, error-recovery traces, or raw data.
Memory composition: at session start the system builds the prompt from sbase ⊕ Mtask ⊕ Mtools ⊕ Moutput and injects Mdata with the user query.
Zero‑token data refresh: generated artifacts must read data via a runtime injection point (no hardcoded values). If schema-compatible, data updates require zero LLM invocations.
Sharing and collaboration: workspaces encapsulate selective memory and versioned artifacts (git). Workspaces can be shared with role-based access (owners, stewards, viewers); collaborators can swap schema-compatible data and reuse artifacts immediately.
Negative baseline: persisting full conversation histories increases input tokens dramatically and can reduce task completion by biasing agents with stale reasoning traces.

Data & Methods

Implementation:
- Deployed collaborative workspace: FastAPI backend, multi-connector data layer (CSV, SQL, REST, MCP), agentic engine using Claude Opus 4 (primary) and Claude Sonnet 4 (subtasks), MongoDB for metadata, git for artifact/version control, draft isolation and undo stack.
- Data profiling produces compact schema summaries (~500 tokens) instead of raw data injection (≈100× token reduction).
Experimental design:
- Corpus: 24 real enterprise data files (200–45K rows, 8–42 columns) across supply chain, sales, process metrics.
- Three use-case families: recurring refresh (n=10), cross-team adaptation (n=8), iterative construction (n=6).
- Two task types: initial generation and recurring generation.
Ablation (enterprise, 24 tasks):
- Conditions: No memory, Full history (complete prior transcript injected), Shared selective memory.
- Metrics: completion rate (raters on render correctness, data fidelity, format compliance, completeness), turns per task, input/output tokens, wall time.
- Results (means):
  - Completion: selective 96% vs no memory 79% vs full history 71%.
  - User turns: selective 1.4 vs no memory 4.3 vs full history 3.1.
  - Time: selective 68s vs no memory 285s vs full history 310s.
  - Input tokens: selective ≈3.4K vs no memory ≈2.1K vs full history ≈18.7K (full history ≈9× more than no memory).
  - Zero-token refresh: for 18/24 tasks with compatible schemas, refresh succeeded with zero LLM invocations.
- Statistical tests: selective vs no-memory p = 0.046; selective vs full-history p = 0.008.
Public replication (4 datasets × 3 runs = 12 trials):
- Datasets: Superstore Sales, UCI Adult Income, NYC 311, World Bank GDP.
- Selective memory achieved 100% completion and zero LLM tokens across all 12 selective-memory trials (i.e., successful zero-token refresh when schemas compatible).
Efficiency claims:
- Summary-driven generation reduces per-invocation token cost by ~97× vs raw data injection.
- Zero-token data refresh yields up to 14× task time reduction by eliminating re-invocation for recurring updates.
Failure modes observed:
- One selective-memory failure: missed cross-file join semantics not captured by schema summary.
- Full-history failures dominated by trace anchoring (LLM biased toward prior solution paths).

Implications for AI Economics

Cost savings (compute/token): selective persistence plus zero-token refresh materially reduces LLM calls and token consumption:
- Fewer model invocations for recurring tasks => lower per-task compute bills.
- Dramatic token reduction (summary-driven ≈97× lower than raw injection) reduces marginal costs where token-pricing is significant.
- Avoiding full-history injection prevents consumption of large context budgets that produce little value, reducing wasted spending.
Productivity and labor economics:
- Time per recurring task shrank (reported ~4×–14× reductions), increasing analyst throughput; recurrent reporting and dashboard maintenance become cheaper and faster.
- Reusable workspace templates can scale expertise: one expert can encode task specs and output constraints that many users reuse, lowering coordination and specification costs across teams.
- Lower friction for repetitive, structured tasks shifts labor away from re-specification/boilerplate work toward higher-level analysis/interpretation.
Assetization and market effects:
- Workspaces with curated selective memory are reusable digital assets that firms can version, share, and potentially monetize (internal templates, vendor-provided templates).
- Reduced marginal cost of producing updated artifacts (dashboards, reports) could increase the frequency of automated analytics, changing demand for downstream decision processes.
Risk management and governance:
- Selective memory reduces risk from stale or misleading reasoning traces; versioning + role-based access supports audits and rollback—important for compliance-sensitive domains.
- However, reliance on schema-compatibility and runtime injection contracts creates operational constraints; schema drift requires supervised regeneration, which must be managed as an operational cost.
Limitations and open economic questions:
- Dollarized cost-benefit: paper reports percentage/token/time gains but not absolute $ savings across different pricing models — further work to map token/time reductions to monetary savings under various cloud/LLM pricing is needed.
- Generalizability to other LLM providers and domains (beyond dashboard/report generation) needs broader evaluation.
- Market externalities: easier reuse of memory/templates may increase vendor lock-in if templates are provider-specific, or conversely create a marketplace for portable workspace templates.
- Labor displacement vs. augmentation: while repetitive tasks are automated, demand may grow for designers of high-quality task specifications and shared workspaces.
Practical advice for adopters:
- Persist structured configuration (task specs, schemas, tool configs, output contracts) and explicitly forbid storing chain-of-thought or tool logs in shared memory.
- Enforce data-injection contracts in generated artifacts to enable zero‑token refresh and auditability.
- Use git-backed versioning and draft isolation to let teams explore changes without model re-invocation and to retain deterministic rollback points.

Overall, the paper shows a pragmatic, implementable strategy that reduces recurring LLM costs and time by converting ephemeral session knowledge into reusable, shareable, structured configuration assets while avoiding the harms of retaining full conversation histories.

Assessment

Paper Typequasi_experimental Evidence Strengthmedium — The paper reports clear, large differences in task completion and resource metrics under alternative memory policies and includes replication on public datasets, supporting internal validity of the system-level claims; however, it lacks details on randomization, sample sizes, statistical testing, possible tuning or calibration to the proposed method, and real-world outcome measures (productivity/wage impacts), limiting causal confidence and external validity. Methods Rigormedium — Strengths include a deployed collaborative workspace, multiple scenarios, heterogeneous data sources, and replication on public datasets; weaknesses are incomplete reporting on experimental design (how tasks were selected and assigned, number of trials per condition), missing information on LLM models and hyperparameters, lack of statistical inference and robustness checks, and potential tailoring of baselines to favor the proposed approach. SampleA deployed collaborative workspace platform where agentic LLMs generate and edit git-versioned artifacts (dashboards, reports, data-driven documents) from heterogeneous sources (CSV, SQL, REST APIs, MCP servers); evaluated on three enterprise scenarios and replicated on four public datasets (zero-token refresh tested in 12/12 trials); exact number of tasks/trials per scenario and models used are not fully specified in the summary. Themesproductivity human_ai_collab org_design IdentificationControlled system-level comparisons: the authors implement three memory treatments (no memory, full-history persistence, and selective shared persistent memory) and compare outcomes (task completion, token usage, task time) across three enterprise scenarios and a replication on four public datasets; identification relies on within-platform A/B-like comparisons rather than randomized field assignment or external instruments. GeneralizabilityEvaluated on a small number of enterprise scenarios which may not represent broader organizational workflows, Platform-specific implementation and engineering choices may not transfer to other agent frameworks, Performance likely depends on the underlying LLM family and prompting/hyperparameter choices which are not fully described, Task set focuses on code- and data-artifact generation (dashboards/reports) and may not generalize to other task types (freeform writing, complex planning), Metrics (task completion, token cost, task time) are system-level proxies and may not map directly to firm-level productivity or labor market outcomes, Shared-memory and access-control assumptions may not hold across organizations with different privacy/security constraints

Claims (8)

Claim	Direction	Outcome	Confidence & Evidence	Details
Shared selective persistent memory achieves 96% task completion (vs. 79% without memory and 71% with full history). Developer Productivity	positive	task completion rate	Reading fidelity high Study strength medium	96% task completion (vs. 79% without memory and 71% with full history) 0.48
Zero-token refresh eliminates LLM re-invocation for recurring updates, yielding a 14x task-time reduction. Task Completion Time	positive	task completion time (for recurring updates)	Reading fidelity high Study strength medium	14x task-time reduction 0.48
Summary-driven generation cuts per-invocation token cost by 97x versus raw data injection. Organizational Efficiency	positive	per-invocation token cost	Reading fidelity high Study strength medium	97x reduction in per-invocation token cost 0.48
A replication on four public datasets confirms generalizability, with zero-token refresh succeeding in 12/12 trials. Adoption Rate	positive	zero-token refresh success rate	Reading fidelity high Study strength medium	n=12 12/12 trials 0.48
Naive full-history persistence actively degrades task completion (by biasing the agent with stale traces) compared to no memory and selective memory. Developer Productivity	negative	task completion rate	Reading fidelity high Study strength medium	full history: 71% vs no memory: 79% (8 percentage points lower) 0.48
The shared selective persistent memory architecture identifies and retains four categories of reusable context: task specifications, data schemas, tool configurations, and output constraints, while discarding session-specific reasoning traces. Other	positive	types of context retained	Reading fidelity high Study strength low	not reported 0.24
The memory is shared across users: workspaces encapsulating selective memory can be transferred across users with role-based access control, enabling collaborative reuse without redundant specification. Other	positive	collaborative reuse (transferability of workspace memory with RBAC)	Reading fidelity high Study strength low	not reported 0.24
Naively persisting entire conversation histories is token-inefficient and counterproductive because irrelevant context degrades generation quality. Output Quality	negative	output generation quality / token efficiency	Reading fidelity high Study strength medium	not reported 0.48