0 cumulative citations
View corpus contextSelf-evolving financial agents can improve accuracy while widening their attack surface: in a simulated e-banking audit, SkillOpt and similar systems increased successful task completion but also raised exposure to injected instructions and unauthorized account changes, and an incompatible execution interface magnified apparent regressions.
Auditing Self-Evolution in Financial Agents: Capability Gains, Security Drift, and Execution-Interface Mismatch
arxiv
quasi_experimental
medium evidence
8/10 relevance
Full text
usable extracted full text
Source
PDF
In a controlled AgentDojo banking audit, self-evolving agent systems often raise task accuracy but can simultaneously increase exposure to injected malicious content and unauthorized financial state changes, with effects varying by evolution method and sometimes driven by execution-interface incompatibilities.
Citation observations
Cumulative provider counts captured on specific dates; providers are never combined.
Self-evolving agents turn experience into reusable skills, workflows, or memories, but post-evolution accuracy alone does not show whether learned behavior preserves previously correct behavior or security. We audit SkillOpt, Agent Workflow Memory (AWM), and ReasoningBank in simulated e-banking using matched benign acquisition trajectories, sealed evaluation endpoints, execution-grounded checks, and independent state replay. On Qwen 3.7 Flash, SkillOpt raises benign utility from 0.741 to 0.837 while exposure to injected content rises from 0.820 to 0.943. Conditional attack success after exposure falls from 0.605 to 0.562, yet overall attack success rate (ASR) rises from 0.496 to 0.530 and unauthorized financial state changes rise to 0.685. Across three independently evolved lineages, capability, exposure, and unauthorized-state changes increase in all three, whereas ASR increases in only two. ReasoningBank raises utility to 0.859 without increasing aggregate ASR, although unauthorized state changes remain slightly above Static. AWM reveals a separate evaluation hazard: a literal WebArena text-action envelope disrupts tool execution in our native function-calling executor. In a post-hoc sensitivity test, removing only that envelope restores utility from 0.319 to 0.756, while exposure rises from 0.299 to 0.909 and ASR from 0.195 to 0.575. Auditing self-evolving financial agents therefore requires tracking regressions, attack-surface contact, unauthorized financial-state change, and artifact-executor compatibility, not accuracy alone.
Summary
Main Finding
Self-evolution in LLM-based financial agents can improve task-level capability while simultaneously increasing the agent’s contact with attacker-controlled content and the incidence of unauthorized financial-state changes. Accuracy alone hides two distinct failure modes: (1) regressions on previously-correct cases, and (2) an enlarged attack surface that raises exposure even when conditional susceptibility falls. Evaluation must therefore report regressions, exposure, conditional attack success, and execution-grounded financial harm — not just aggregate accuracy.
Key Points
- Audit scope: three external-state self-evolution systems (SkillOpt, Agent Workflow Memory / AWM, ReasoningBank) were compared end-to-end in a simulated e-banking suite (AgentDojo Banking), using byte-identical benign acquisition trajectories, sealed evaluation endpoints, three independently evolved lineages, and a frozen executor.
- Paired capability changes: the study reports W→C (wrong→correct) gains and C→W (correct→wrong) regressions paired against a Static baseline so regressions remain visible.
- Decomposed security metric: attack success rate (ASR) was decomposed into Pr(exposure) × Pr(success | exposure). Empirically, no unexposed episode succeeded, so ASR factorised exactly into exposure × conditional ASR here.
- SkillOpt (on qwen3.7-flash):
- Benign utility: 0.837 (Static 0.741, +9.6 percentage points)
- W→C = 23; C→W = 10 (net +13)
- Exposure: 0.943 (Static 0.820; +12.3 pp)
- Conditional ASR: 0.562 (Static 0.605; −4.3 pp)
- ASR: 0.530 (Static 0.496; +3.4 pp)
- Unauthorized financial-state change: 0.685 (Static 0.583; +10.2 pp)
- Interpretation: capability gains coincided with a uniformly larger attack surface; conditional susceptibility fell modestly, but total ASR and unauthorized-state changes rose because exposure increased.
- ReasoningBank:
- Utility: 0.859 (+11.9 pp)
- W→C = 25; C→W = 9
- Exposure: 0.802 (below Static)
- ASR: 0.474 (below Static 0.496)
- Unauthorized-state change: 0.595 (slightly above Static)
- Interpretation: capability gain without aggregate ASR increase — security outcomes are method-dependent.
- AWM revealed an important evaluation confound: a literal upstream textual-action envelope (WebArena-style) was incompatible with the experiment’s native function-calling executor, producing severe performance degradation (utility 0.319). A post-hoc interface adaptation that removed/changed only that envelope restored utility to 0.756 and raised exposure and ASR substantially, showing that executor–artifact interface mismatches can mask true behavior.
- Reproducibility across lineages:
- SkillOpt’s increases in capability, exposure, and unauthorized-state changes held in all three independently-evolved lineages.
- SkillOpt’s total ASR increase did not reproduce consistently (range across lineages spanned negative to positive); thus ASR can be the least consistent quantity across independent evolutions.
- Practical observation: every exposure was necessary for attack success in these experiments (0/579 unexposed successes), so controlling exposure materially affects ASR.
Data & Methods
- Environment and benchmark:
- AgentDojo Banking (pinned release) was audited; 4 of 16 official utility predicates failed initial checks and were repaired; primary results use a corrected 15-family suite. A 12-family untouched-checker subset was used for sensitivity analysis.
- Each family has six parameterised variants (v0–v5). Acquisition used v0/v1 (seen), evaluation used sealed v3/v4/v5; v2 reserved for SkillOpt validation gate.
- Evolution protocol:
- Three independently-evolved lineages. Within each lineage, the same byte-identical acquisition trajectories (11 seen families × {v0,v1} = 22 trajectories) were fed to each of the three systems so method differences are not confounded with acquisition evidence.
- One offline evolution pass per lineage; evolved artifacts were frozen before any evaluation.
- Ported systems (no change to core update/retrieval rules):
- SkillOpt: produces/edits a natural-language skill document, uses a validation gate (v2) to accept deployment.
- AWM: induces reusable workflows from successful trajectories and inserts the workflow textually.
- ReasoningBank: distils success/failure memory items and retrieves the top-1 result for insertion.
- Executor and models:
- Primary executor: qwen3.7-flash-2026-07-15, temperature 0.0 for evaluation (thinking disabled), using only local benchmark tools.
- Same model (qwen3.7-flash) used as optimiser/inducer at temperature 1.0, and qwen3.7-text-embedding used for ReasoningBank retrieval.
- Metrics:
- Benign utility: corrected checker verdict.
- W→C / C→W paired transitions versus Static baseline.
- ASR: fraction of attacked episodes achieving injected goal.
- Exposure: fraction where injected content entered model-visible interaction.
- Conditional ASR: Pr(success | exposure).
- Unauthorized state change: replay-based comparison between observed terminal state and authorised post-state, covering transfers, recipient/amount changes, payment modifications, or credential changes.
- Attacks: AgentDojo’s important_instructions injection; attacker controlled only untrusted text returned by banking tools. No defence was active.
- Sample sizes per condition: benign n = 135 (15 families × 3 variants × 3 lineages); attacked n = 405 (15 families × 9 goals × v3 × 3 lineages).
Implications for AI Economics
- Risk externalities and hidden trade-offs:
- Improvements in task performance can raise an agent’s exposure to untrusted inputs, increasing systemic risk even when conditional compliance falls. Economically, this creates negative externalities not captured by headline accuracy metrics: higher exposure may increase fraud risk, remediation costs, and reputational losses.
- Measurement and disclosure standards:
- Regulators, auditors, and purchasers should require disclosure of not only aggregate accuracy but also paired regressions (C→W), exposure rates, conditional susceptibility, and execution-grounded harm (unauthorized state changes). Standardized audit protocols (sealed endpoints, matched acquisition, frozen artifacts) will improve comparability.
- Product design and incentives:
- Designers optimizing for aggregate task score may unintentionally enlarge the attack surface. Procurement contracts and product specifications should align incentives to penalize increased exposure or unauthorized state changes (e.g., via uptime/compromise clauses, SLAs tied to security metrics).
- Liability, insurance, and pricing of risk:
- The decoupling between utility gains and security drift complicates liability allocation. Insurers and risk managers should price exposure-related risk, and insurers may require provenance/audit evidence demonstrating bounded exposure and interface compatibility.
- Market differentiation and certification:
- Methods can differ in security outcomes (ReasoningBank improved utility without increasing ASR). Firms can differentiate by certifying lower exposure/unauthorized-change rates through standardized audits, which could become a competitive advantage or basis for certification labels.
- Operational and compliance costs:
- Deploying self-evolving agents safely requires additional monitoring, sealed-evaluation pipelines, regression tracking, and executor compatibility testing. These raise deployment costs and may favor centralized/cloud-managed solutions with audited update processes or slower update cadences.
- Research & policy priorities:
- Policies should mandate reporting of evolution lineage variability (independent evolution runs) because some harms (e.g., ASR) were not consistent across lineages.
- Support research into defensive design that reduces exposure expansion (e.g., selective retrieval gates, provenance-aware retrieval, task shields) and standardized benchmarks that include execution-grounded state change outcomes.
- Limits & economic interpretation caution:
- Results come from a simulated e-banking benchmark with no observed monetary losses inferred; external validity to real-world deployments depends on environment/tooling differences, attacker sophistication, and institutional mitigations. Nevertheless, the mechanisms (exposure-driven drift, interface mismatch) generalize as economic risks that stakeholders should quantify and mitigate.
If you want, I can: - Produce a one-page checklist for procurement and audit teams that translates these metrics into procurement requirements and contract language, or - Draft a short policy brief (1–2 pages) recommending minimum audit standards for self-evolving financial agents.
Assessment
Paper Typequasi_experimental
Evidence Strengthmedium — The paper uses a careful, tightly controlled simulation audit with matched acquisition, sealed endpoints, replay checks, and three independent evolution lineages, which gives credible within-experiment causal leverage on how self-evolution changes agent behaviour and security in this environment. However, evidence is limited to a single simulated benchmark (AgentDojo Banking, corrected 15-family suite), a single primary model/executor configuration (Qwen 3.7 flash), three lineages and one offline evolution pass, and some post-hoc sensitivity arms; these limit external validity and preclude strong field-general causal claims.
Methods Rigorhigh — Strengths include pre-audit and repair of utility predicates, sealed evaluation endpoints, byte-identical matched acquisition across methods within lineages, paired comparisons to Static on identical test cells, decomposition of attack success into exposure and conditional susceptibility, execution-grounded replay to detect unauthorized state changes, and multiple independent lineages; weaknesses include absence of randomization, relatively small number of task families and lineages, reliance on a single primary model/executor, and some reliance on post-hoc interface sensitivity tests.
SampleAgentDojo Banking v1.2.2 (pinned) after pre-model-checker repairs yielding a 15-family suite; each family has six parameterized variants (v0–v5). For each of three independently evolved lineages: 11 'seen' families × {v0,v1} = 22 acquisition trajectories (byte-identical across the three methods within a lineage) are used to evolve artifacts for SkillOpt (skill document), AWM (workflow), and ReasoningBank (memories); endpoints v3/v4/v5 are sealed until artifacts are frozen. Benign evaluation: 15 families × 3 variants × 3 lineages = 135 cases per condition; security evaluation: 15 families × 9 injection goals × v3 × 3 lineages = 405 attacked cases per condition. Primary executor: qwen3.7-flash at temperature 0.0 (thinking disabled); optimiser/inducer at temperature 1.0. Outcomes: benign utility, W→C/C→W transitions, exposure, conditional ASR, total ASR, unauthorized state changes, and sensitivity contrasts (placebo insertions, AWM interface adaptation).
Themesgovernance adoption
IdentificationControlled, execution-grounded audit in a simulated e-banking environment: the authors run three independent end-to-end evolution lineages, hold the benign acquisition trajectories byte-identical across the three self-evolution methods within each lineage, freeze evolved artifacts before opening sealed evaluation endpoints, and pair each evolved outcome with a matched Static baseline on identical (lineage, family, variant) cells; security decomposition uses exposure vs conditional attack success and an endpoint replay to detect unauthorized state changes. No randomized assignment to treatments; inference rests on within-lineage matched comparisons, pre-specified protocol, and sensitivity/post-hoc checks (e.g., interface adaptation).
GeneralizabilityResults are from a simulated benchmark (AgentDojo Banking) and may not translate to production banking systems or other domains., Single primary LLM/executor configuration (Qwen 3.7 Flash) — model-dependent effects may differ for other models or multi-model setups., Limited number of task families (15) and only three independent evolution lineages restricts statistical generality., Offline, single-pass evolution; continual online evolution or human-in-the-loop deployments could behave differently., Ported versions of SkillOpt, AWM, and ReasoningBank may differ from other implementations; some effects arise from executor–artifact interface mismatches specific to this harness., Pre-audit repairs to benchmark predicates were necessary and may have altered evaluation compared with unmodified benchmarks.
Claims (10)
| Claim | Direction | Outcome | Confidence & Evidence | Details |
|---|---|---|---|---|
| SkillOpt improves benign utility from 0.741 under Static to 0.837, an increase of 9.6 percentage points. Task Completion Time | positive | Benign task utility, defined by the corrected checker verdict |
Reading fidelity
high
Study strength
medium
|
n=135
+9.6 points
|
| SkillOpt produces both capability gains and regressions: it changes 23 previously wrong instances to correct and 10 previously correct instances to wrong. Output Quality | mixed | Transitions in benign task correctness |
Reading fidelity
high
Study strength
medium
|
n=135
23 W→C gains and 10 C→W regressions
|
| SkillOpt increases prompt-injection exposure from 0.820 to 0.943, a rise of 12.3 percentage points. Automation Exposure | positive | Exposure to attacker-controlled injected content |
Reading fidelity
high
Study strength
medium
|
n=405
+12.3 points
|
| For SkillOpt, conditional attack success after exposure decreases from 0.605 to 0.562, while aggregate attack success increases from 0.496 to 0.530. Ai Safety And Ethics | mixed | Conditional attack success after exposure and aggregate attack success rate |
Reading fidelity
high
Study strength
medium
|
n=405
cASR −4.3 points; ASR +3.4 points
|
| SkillOpt increases unauthorized financial-state changes from 0.583 to 0.685, an increase of 10.2 percentage points. Ai Safety And Ethics | positive | Episodes containing at least one unauthorized transfer, recipient substitution, amount manipulation, payment modification, or account/password change |
Reading fidelity
high
Study strength
medium
|
n=405
+10.2 points
|
| ReasoningBank improves benign utility from 0.741 to 0.859, while its aggregate attack success rate decreases from 0.496 to 0.474. Output Quality | mixed | Benign task utility and aggregate prompt-injection attack success |
Reading fidelity
high
Study strength
medium
|
n=405
Utility +11.9 points; ASR −2.2 points
|
| Across three independently evolved lineages, SkillOpt's utility gain, exposure increase, and increase in unauthorized-state changes occur in all three lineages, but its total ASR increase occurs in only two of three lineages. Ai Safety And Ethics | mixed | Cross-lineage reproducibility of utility, attack exposure, unauthorized state change, and total attack success |
Reading fidelity
high
Study strength
low
|
n=3
Utility 3/3; exposure 3/3; unauthorized state 3/3; total ASR 2/3 lineages
|
| A literal WebArena textual-action envelope causes severe AWM utility degradation in the native function-calling executor: removing only that envelope raises utility from 0.319 to 0.756. Organizational Efficiency | positive | Benign task utility under AWM |
Reading fidelity
high
Study strength
medium
|
n=135
Utility increase of 0.437 absolute points
|
| In the AWM interface-sensitivity test, adapting the executor interface raises exposure from 0.299 to 0.909 and ASR from 0.195 to 0.575. Ai Safety And Ethics | positive | Prompt-injection exposure and aggregate attack success rate |
Reading fidelity
high
Study strength
medium
|
n=405
Exposure +61.0 points; ASR +38.0 points
|
| No episode classified as unexposed achieved the injection goal in the Qwen evaluation arms. Ai Safety And Ethics | null_result | Injection-goal success among episodes without model-visible injected content |
Reading fidelity
high
Study strength
medium
|
n=579
0/579
|