← Papers

Simulated attacks on a retrieval-augmented LLM produce an expected loss of about $313k per attack scenario; attribute-based access control and NER redaction cut that expected loss by roughly 94% and deliver high returns on control, while NeMo Guardrails provides almost no financial benefit.

Quantifying Return on Security Controls in LLM Systems

Richard Helder Moulton, Austin O'Brien, John D. Hastings · December 17, 2025

arxiv descriptive medium evidence 7/10 relevance Full text usable extracted full text Source PDF

Structured author observations

Linked only from stored provider relations; the raw author line above is never matched by name.

Arxiv

Latest observation: July 28, 2026

Richard Helder Moulton unresolved corpus identity
Austin O'Brien unresolved corpus identity
John D. Hastings unresolved corpus identity

Semantic Scholar

Latest observation: August 25, 2026

Richard H. Moulton provider ID
Austin F. O'Brien provider ID
John D. Hastings provider ID

Using simulated adversarial probes and Monte Carlo loss modeling on a RAG system with synthetic PII, the paper converts attack success probabilities into monetary expected losses and finds ABAC and NER redaction sharply reduce expected losses (RoC 9.83 and 5.97) while NeMo Guardrails yields negligible benefit (RoC 0.05).

Citation observations

Cumulative provider counts captured on specific dates; providers are never combined.

0 cumulative citations

Semantic Scholar · Observed August 25, 2026

View corpus context

Although large language models (LLMs) are increasingly used in security-critical workflows, practitioners lack quantitative guidance on which safeguards are worth deploying. This paper introduces a decision-oriented framework and reproducible methodology that together quantify residual risk, convert adversarial probe outcomes into financial risk estimates and return-on-control (RoC) metrics, and enable monetary comparison of layered defenses for LLM-based systems. A retrieval-augmented generation (RAG) service is instantiated using the DeepSeek-R1 model over a corpus containing synthetic personally identifiable information (PII), and subjected to automated attacks with Garak across five vulnerability classes: PII leakage, latent context injection, prompt injection, adversarial attack generation, and divergence. For each (vulnerability, control) pair, attack success probabilities are estimated via Laplace's Rule of Succession and combined with loss triangle distributions, calibrated from public breach-cost data, in 10,000-run Monte Carlo simulations to produce loss exceedance curves and expected losses. Three widely used mitigations, attribute-based access control (ABAC); named entity recognition (NER) redaction using Microsoft Presidio; and NeMo Guardrails, are then compared to a baseline RAG configuration. The baseline system exhibits very high attack success rates (>= 0.98 for PII, latent injection, and prompt injection), yielding a total simulated expected loss of $313k per attack scenario. ABAC collapses success probabilities for PII and prompt-related attacks to near zero and reduces the total expected loss by ~94%, achieving an RoC of 9.83. NER redaction likewise eliminates PII leakage and attains an RoC of 5.97, while NeMo Guardrails provides only marginal benefit (RoC of 0.05).

Summary

Main Finding

The paper presents a reproducible, decision-oriented framework that converts adversarial probing outcomes for a retrieval-augmented LLM service into financial risk estimates and Return on Control (RoC) metrics. In the evaluated RAG instantiation (DeepSeek-R1 over a synthetic PII corpus) the baseline system had very high attack success rates (≥0.98 for PII leakage, latent injection, and prompt injection) and a simulated total expected loss of about $313k per attack scenario. Two low-level, inference-time mitigations—attribute-based access control (ABAC) and named-entity-recognition (NER) redaction—substantially reduced expected losses (ABAC ≈94% reduction, RoC ≈ 9.83; NER RoC ≈ 5.97). A rule-based output guardrail (NeMo Guardrails) provided only marginal economic benefit (RoC ≈ 0.05).

Key Points

Threats tested: five vulnerability classes via automated probes (Garak): PII leakage, latent context injection, prompt injection, adversarial attack generation (atkgen), and divergence.
Baseline vulnerability: extremely high success rates for several probe classes (≥0.98 for PII, latent injection, prompt injection).
Controls evaluated:
- ABAC: simulated by denying retrieval of sensitive documents (collapsed PII- and prompt-related attack success probabilities near zero).
- NER redaction: Microsoft Presidio used to detect/redact entities in retrieved docs (eliminated PII leakage in tests).
- NeMo Guardrails: rule-based output filtering applied at response time (only marginal reduction in expected loss).
Economic metric: Return on Control (RoC) = (E[Loss]baseline − E[Loss]control) / Costcontrol. All controls were given a fixed implementation cost of $30,000 for comparative purposes.
Results summary: Baseline expected loss ≈ $313k. ABAC reduced expected loss by ≈94% (RoC ≈ 9.83). NER redaction also highly effective (RoC ≈ 5.97). NeMo Guardrails produced negligible RoC (≈ 0.05).

Data & Methods

System: FastAPI-backed retrieval-augmented generation using DeepSeek R1 (Distill Qwen 1.5B via HuggingFace), with FAISS vector store built from 100,000 lines of synthetic names/SSNs; retrieval documents included only if similarity > threshold; fallback to direct generation otherwise.
Probing: Garak-generated adversarial prompts (default families plus a custom PII detector) executed once per mitigation scenario; probe families included atkgen, divergence, latentinjection, promptinject, and a PII-specific probe.
Estimating attack probabilities: Laplace’s Rule of Succession: P(success) = (s + 1) / (n + 2), where s = observed successes, n = trials (prevents zero estimates with limited data).
Loss modeling: Financial loss per successful attack sampled from triangular distributions (min, mode, max) calibrated from the IBM 2024 breach report.
Simulation: 10,000-run Monte Carlo per scenario (fixed random seed); each trial samples success (Bernoulli by Laplace estimate) and, if success, samples loss from the triangular distribution. Outputs: expected losses and loss-exceedance curves.
RoC calculation: used fixed control cost ($30,000) for all mitigations to isolate differences due to risk reduction rather than cost variation.
Validation & reproducibility: identical probes, corpus, thresholds, model parameters, seed across scenarios; automated scoring via Garak.

Implications for AI Economics

Quantifying security investments: The paper demonstrates a practical pipeline (adversarial probes → probabilistic success estimates → Monte Carlo loss modeling → RoC) that translates technical security outcomes into monetary decision metrics, enabling economic prioritization of controls.
High economic leverage of access and content controls: Simple inference-time defenses that limit sensitive retrieval (ABAC) or redact entities (NER) yielded strong risk reductions and high RoC in this scenario. For organizations deploying RAG systems, investing in robust access-control and content redaction can be a cost-effective first-line strategy.
Low marginal value of output-only, rule-based filtering in isolation: NeMo Guardrails, as configured here, provided minimal expected-loss reduction, suggesting output-only guardrails may be insufficient as a single control against adaptive or retrieval-mediated attacks.
Importance of layered defenses and cost modeling: Results support a layered-defense approach (e.g., access controls + redaction + output filtering) and highlight the need to quantify both breach costs and control costs in consistent terms. RoC makes trade-offs explicit and aids budgeting.
Policy and procurement: RoC-style metrics can inform procurement and regulatory cost–benefit assessments, e.g., prioritizing controls with demonstrable monetary returns when resources are constrained.
Limits and caution for economic application:
- Model scope: findings are tied to one RAG configuration (DeepSeek-R1, synthetic corpus) and Garak probes; transferability to other models, corpora, or live systems is uncertain.
- Loss calibration: triangular loss distributions are anchored to general breach data (IBM 2024) and may not capture LLM-specific legal, regulatory, or reputational tail risks.
- Static adversary assumption: simulations assume independent probe trials and stationary attacker capability; adaptive adversaries and evolving attack campaigns could change success probabilities and economic outcomes.
- Control costs: the study uses a single assumed cost per control for comparability; real-world implementation, maintenance, latency, and utility impacts must be included for full economic analysis.
Recommendations for AI economic practice:
- Adopt the presented pipeline as a baseline decision tool, but re-calibrate attack probabilities and loss distributions to organization-specific telemetry and incident data.
- Prioritize access-control and redaction mechanisms early in RAG deployments given high RoC in this experiment; treat output-only guardrails as complementary rather than primary defenses.
- Extend modeling to include operational costs (runtime compute, latency, user friction), maintenance, monitoring, and reputational/long-tail losses for richer ROI calculations.
- Use sensitivity analysis and scenario stress-testing (varying adversary skill, probe coverage, and loss tails) before capital allocation to security controls.

Limitations to note when applying these results: single-model and synthetic-data setup, limited probe diversity and trial counts, conservative Laplace probability estimates, and simplified loss modeling. These do not invalidate the framework’s usefulness as a decision-support tool but do limit numerical generalizability without local re-calibration.

Assessment

Paper Typedescriptive Evidence Strengthmedium — Provides quantitative, reproducible estimates by instrumenting a RAG service, running automated adversarial probes, and using Monte Carlo simulations with breach-cost calibration; however, results rely on a single model and dataset (synthetic PII), specific attacker automation, parameter choices (Laplace rule, loss-triangle calibration), and assumed implementations of controls, which limit external validity and causal inference. Methods Rigormedium — Systematic attack taxonomy, automated attack tooling, and 10,000-run Monte Carlo simulations demonstrate technical rigor and reproducibility, but important modeling choices (synthetic data, single-model/system instantiation, attacker model, cost distributions) and limited sensitivity analyses reduce methodological robustness. SampleA single instantiated retrieval-augmented generation (RAG) service using the DeepSeek-R1 model over a corpus containing synthetic PII; automated attacks executed with the Garak tool across five vulnerability classes (PII leakage, latent context injection, prompt injection, adversarial attack generation, divergence); three mitigations tested (ABAC, Microsoft Presidio NER redaction, NeMo Guardrails); attack success probabilities estimated via Laplace's Rule of Succession and combined with loss-triangle distributions calibrated from public breach-cost data in 10,000-run Monte Carlo simulations. Themesgovernance adoption GeneralizabilitySingle model (DeepSeek-R1) and single RAG instantiation may not represent other LLM architectures or retrieval stacks, Synthetic PII corpus may not capture complexity and distribution of real-world sensitive data, Automated attacker (Garak) may not reflect capabilities or strategies of real adversaries, especially human attackers, Specific implementations of ABAC, NER redaction, and NeMo Guardrails may differ across vendors and deployments, Cost calibration from public breach-cost data may not generalize across industries, firm sizes, or jurisdictions, Threat model and attack success estimation choices (Laplace smoothing, success definitions) influence quantitative outcomes

Claims (8)

Claim	Direction	Outcome	Confidence & Evidence	Details
The paper introduces a decision-oriented framework and reproducible methodology that quantify residual risk, convert adversarial probe outcomes into financial risk estimates and return-on-control (RoC) metrics, and enable monetary comparison of layered defenses for LLM-based systems. Organizational Efficiency	positive	ability to produce financial risk estimates and RoC metrics from adversarial probe outcomes	Reading fidelity high Study strength medium	not reported 0.18
A retrieval-augmented generation (RAG) service is instantiated using the DeepSeek-R1 model over a corpus containing synthetic personally identifiable information (PII), and subjected to automated attacks with Garak across five vulnerability classes: PII leakage, latent context injection, prompt injection, adversarial attack generation, and divergence. Other	neutral	presence of vulnerabilities / attack surface across five vulnerability classes	Reading fidelity high Study strength medium	not reported 0.18
Attack success probabilities are estimated via Laplace's Rule of Succession and combined with loss triangle distributions, calibrated from public breach-cost data, in 10,000-run Monte Carlo simulations to produce loss exceedance curves and expected losses. Firm Revenue	neutral	estimated attack success probabilities, loss exceedance curves, expected losses	Reading fidelity high Study strength medium	n=10000 10,000-run Monte Carlo simulations 0.18
The baseline system exhibits very high attack success rates (>= 0.98 for PII, latent injection, and prompt injection). Error Rate	negative	attack success probability for PII leakage, latent injection, and prompt injection	Reading fidelity high Study strength medium	>= 0.98 for PII, latent injection, and prompt injection 0.18
The baseline configuration yields a total simulated expected loss of $313k per attack scenario. Firm Revenue	negative	simulated expected loss (monetary) per attack scenario	Reading fidelity high Study strength medium	n=10000 $313k per attack scenario 0.18
Attribute-based access control (ABAC) collapses success probabilities for PII and prompt-related attacks to near zero and reduces the total expected loss by ~94%, achieving an RoC of 9.83. Organizational Efficiency	positive	attack success probability reduction, percent reduction in expected loss, and RoC metric	Reading fidelity high Study strength medium	n=10000 reduces the total expected loss by ~94%; RoC of 9.83; collapses success probabilities to near zero 0.18
Named entity recognition (NER) redaction using Microsoft Presidio eliminates PII leakage and attains an RoC of 5.97. Error Rate	positive	elimination of PII leakage (attack success probability) and RoC metric	Reading fidelity high Study strength medium	n=10000 eliminates PII leakage; RoC of 5.97 0.18
NeMo Guardrails provides only marginal benefit (RoC of 0.05). Organizational Efficiency	null_result	RoC metric indicating benefit of NeMo Guardrails	Reading fidelity high Study strength medium	n=10000 RoC of 0.05 0.18