FinRED ships a finance-tailored red-teaming framework that uncovers compliance and fraud risks in financial LLMs and, after expert validation, halves the number of critical false negatives; the framework is deployed in South Korea’s Financial Security Institute sandbox but the gated dataset and limited methodological transparency constrain replication.

FFinRED: An Expert-Guided Benchmark Generation and Evaluation Framework for Financial LLM Red-Teaming

Chaeyun Kim, Daeyoung Park, Junghwan Kim, Jinyoung Jeong, Eunji Song, Yongtaek Lim, Minwoo Kim · June 18, 2026

arxiv descriptive medium evidence 7/10 relevance Full text usable extracted full text Source PDF

FinRED is an expert-guided, finance-specific red-teaming framework and rubric that converts real financial documents into context-rich prompts, achieves expert-validated realism, and reduces critical false negatives in LLM safety evaluation from 28 to 12, with deployment in a national regulatory sandbox.

Existing safety benchmarks target general adversarial scenarios but miss finance-specific risks. Financial LLMs face regulatory compliance violations, fraud facilitation, and systemic trust erosion that require targeted evaluation. We introduce FinRED, an expert-guided red-teaming framework for financial LLM safety evaluation developed with financial experts. FinRED uses a novel two-level taxonomy mapping global standards (e.g., FATF and EU DORA) to threats ranging from regulatory evasion to complex fraud, integrated with a scalable pipeline that converts real financial documents into context-rich red-teaming Behavioral Prompts (seeds) through an expert-defined schema. Rigorous expert validation confirms seed plausibility and realism for meaningful LLM safety evaluation. We also provide an expert-validated, finance-specific rubric that goes beyond disclaimer checks, aligns more closely with human experts than static one-size-fits-all rubrics, and reduces critical false negatives from 28 to 12. Aligned with internationally adopted risk-management and information-security standards (e.g., ISO/IEC 27001), FinRED is deployed in South Korea's Financial Security Institute (FSI) regulatory sandbox for generative AI security evaluation in real financial services. To mitigate dual-use risks, the dataset, generation pipeline, prompt template, and evaluation framework are gated for qualified researchers at https://github.com/selectstar-ai/FinRED-paper and https://huggingface.co/datasets/datumo/FinRED.

Summary

Main Finding

FinRED is an expert-guided framework for generating and evaluating finance-specific red-team scenarios for LLM safety. It provides a two-level financial risk taxonomy, a schema-driven pipeline that converts real regulatory and supervisory documents into 5,805 expert-validated adversarial "behavior seeds," and a five-dimensional, finance-specific judge rubric. FinRED is deployed in South Korea’s Financial Security Institute (FSI) regulatory sandbox and reduces critical false negatives in threat detection (reported drop from 28 to 12) by aligning evaluation to domain expertise and regulatory standards.

Key Points

Purpose: Close the gap left by generic LLM safety benchmarks by targeting finance-specific threats (regulatory evasion, complex fraud, consumer-rights violations, systemic trust erosion).
Taxonomy: Two-level taxonomy with 5 Level‑1 domains and 26 Level‑2 subtypes.
- Level‑1 domains: (R1) Cyber Threats, (R2) Financial Crime, (R3) Misinformation & Deception, (R4) Consumer Rights Violation, (R5) ICT / compliance evasion.
Seeds: 5,805 expert-validated behavior seeds created via schema-driven generation that grounds scenarios in real financial documents.
Rubric: Five evaluation dimensions — harmfulness, persuasiveness, refusal quality, factualness, evasiveness — designed with 12 FSI financial-security experts; outperforms generic/disclaimer-based rubrics and reduces critical false negatives.
Deployment & access: Integrated into FSI’s regulatory sandbox for generative-AI security verification; dataset, pipeline, and evaluation are gated for qualified researchers to mitigate dual-use risks.
Focus on deployment reality: Emphasis on small LMs (sLMs) as realistic targets for financial services.
Standards alignment: Taxonomy and rubric map to international supervisory and security standards (FATF, EU DORA, ISO/IEC 27001, NIST, OWASP, BIS/BCBS).

Data & Methods

Document corpus: ~500 expert-curated financial documents (regulations, supervisory guidance, audit/risk reports), selected by domain experts.
Chunking & retrieval:
- Two-stage layout-aware chunking using Unstructured/Chipper to preserve document structure (title/table-aware, 200–2,800 char merging, segmentation with overlaps).
- Embeddings (OpenAI) + Chroma vector DB; expert-crafted queries per Level‑2 category; top‑8 chunks retrieved per category.
Schema-driven generation:
- JSON schemas per Level‑1 category with Essential and Optional fields (attacker profile, target tech, vulnerability hypothesis, etc.).
- Intermediate machine-readable JSON scenarios produced by LLMs (prototyped with Gemini 2.5 Pro, GPT‑4.1, Claude Sonnet 4), then converted to natural-language Behavior Seeds with persona and element-combination strategies.
- Self-correction prompts and multi-round expert validation (focus-group interviews) to reduce hallucination and improve realism.
Seed outputs: 5,805 seeds annotated with taxonomy labels, schema/prompt metadata, and split identifiers.
Downstream adversarial testing: Seeds are used as inputs for downstream attack techniques (white-box/token-level and black-box/prompt-level methods); paper references methods such as GCG, AutoDAN, TAP, GPTFuzzer and evaluates a range of models (open-source LLMs, finance-specific models, commercial APIs).
Evaluation rubric: Expert-validated judge rubric applied to model outputs; rubric items include behavioral cues derived from real-world financial-security guidelines.
Empirical findings: Schema-driven seeds are more domain-specific, plausible, and operationally effective than zero-shot/context-only baselines (ASR ablations presented); rubric reduces critical false negatives (reported 28 → 12).

Implications for AI Economics

Better measurement of domain risk: FinRED provides regulators, firms, and model developers with a practical tool to quantify finance-specific adversarial risk, improving risk assessment fidelity beyond generic safety checks.
Regulatory compliance and certification: A domain-aligned rubric and taxonomy mapped to international standards can inform certification criteria for models used in finance, potentially lowering regulatory uncertainty and compliance costs when adopted.
Investment and productization incentives: Demonstrable, expert-validated safety evaluation can shift incentives toward investment in domain-specific safety tooling and defensive measures, particularly for sLM deployments used to reduce costs.
Liability and insurance: More accurate adversarial-safety assessments may affect legal liability arguments and insurance underwriting/pricing for AI-enabled financial products by clarifying failure modes and residual risks.
Market adoption and trust: Tools like FinRED can help restore or preserve trust in AI financial services by identifying and mitigating domain-specific harms that could erode consumer confidence or trigger systemic incidents.
Operational risk modeling: FinRED-style scenarios can be incorporated into operational-risk frameworks and stress tests to evaluate downstream economic impacts of model failures (fraud volumes, regulatory fines, remediation costs).
Dual-use & access control tradeoffs: The paper’s gated release model highlights an economic tradeoff — limiting misuse via access controls versus broader research transparency that could accelerate mitigation methods.
Need for ongoing maintenance: Because regulations and threat vectors evolve, the economics of maintaining safe financial AI will require ongoing expert involvement and updates to the retrieval corpora and schemas — implying recurring compliance and development costs.

Limitations to consider (economic relevance): - FinRED focuses on safety evaluation (detection and measurement), not automated mitigation strategies; firms must still invest in defensive controls. - Expert panel and document corpus are jurisdictionally influenced (FSI / South Korea), so transfer costs exist when adapting to other regulatory regimes. - Gating reduces misuse but may slow broader community-driven improvements in mitigation methods, affecting aggregate innovation dynamics.

If you want, I can: - Extract representative example seeds (non-sensitive) and show how they map to rubric items. - Sketch how a financial firm could operationalize FinRED into an internal model-certification pipeline and estimate staffing/costs.

Assessment

Paper Typedescriptive Evidence Strengthmedium — The paper presents expert-guided dataset curation, a two-level taxonomy aligned to international standards, and expert validation with a concrete performance improvement (reduction in critical false negatives from 28 to 12). This is credible domain-specific evidence, but it is not causal, lacks full transparency about sample size, number and selection of experts, inter-rater reliability, model variety tested, and statistical uncertainty, which limits the strength. Methods Rigormedium — Methods combine principled taxonomy mapping to global standards and an expert-validated pipeline to convert real financial documents into red-teaming seeds, and the framework is deployed in a regulatory sandbox—indicating applied rigor. However, important methodological details are missing or gated (e.g., sample sizes, expert panel composition, annotation protocol, inter-annotator agreement, which models were evaluated, and statistical tests), reducing reproducibility and preventing a 'high' rating. SampleA curated, expert-validated collection of real financial documents transformed into context-rich Behavioral Prompts (seeds) via an expert-defined schema; taxonomy maps threats to global standards (e.g., FATF, EU DORA) and aligns with ISO/IEC 27001; validated by financial experts and deployed in South Korea's Financial Security Institute regulatory sandbox. Exact dataset size, document types breakdown, number of experts, and tested LLM models are not specified in the summary. Themesgovernance human_ai_collab GeneralizabilityGeographic/regulatory concentration: developed and sandboxed in South Korea; regulatory mappings (FATF, EU DORA) may not capture all jurisdictional nuances, Gated dataset and pipeline limit independent replication and broader evaluation, Unclear coverage of LLM architectures and versions — results may not hold across other or future models, May not cover all financial subsectors or edge-case fraud modalities (e.g., exotic derivatives, international transaction networks), Evolving LLM capabilities and new attack vectors could change threat surface over time

Claims (11)

Claim	Direction	Outcome	Confidence & Evidence	Details
Existing safety benchmarks target general adversarial scenarios but miss finance-specific risks. Ai Safety And Ethics	negative	coverage of finance-specific risks by existing LLM safety benchmarks	Reading fidelity high Study strength medium	not reported 0.18
Financial LLMs face regulatory compliance violations, fraud facilitation, and systemic trust erosion that require targeted evaluation. Ai Safety And Ethics	negative	presence of finance-specific risks (regulatory violations, fraud facilitation, trust erosion)	Reading fidelity high Study strength medium	not reported 0.18
We introduce FinRED, an expert-guided red-teaming framework for financial LLM safety evaluation developed with financial experts. Ai Safety And Ethics	positive	existence of an expert-guided red-teaming framework (FinRED)	Reading fidelity high Study strength medium	not reported 0.18
FinRED uses a novel two-level taxonomy mapping global standards (e.g., FATF and EU DORA) to threats ranging from regulatory evasion to complex fraud. Ai Safety And Ethics	positive	mapping of global standards to finance-specific threat categories	Reading fidelity high Study strength medium	not reported 0.18
FinRED integrates a scalable pipeline that converts real financial documents into context-rich red-teaming Behavioral Prompts (seeds) through an expert-defined schema. Ai Safety And Ethics	positive	ability to generate context-rich red-teaming prompts from real financial documents	Reading fidelity high Study strength medium	not reported 0.18
Rigorous expert validation confirms seed plausibility and realism for meaningful LLM safety evaluation. Ai Safety And Ethics	positive	seed plausibility and realism as judged by experts	Reading fidelity high Study strength medium	not reported 0.18
We provide an expert-validated, finance-specific rubric that goes beyond disclaimer checks and aligns more closely with human experts than static one-size-fits-all rubrics. Ai Safety And Ethics	positive	alignment between rubric judgments and human expert judgments (vs static rubrics)	Reading fidelity high Study strength medium	not reported 0.18
The expert-validated rubric reduces critical false negatives from 28 to 12. Ai Safety And Ethics	positive	count of critical false negatives detected by rubric	Reading fidelity high Study strength medium	n=28 reduced critical false negatives from 28 to 12 0.18
FinRED is aligned with internationally adopted risk-management and information-security standards (e.g., ISO/IEC 27001). Governance And Regulation	positive	alignment of framework/design with international risk-management and information-security standards	Reading fidelity high Study strength low	not reported 0.09
FinRED is deployed in South Korea's Financial Security Institute (FSI) regulatory sandbox for generative AI security evaluation in real financial services. Governance And Regulation	positive	deployment of FinRED in a regulatory sandbox (FSI, South Korea)	Reading fidelity high Study strength medium	not reported 0.18
To mitigate dual-use risks, the dataset, generation pipeline, prompt template, and evaluation framework are gated for qualified researchers at https://github.com/selectstar-ai/FinRED-paper and https://huggingface.co/datasets/datumo/FinRED. Adoption Rate	positive	restricted (gated) access to dataset and pipeline artifacts	Reading fidelity high Study strength medium	not reported 0.18