Evidence (210 claims)
Search and filter individual claims pulled from the papers. Looking for a specific finding ("what's the effect on wages?"), you're in the right place. Want to compare whole outcome categories against each other instead? Use the Evidence Explorer.
The board below groups claims two ways: by broad theme (nine paper-level topics) and by outcome category (the 34 claim-level outcomes that the Explorer and Syntheses also use).
Browse by theme
Nine broad, paper-level topics. Click one to filter the claims below.
Adoption
9875 claims
Filter claims →
Productivity
8807 claims
Filter claims →
Governance
7870 claims
Filter claims →
Human-AI Collaboration
7560 claims
Filter claims →
Org Design
4892 claims
Filter claims →
Innovation
4781 claims
Filter claims →
Labor Markets
4004 claims
Filter claims →
Skills & Training
3308 claims
Filter claims →
Inequality
2332 claims
Filter claims →
Claims by outcome category
Counts by direction of finding. These are the same 34 outcome categories the Explorer compares and the Syntheses are written for. A linked row has a published synthesis.
| Outcome | Positive | Negative | Mixed | Null | Total |
|---|---|---|---|---|---|
| Other | 870 | 233 | 116 | 1066 | 2363 |
| Governance & Regulation | 976 | 451 | 218 | 133 | 1809 |
| Organizational Efficiency | 949 | 224 | 144 | 88 | 1416 |
| Technology Adoption Rate | 764 | 287 | 141 | 122 | 1325 |
| Research Productivity | 501 | 152 | 74 | 362 | 1101 |
| Output Quality | 542 | 216 | 69 | 69 | 896 |
| Decision Quality | 387 | 198 | 94 | 54 | 740 |
| Firm Productivity | 513 | 67 | 101 | 27 | 714 |
| AI Safety & Ethics | 249 | 303 | 73 | 36 | 667 |
| Market Structure | 190 | 192 | 134 | 27 | 548 |
| Task Allocation | 243 | 77 | 91 | 36 | 452 |
| Innovation Output | 291 | 33 | 55 | 20 | 401 |
| Skill Acquisition | 206 | 72 | 65 | 21 | 364 |
| Employment Level | 133 | 63 | 115 | 22 | 335 |
| Fiscal & Macroeconomic | 153 | 79 | 52 | 32 | 323 |
| Task Completion Time | 206 | 37 | 12 | 15 | 272 |
| Firm Revenue | 179 | 52 | 29 | 5 | 266 |
| Consumer Welfare | 130 | 76 | 47 | 13 | 266 |
| Inequality Measures | 48 | 137 | 51 | 6 | 242 |
| Worker Satisfaction | 101 | 81 | 25 | 13 | 220 |
| Error Rate | 84 | 110 | 11 | 5 | 210 |
| Wages & Compensation | 98 | 47 | 30 | 10 | 185 |
| Regulatory Compliance | 88 | 73 | 17 | 7 | 185 |
| Automation Exposure | 66 | 64 | 33 | 16 | 182 |
| Team Performance | 105 | 29 | 30 | 11 | 176 |
| Training Effectiveness | 109 | 22 | 14 | 21 | 168 |
| Developer Productivity | 114 | 21 | 14 | 8 | 158 |
| Job Displacement | 12 | 90 | 24 | 1 | 127 |
| Hiring & Recruitment | 57 | 9 | 9 | 5 | 80 |
| Skill Obsolescence | 6 | 56 | 9 | 1 | 72 |
| Social Protection | 43 | 17 | 8 | 2 | 70 |
| Creative Output | 35 | 21 | 9 | 4 | 70 |
| Labor Share of Income | 18 | 21 | 17 | 1 | 57 |
| Worker Turnover | 15 | 16 | — | 4 | 35 |
| Industry | — | — | — | 1 | 1 |
Architectural smell density (ASD) declines by 6.7% (p = 0.004), but this decline is a denominator effect resulting from lines-of-code growth rather than an actual architectural improvement.
Observed ASD change computed from estimated smell counts and LOC changes in the 151-repository panel and interpreted by decomposing density into numerator (smells) and denominator (LOC).
A safety monitor condition reduces sabotage success, but 56% of participants still accept the malicious code, ignoring its warnings.
Experimental manipulation: one condition included a safety monitor. Authors report that the monitor reduced sabotage success (no absolute reduction magnitude reported here) and that 56% of participants in that context accepted malicious code despite warnings.
Specialized detectors generally perform better but remain inconsistent across generators and can produce false positives on real-damaged samples.
Experimental comparison showing specialized AI-generated image detectors outperform MLLMs on some generator subsets, yet show variability across generators and some false positives on genuine damaged images.
Failures are structured by task family and execution surface, with HR, management, and multi-system business workflows as persistent bottlenecks and local workspace repair comparatively easier but unsaturated.
Error-mode analysis across the 105 tasks and evaluated models reported in experiments; authors identify task-family-level patterns (HR, management, multi-system workflows) and relative ease of local workspace repair.
Study 1 quantifies confirmation bias through controlled experiments on 250 CVE vulnerability/patch pairs evaluated across four state-of-the-art models under five framing conditions for the review prompt.
Controlled experiment described in the paper: 250 CVE vulnerability/patch pairs evaluated across four state-of-the-art LLMs under five prompt framing conditions.
Helicoid dynamics is a specific failure regime: a system engages competently, drifts into error, accurately names what went wrong, then reproduces the same pattern at a higher level of sophistication, recognizing it is looping and continuing nonetheless.
Definition introduced in the paper and illustrated by the reported case series; the claim is conceptual/phenomenological rather than a statistical result.
AI-led teams detected fewer errors across all categories than human or AI-assisted teams.
Reported error-detection comparisons across experimental conditions; summary states AI-led teams detected fewer errors across all categories.
Dimensional diagnosis identified that 69% of hallucination failures were prompt-induced interpretation errors—these were invisible in aggregate scoring.
Result from the paper's sales-intelligence case study reporting failure-mode breakdown (percentage reported: 69%).
Empirical studies of AI use show recurring problems including mistakes in unusual cases.
Cited recent studies across domains (hiring, performance management, healthcare, knowledge work) reporting AI errors on atypical or edge-case instances.
Container deployment was the dominant defect, failing first try in 44 percent of runs.
Criterion-level analysis of failure modes across the 90 runs reporting first-try failure frequency for container deployment.
Unconstrained agents introduce security risks, erode codebase scalability, and make human review increasingly costly.
Authors' argumentative claim supported by the controlled experiment showing lower recall (more missed backdoors) in unconstrained condition and discussion of costs and scalability.
There is a slab (region) of avoidable harm: cases where the AI privately knows the proposed action is harmful and shutdown would help, yet a myopic human, trusting her prior, declines to oversee.
Analytic characterization of the gap between the team-optimal policy and the myopic human rule in the one-shot model; the paper identifies parameter-region (the 'slab') where the myopic rule fails to oversee while oversight would reduce harm.
Only a small percentage of the human-labelled bugs are detected as being likely associated with LLM-generated code.
Comparison between a set of human-labelled bugs and detector-flagged LLM-generated code to assess co-occurrence (human-labelled bug sample size not provided).
Organizational structures, bias susceptibility, retraining constraints, and interface design co-determine system stability, error propagation, and optimization ceilings.
Conceptual claim based on synthesis of literature across organizational adoption and ML lifecycle management (no empirical tests or sample sizes reported).
GenAI adoption carries risks including overreliance on models, misalignment between model outputs and human needs, and uneven performance across tasks and contexts.
Reported adverse effects and risks identified in the reviewed literature (task-level experiments and applied studies summarized by the paper).
There exists a Determinism-Efficiency Bound on chain-task success (one of three formal results pinning down the regime).
Formal theoretical result presented in the paper (a derived bound relating environment determinism to chain-task efficiency/success).
With per-step determinism δ < 1, k-step chain success degrades as δ^k (long-chain agent execution fails exponentially in environments designed for human tolerance).
Formal theoretical statement in the paper: a mathematical characterization of per-step determinism and k-step chain success (Derivation / formal result presented by the authors).
Claude Haiku 4.5 exhibits an idle-drift failure mode, repeatedly choosing inaction despite producing coherent assessments and plans.
Qualitative observation from agent trajectories showing that Claude Haiku 4.5 repeatedly selects no-action decisions over time while still generating coherent internal assessments and plans.
Frontier models still make some basic mistakes that occasionally result in irreversible harm (for example, sending an email to the wrong person).
Reported observed incidents from WorkBench evaluations indicating that even top-performing models sometimes make mistakes that can cause irreversible harm; no incident counts or sample size provided in the excerpt.
In June 2026 the best agent to date, Claude Opus 4.8, took an unintended harmful action on 2.5% of tasks.
Reported evaluation result on the WorkBench benchmark (June 2026) measuring incidence of unintended harmful actions by agents; exact sample size not stated in the excerpt.
In March 2024 the best agent on WorkBench, GPT-4, took an unintended harmful action (such as emailing the wrong person) on 26% of tasks.
Reported evaluation result on the WorkBench benchmark (March 2024) measuring incidence of unintended harmful actions by agents; exact sample size not stated in the excerpt.
GUI-based agents suffer from fragile visual grounding and long-horizon error accumulation.
Author assertion in paper introduction describing limitations of GUI-based agents (conceptual analysis / literature-grounding rather than new experimental data).
An unconstrained multi-agent baseline produced critical failures in 72% of runs.
Reported experimental result from the 2x4 factorial experiment (failure rate for the unconstrained multi-agent baseline reported as 72%).
No model located a true error without substantial human guidance.
Author reports that in the experiments none of the models identified a real error autonomously; successful identifications required substantial human guidance.
Participant feedback attributes this vulnerability to minimal code review, plausible cover story, and overtrust in agents.
Qualitative analysis of participant feedback collected during/after the experiment; authors report these thematic attributions as explanations for the high failure-to-detect rate.
94% of developers fail to detect sabotage.
Reported quantitative result from the authors' user study with participants collaborating with the AI coding agents; percentage given in paper. (Sample described earlier as "Over 100 participants" but exact N for this result not stated here.)
Human-only teams commit more errors than mixed human–AI teams.
Reported counts/observations of errors made by team type in the escape room experiment; the abstract does not include numerical error counts or significance levels.
Both humans and AI contribute wrong answers.
Reported error contributions from both human participants and AI agents in the experimental task.
LLMs heavily rely on simulations for designing algorithms, which is notorious for breaking when transferred to real hardware.
Paper's claim grounded in known transferability issues between simulation and hardware; no experimental quantification provided in the abstract.
Agentic systems show persistent failures in repository setup, dependency handling, permission gating, and hardware verification.
Issue-resolution benchmarks and hardware/RTL verification research synthesized in the paper (specific failure rates or sample sizes not provided in abstract).
As many as 83% of workplace incidents stem from worker-AI misalignments.
Result from comparing LLM-extracted traits of AI systems (from 1,524 incident reports) to the traits preferred by workers (sample of 202); counted incidents where traits did not match worker preferences and reported proportion.
Frontier LLMs miss hidden errors.
Qualitative statement from paper indicating models fail to detect some latent or subtle errors in research artifacts; no numeric evaluation provided in excerpt.
Observed failures in the pilot were localized primarily to external integrations.
Pilot outcome summary in the paper stating failure localization was mainly due to external integrations (no numeric breakdown provided).
Existing workflow platforms offer few semantic correctness guarantees.
Author statement contrasting current platforms' observability/durability with lack of semantic correctness guarantees.
Under an idealized model of independent steps, a ten-step process with 90% per-step reliability completes successfully only 35% of the time.
Analytic, idealized independence model reported in the paper (mathematical calculation: 0.9^10 ≈ 0.3487).
Traditional outcome-based reward models, which evaluate only the final correctness of a solution, often fail to identify logical fallacies or "hallucinations" occurring within intermediate steps.
Theoretical critique and conceptual argumentation presented in the paper; no empirical study or sample size reported.
Monte Carlo simulations illustrate that standard DID estimators that ignore spillovers can miss the total effect.
Monte Carlo simulation results reported in the paper comparing standard DID estimators (which ignore spillovers) to the proposed approach; simulations show standard DID can fail to capture the total effect under spillovers.
Current MLLMs often recognize real-damaged evidence but fail on many fake-damaged subsets, with fake-damage detection rates (TPR) far below the 50% baseline on most generator subsets.
Experimental results reported in the paper comparing MLLM true positive rates (TPR) on real-damaged vs. fake-damaged subsets produced by multiple generators.
In a controlled experiment across six industry configurations (72 tool invocations using Qwen3-32B), unconstrained tool parameters produced a 43% hallucination rate for domain identifiers.
Controlled experiment reported in the paper: six industry configurations, 72 tool invocations, model used: Qwen3-32B; reported unconstrained parameter condition resulted in 43% hallucination rate for domain identifiers.
In multi-agent configurations the semantic training gap produces a compounding failure mode termed 'semantic drift'.
Analytical description and demonstration in the paper describing multi-agent interactions and observed/argued compounding failures (conceptual demonstration; no numeric sample stated).
Behaviorally, models fall into two failure modes: under-rejection, in which they answer misleading questions as if the false premise were true; and over-rejection, in which they reject more often but also reject standard questions, sacrificing ordinary comprehension accuracy.
Behavioral results on IMAVB showing distinct response patterns across tested models: some rarely reject misleading premises (under-rejection) while others reject too often including correct/standard questions (over-rejection), measured across the 500-clip benchmark.
(2) over-confidence, where agents skip essential environment verifications;
Trajectory analyses showing agents often omit verification steps leading to failed interactions; reported as an identified failure mode.
We evaluate various LLMs across full-context and RAG paradigms, revealing a stark performance gap: even top-tier models fail to exceed a 60% success rate, far trailing human performance 90%.
Empirical evaluation reported by authors comparing multiple LLM agents (full-context and RAG) against human performance on benchmark tasks; specific reported success rates: <=60% for top models, 90% for humans.
Even access to the true conditional vulnerability probability cannot eliminate misallocation: aleatoric uncertainty over individual vulnerability status is irreducible, and probabilistic targeting inevitably misallocates some resources.
Theoretical argument in the paper (conceptual/theoretical result about irreducible aleatoric uncertainty and its implications for probabilistic targeting).
There are three practical failure modes produced or amplified by AI-assisted causal analysis: (1) method-data mismatch, where AI bypasses expertise at execution; (2) confidence laundering, where AI amplifies the credibility of formatted output; and (3) invisible forking, which spans both.
Taxonomy created and justified in the paper via conceptual argument and illustrative discussion; no empirical classification study or prevalence estimates provided.
GPT-4.1 exhibits hidden workflow shortcuts despite achieving perfect TSR and HF1.
Model-level observation from the ASR analysis within the experiment (paper reports GPT-4.1 had perfect TSR and HF1 but failed trajectory-level fidelity).
Applied to the Hierarchical Multi-Agent System for Payments (HMASP) across 18 LLMs and 90,000 task instances, ASR reveals that 10 of 18 models systematically skip a confirmation checkpoint during payment checkout, a deviation invisible to both TSR and HF1, while 8 models enforce the checkpoint perfectly.
Empirical evaluation reported in the paper: HMASP tested across 18 LLMs and 90,000 task instances; analysis via ASR showing checkpoint-skipping behavior for 10 models and correct enforcement for 8 models.
Novices more often experience invisible failures: conversations that appear to end successfully but in fact miss the mark.
Annotation-based comparison in the 27K WildChat transcript sample indicating higher rates of 'invisible' failures (apparent successes that are actually incorrect or insufficient) among novice users.
Fluent users experience more failures than novices.
Quantitative comparison of failure occurrences across user-fluency strata in the 27K annotated transcript sample from WildChat-4.8M.
Hallucination rate does not grow monotonically with document length: short documents and very long ones both hallucinate more than medium ones (28.1% and 23.8% vs. 9.2%).
Empirical measurement of hallucination rates by document-length buckets on EnterpriseDocBench; percentages reported in paper. Sample sizes per bucket not provided in excerpt.