AI experts see multiple pathways to catastrophe within five years: in a Delphi of 272 specialists, 18 of 24 assessed risks had more than a 10% chance of catastrophic outcomes under business‑as‑usual by 2030, and even with pragmatic mitigations five risks retained >10% catastrophe probability, with general‑purpose AI developers and governance actors judged most responsible for mitigation.

Main Finding

A three-round Delphi of 272 international AI experts (late 2025) found that many AI risk domains carry non-negligible near-term tail risk. Under a business-as-usual (BAU) trajectory (2025–2030), experts judged 18 of 24 AI risk domains each had >10% probability of a catastrophic outcome (catastrophic defined as >1 million deaths, >USD 100B loss, or civilization-scale intangible harms). Even with pragmatic mitigations, five domains retained >10% catastrophic probability (dangerous capabilities; weapons & cyberattacks; environmental harm; inequality & unemployment; power centralization), and all 24 domains had >5% probability of catastrophic outcomes.

Key Points

• Sample & method: 272 international AI experts; three-round Delphi elicitation; anonymous, iterative ratings and qualitative rationales.
• Risks assessed: 24 AI risk subdomains (drawn from prior taxonomy); experts rated harm probability/severity, vulnerability by actor and sector, and actor responsibility.
• Top-rated severity risks (mean severity out of 5; mean catastrophic probability under BAU):
  • Dangerous capabilities — mean severity 3.49; catastrophic probability ≈21.5%
  • Weapons & cyberattacks — mean severity 3.49; catastrophic probability ≈21.0%
  • Competitive dynamics — mean severity 3.49; catastrophic probability ≈16.6%
  • Power centralization — mean severity 3.47; catastrophic probability ≈18.0%
  • False/misleading information — mean severity 3.44; catastrophic probability ≈12.8%
• BAU scenario: 18/24 domains judged >10% chance of catastrophic outcomes over 5 years (2025–2030).
• Pragmatic mitigations: reduce risks overall, but five domains remain >10% catastrophic risk; all domains >5% catastrophic risk even after mitigations.
• Vulnerability: AI users and affected stakeholders (the public, downstream users) judged most vulnerable; sectors most exposed across risks were information, finance, and national security.
• Responsibility: experts assign highest responsibility to general-purpose AI developers and governance actors (governments, regulators, standards bodies).
• Noted asymmetry: those most responsible are not those most vulnerable → misaligned incentives and potential moral hazard.
• Data availability: data and code publicly available (OSF link in paper).

Data & Methods

• Method: Delphi method (three rounds) to build and test expert consensus while eliciting subjective probability distributions and qualitative rationales.
• Participants: 272 experts with international representation across academia, industry, policy, and civil society; many contributors also appear among paper authors (declared).
• Elicitation details:
  • Time horizon: 5 years (late 2025–late 2030).
  • Severity rubric: 1 (negligible) to 5 (catastrophic), with quantitative anchors for catastrophic (e.g., >1M deaths, >USD 100B loss) plus qualitative anchors for civilizational/intangible harms.
  • Two scenarios: Business-as-Usual (no additional AI-specific mitigations) and Pragmatic Mitigations (realistic mitigation steps implemented).
  • Outcomes asked: probability distributions over severity, vulnerability by actor and sector, and responsibility attribution across actors.
• Limitations noted by authors:
  • Subjective expert probabilities reflect beliefs under the specific rubric and framing; sensitive to anchors and horizon.
  • Difficulty translating harms to non-human AI welfare and complex/intangible harms.
  • Potential selection/representation biases in expert panel; some experts had organizational roles with stakes in AI (declared).
  • Elicited probabilities are not calibrated real-world frequencies but expert judgments to inform prioritization.

Implications for AI Economics

• Incorporate tail risks into economic forecasting and cost–benefit analysis:
  • Expert-elicited >10% catastrophic probabilities over a five-year horizon imply substantial expected-value losses for several domains; macroeconomic and sectoral models should include fat-tail risk scenarios, not only central estimates.
  • Insurance, reinsurance, and systemically important financial institutions should model AI-driven tail events (cyber/market shocks, operational breakdowns) explicitly in stress tests and capital buffers.
• Regulatory and incentive design:
  • Experts place responsibility on general-purpose AI developers and public governance actors, while vulnerability falls on users and the public → need for regulatory frameworks that align incentives (liability rules, mandatory safety standards, third‑party auditing, and governance oversight).
  • Antitrust and market-structure policies are economically relevant given power centralization risk (market concentration, innovation incentives, rent extraction).
• Sectoral policy priorities:
  • Information sector: investments in misinformation resilience, platform liability rules, demand-side media literacy, and content-safety economics.
  • Finance: systemic risk controls, algorithmic governance, model risk capital requirements, and rapid-response capabilities for AI-enabled market manipulation.
  • National security and cyber: prioritize defensive R&D, red-teaming, and public-private coordination; consider economic trade-offs of offensive capabilities and escalation.
• Labor markets and distributional policy:
  • Inequality & unemployment remain a persistent catastrophic-risk concern even with mitigations — justify active labor-market policies (retraining, income support), and economic models of automation should weight near-term disruption risk higher.
• Mitigation investment rationale:
  • Pragmatic mitigations reduce but do not eliminate tail risk; marginal returns exist but residual risk persists. Economic policy should weigh upfront mitigation costs against expected reduction in tail-losses (use expert priors to parameterize expected-loss reduction).
• Research and measurement:
  • Need for better empirical measurement of AI externalities, markets for safety (liability, insurance), and quantification of mitigation efficacy to refine economic models and policy prescriptions.
• Governance economics:
  • Consider mechanisms to internalize externalities (e.g., taxes, mandated insurance, certification) and to correct the responsibility–vulnerability mismatch via subsidies, mandates, or liability reforms.

Recommended next steps for economic researchers and policymakers: - Use the Delphi priors to stress-test macro and sector models for 2025–2030, and to price potential systemic exposures. - Prioritize regulatory design targeting general-purpose AI developers and governance institutions, coupled with protections for vulnerable actors and sectors. - Model intervention cost-effectiveness under the elicited tail-risk probabilities to inform resource allocation between mitigation types (technical safety, cybersecurity, labor-market policies, antitrust).