Current industry AI risk assessments are overly subjective; a socio‑technical framework translating organizational factors into measurable risk indicators can make governance demands for 'objectivity' operational — but it remains untested in practice.

Main Finding

The paper proposes a pragmatic, evidence-based 64-factor framework that decomposes traditional AI risk assessments (likelihood and impact) into specific, observable organizational and technical indicators. This sociotechnical approach aims to reduce subjective judgment and cognitive bias in organizational AI risk practice, thereby helping organizations meet emerging U.S. policy demands for "objectivity" in AI systems.

Key Points

• Policy context: New U.S. directives (Executive Order 14319, America's AI Action Plan) press for demonstrable objectivity and absence of ideological bias in AI, shifting expectations from flexible guidance to verifiable evidence.
• Problem identified: Existing AI risk assessment practices rely heavily on subjective likelihood/impact scoring, which is vulnerable to human cognitive biases (optimism, anchoring, availability, confirmation) and organizational incentives that understate risk.
• Contribution: A 64-factor observable-factor framework organized into seven risk categories (Security & Control; Data; Perception & Understanding; Ethical & Human Rights; Model; Business & Legal; Third Party).
• Methodological innovation: Decomposes likelihood and impact into separate, documentable factors (e.g., "model complexity level" for likelihood vs "system autonomy level" for impact) to reduce holistic, error-prone probability judgments and enable auditability.
• Materiality focus: Shifts emphasis from trying to predict precise failure probabilities to identifying contextual vulnerabilities (material conditions) that make failures more likely or more severe.
• Practical design: Framework targets non‑AI specialist organizations, intended to be usable by practitioners by mapping abstract risk concepts to verifiable evidence items (tests performed, documentation depth, historical incident frequency).
• Tradeoffs and limitations acknowledged: Some overlap between likelihood and impact factors is inevitable; authors use additive scoring pragmatically. The taxonomy is illustrative, not exhaustive, and requires organization-specific customization and empirical validation.

Data & Methods

• Study type: Conceptual / framework development rather than empirical testing.
• Foundations: Draws on socio-technical systems theory, established risk taxonomies (Model Risk Management from financial services, Enterprise Risk Management), and literature on cognitive biases and organizational accidents.
• Core methodological components:
  • Risk taxonomy (illustrative; full taxonomy and appendices available via SSRN as cited).
  • Seven high-level risk categories to group factors for usability.
  • A 64-item observable-factor list decomposing likelihood and impact into measurable indicators (examples in the paper show pairings such as "attack surface scope" for likelihood and "dependency on continuous operation" for impact).
  • Materiality-based approach: evaluate observable organizational/system conditions instead of assigning single probabilistic estimates.
  • Suggested scoring/aggregation is additive and separates likelihood vs impact to avoid double-counting as far as practicable.
• Evidence base: Synthesizes prior case studies of AI failures (e.g., Amazon hiring, Zillow pricing, Knight Capital trading), surveys and reviews of AI risk practice, and risk-management theory.
• Limitations in methods: No field trial or quantitative validation reported; calibration, weighting, and external validation are left as future work.

Implications for AI Economics

• Compliance costs and adoption thresholds:
  • Implementing the framework will impose measurable compliance costs (staff time, documentation, audits, testing), raising the fixed cost of deploying AI—especially for smaller firms—potentially slowing adoption or favoring larger incumbents.
  • Economists should model these costs when projecting AI diffusion, productivity gains, and sectoral adoption curves.
• Market structure and vendor dynamics:
  • Procurement rules emphasizing documented objectivity will increase demand for vendors and consultancies that can demonstrate traceable risk practices, shifting market power toward suppliers with established compliance capabilities.
  • Smaller or niche AI providers might face higher barriers to entry unless standard assessment-as-a-service markets or certification mechanisms emerge.
• Valuation and risk pricing:
  • Observable, auditable indicators enable more precise risk-adjusted valuation of AI products and projects, and may facilitate pricing of insurance/guarantees and risk capital allocation (analogous to model risk capital in finance).
  • Standardized factors could improve information symmetry across buyers, insurers, and regulators, reducing adverse selection but potentially increasing the cost of capital for riskier deployments.
• Innovation vs safety trade-offs:
  • The framework may re-shape incentives: stricter materiality-based assessment can reduce catastrophic failures but could also slow rapid iteration / experimental deployments, altering short-run innovation dynamics. Empirical work should estimate welfare trade-offs.
• Labor markets and human capital:
  • Demand for roles bridging AI technical skills and risk/compliance expertise will grow; economists should track wage premia and reallocation effects in data science and compliance professions.
• Regulatory economics and policy design:
  • The framework provides a basis for measurable compliance standards; economists can model the welfare impacts of moving from flexible guidance (NIST-style) to demonstrable-objectivity mandates (Executive Order style), including enforcement costs and regulatory arbitrage across jurisdictions.
• Research and measurement opportunities:
  • The framework creates testable indicators for empirical research: measure how factor scores correlate with realized harms, deployment rates, investment levels, and litigation/regulatory outcomes.
  • Economists can use the framework to build calibrated models of risk externalities, insurance market responses, and macro-level productivity effects of safer AI deployment.

Suggested research actions for AI economists: - Quantify implementation costs per firm size and sector; simulate effects on adoption and competition. - Empirically validate which observable factors best predict realized AI incidents to refine weights and scoring rules. - Model insurance pricing and capital requirements using factor-based risk metrics. - Estimate labor market impacts for compliance/assurance roles and effects on skill premiums.

Overall, the paper provides a practical, auditable approach to reduce subjectivity in AI risk assessments; for economists, it offers a concrete set of indicators to incorporate into models of adoption costs, market structure, and risk pricing—but it needs empirical calibration and validation before being deployed as a regulatory standard.