Evaluation should move beyond model accuracy to measure whether human-AI teams are ready to collaborate: the paper offers a four-part taxonomy and trace-based metrics to assess outcomes, reliance, safety signals and learning so organizations can better calibrate, recover from errors and govern AI-assisted decisions.

Main Finding

The paper argues that evaluating AI systems by model accuracy alone is insufficient for safe, effective deployment as human collaborators. It proposes a trace-based measurement framework centered on "team readiness" — whether human–AI teams can recognize failures, calibrate reliance, recover from errors, and govern AI use. The author introduces a taxonomy of four metric families (Outcomes; Reliance & Interaction; Safety & Harm; Learning & Readiness) and maps them to an onboarding lifecycle (Understand–Control–Improve, U–C–I) to make readiness observable and actionable.

Key Points

• Problem diagnosis

  • Standard evaluation (accuracy, explanation fidelity, self-reported trust) misses critical failure modes that emerge in real workflows: overreliance on incorrect advice, underuse of helpful advice, brittle local strategies, and governance breakdowns.
  • Trust and accuracy poorly predict situated reliance behavior; many harms arise from miscalibrated human reliance rather than pure model errors.

• Conceptual contribution

  • Reframes onboarding as a measurable learning intervention (U–C–I): Understand (learn model limits), Control (calibrate reliance, use controls/escalation), Improve (iterate policies and training from failures).
  • Proposes making governance and accountability first-class, measured via behavioral signals (rollback, escalation, contradiction detection), not only documentation.

• Metric taxonomy (trace-based)

  • Outcome metrics: team gain vs. human-only/AI-only; regret_best (avoidable error relative to oracle that picks the better of human or AI per case); error recovery vs. amplification.
  • Reliance & Interaction metrics: accept-on-wrong (agreeing with incorrect AI), changed-to-wrong (switching from correct human judgment to incorrect final outcome after AI), override frequency/timing, reliance slope (sensitivity of acceptance to AI correctness), local vs. global update asymmetry.
  • Safety & Harm metrics: AI-induced harm (cases where AI causes harm), near-misses, governance-in-use signals (rollback/escalation frequency, rule–behavior contradictions).
  • Learning & Readiness metrics: calibration gap (confidence vs. correctness), time-to-calibration, stability under distribution shift, transfer across tasks/versions.

• Operational stance

  • Metrics should be computed from interaction traces (initial human judgments, AI recommendations, final decisions, timestamps, and outcome labels) rather than self-report or model properties.
  • Mapping metrics to U–C–I suggests concrete interventions (curated failure sets for Understand; calibration cards and safe levers for Control; feedback loops and governance update processes for Improve).

• Open questions / research agenda

  • Formal criteria for when a user is “AI-ready”.
  • Which onboarding metrics generalize across domains and expertise levels.
  • How to evaluate governance empirically through behavior.
  • How to design benchmarks that include calibration, error recovery, and governance measures.

Data & Methods

• Paper type: conceptual / methods paper (framework and taxonomy), built on literature synthesis across HCI, HAI, XAI, clinical AI onboarding, and accountable-AI research.
• Methodological approach:
  • Review of empirical findings that show divergence between accuracy/trust and real-world reliance behaviors.
  • Specification of trace-based metric definitions (formal equations and examples referenced in appendix) that can be computed from logged interactions: action traces (accept/override/change), error attribution (AI-influenced vs. independent), and longitudinal learning signals (pre/post probes, case sequences).
  • Mapping between metrics and the U–C–I lifecycle to suggest evaluation timing and interventions.
• Empirical content: No novel large-scale empirical dataset is presented; the paper synthesizes prior experimental and observational work and offers operational definitions for future measurement and benchmarking.
• Practical considerations discussed:
  • Implementation requires event-logging in deployed systems; when ground-truth labels are delayed or costly, sampling or proxies (e.g., disagreement/escalation) can be used.
  • Privacy and aggregation/anonymization concerns for behavioral trace collection.

Implications for AI Economics

• Reassessing valuation and ROI of AI investments

  • Productivity estimates that rely on model accuracy will misstate returns if they ignore human–AI interaction effects. Team-level metrics (team gain, regret_best) are necessary to compute realistic welfare gains or losses from deployment.
  • Onboarding and governance create recurring costs (training, monitoring, audit trails, controls). Economic models should incorporate these operational and learning costs into deployment cost-benefit calculations.

• Risk assessment, insurance, and liability

  • Metrics like AI-harm, changed-to-wrong, and near-miss rates enable quantification of expected harm attributable to AI influence — critical inputs for liability assignment, professional indemnity, and insurance pricing.
  • Behavioral governance signals (rollback/escalation rates, intervention latency) offer measurable indicators of whether governance mechanisms are effective in practice, informing regulatory compliance assessments and liability adjudication.

• Market design and procurement

  • Procurement criteria and vendor competition should shift from model-centric benchmarks (accuracy) to team-readiness benchmarks (calibration, error recovery, governance-in-use). This changes incentives for vendors toward investable onboarding tools, safer defaults, and auditability.
  • Standardized readiness metrics would improve comparability across products and enable better procurement decisions by firms and public agencies.

• Labor markets and task allocation

  • Human–AI complementarity estimates depend on whether humans are effectively calibrated. Miscalibration can reduce effective labor productivity or increase error externalities. Empirical production-function estimation should use interaction-derived measures (reliance slope, calibration gap) to capture effective complementarity.
  • Onboarding quality and transferability (stability under shift) influence how quickly firms can redeploy AI to new tasks and scale labor reallocation; thus, persistence of readiness affects dynamic labor demand.

• Policy and regulation

  • Regulators should require operational readiness evidence (interaction logs, governance-in-use metrics) in addition to model documentation. Trace-based metrics enable ex post audits and continuous monitoring for safety compliance.
  • Standardized readiness metrics would facilitate evidence-based policy (e.g., sector-specific thresholds for acceptable AI-harm rates, required rollback/escala­tion mechanisms).

• Research and measurement economics

  • Availability of standardized, trace-based metrics enables better empirical identification of causal effects of AI deployments (e.g., difference-in-differences or instrumental-variable estimates that condition on readiness metrics), improving policy and ROI inference.
  • Benchmarking and public datasets that include interaction traces would permit cross-study meta-analyses of economic impacts and more reliable external validity.

• Caveats and cost-benefit trade-offs

  • Measuring readiness imposes data, logging, labeling, and privacy costs; economic assessments must weigh measurement and governance expenses against benefits from reduced harm and higher effective productivity.
  • The proposed metrics are conceptual and require empirical validation and standardization across domains before being used in regulatory or financial valuations.

Overall, this framework suggests that economists and decision-makers should move beyond model accuracy when assessing the economic value, risks, and policy needs of AI deployments. Incorporating team-level, trace-based readiness metrics will produce more accurate assessments of returns, liabilities, and labor-market impacts, and will better align incentives for vendors and organizations toward safer, governed human–AI collaboration.