Academic fairness evaluations in hiring understate risk by ignoring uncertainty: only one of 21 common metrics quantified probabilistic uncertainty and 75% of evaluations reported no variability estimates. The authors demonstrate a risk-science reanalysis of a resume‑screening dataset and propose an 'AI Risk Report Card' to standardize risk-aware evaluation and communication.
Citation observations
Cumulative provider counts captured on specific dates; providers are never combined.
No provider observation is available for this paper.
Missing data, not a zero citation count.
Scholarly work which aims to describe potential societal impacts (e.g., risks) of proliferating technology (especially related to artificial intelligence or other algorithmic systems) is likely to have an impact beyond the scientific communities it was written for, given that general society itself is a primary object of study. However, it is an open question whether the current practices of AI evaluation scholarship follow the principles and best practices established by risk science, which aims to systematically generate knowledge related to understanding, assessing, communicating, managing, and governing risk. In this work, we examine this in depth by conducting a literature review of scholarly works purporting to evaluate the bias or fairness of technological systems used for tasks related to hiring and employment. Through analysis of 22 common fairness evaluation metrics and studies using them, we find that most characterize the severity of bias- or fairness-related consequences but do not follow best practices to characterize the uncertainty around either the occurrence of these consequences or severity estimates. Next, we conduct a case study of fairness evaluation for an AI-mediated resume screening task and demonstrate how principles of risk science can be incorporated into such an evaluation. Finally, we propose the AI Risk Report Card, which facilitates the reporting and communication of risk assessment results to stakeholders in positions to act based on the predicted risks. The outcomes of these activities suggest that further research at the convergence of risk science and AI evaluation can lead to advancements in AI assessments of societal impact by enabling shared frameworks to evaluate and discuss AI risks both within and outside of the scientific community.
Summary
Main Finding
Most AI fairness evaluations in the hiring/employment domain quantify consequence severity (bias/fairness metrics) but largely fail to quantify or communicate uncertainty. The paper shows that neglecting uncertainty obscures the true risk picture and can mislead stakeholders; it demonstrates how risk-science principles (uncertainty + severity, aleatory vs epistemic uncertainty, strength-of-knowledge) can be applied to fairness evaluation and proposes an “AI Risk Report Card” to standardize risk-aware reporting.
Key Points
- Definition and framing
- Adopts risk-science framing: risk = uncertainty about and severity of consequences to things people value.
- Distinguishes aleatory (stochastic) vs epistemic (knowledge) uncertainty; argues both must be represented in AI impact assessments.
- Literature review results
- Reviewed common fairness metrics and empirical studies for hiring/employment tasks.
- Of ~22 common fairness metrics/studies analyzed, only one metric provided a way to quantify probabilistic uncertainty.
- 75% of reported evaluation results lacked variability/uncertainty estimates (e.g., confidence intervals, error bars).
- Consequence of ignoring uncertainty
- Identical point estimates of harm can imply very different risk when uncertainty differs (illustrated with disparate-impact threshold examples).
- Omitting uncertainty can mask high-probability low-severity vs low-probability high-severity scenarios and misguide decisions.
- Case study
- Re-analyzed an AI-mediated resume-screening dataset (from Wilson & Caliskan 2024) using risk-science methods.
- Demonstrated how including stochastic and epistemic uncertainty changes the interpretation of fairness risk and mitigation priorities.
- Reporting proposal
- Introduces the AI Risk Report Card to standardize reporting of both consequence severity and uncertainty along with strength-of-knowledge qualifiers, to make results actionable for non-research stakeholders.
- Resources
- Code and datasets made available: https://github.com/kyrawilson/FairRisk
- Extended version on arXiv.
Data & Methods
- Systematic literature review
- Collected scholarly evaluations of algorithmic fairness for hiring/employment tasks (benchmarking, audits, fairness metric papers).
- Cataloged ~22 commonly used fairness metrics and analyzed published uses for whether they (a) measure consequence severity, (b) provide probabilistic uncertainty, and (c) report variability/uncertainty estimates.
- Coded reporting practices (presence/absence of confidence intervals, error bars, uncertainty discussion).
- Metric-level assessment
- Classified metrics by what aspect of risk they capture (e.g., disparity magnitude, classification error differences) versus whether they allow probabilistic or interval uncertainty quantification.
- Case study re-analysis
- Re-evaluated an existing resume-screening dataset using risk-science techniques:
- Derived distributions over harm/severity (to capture stochastic uncertainty).
- Represented epistemic uncertainty with interval probabilities / strength-of-knowledge qualifiers.
- Compared point-estimate-only interpretation vs full risk-aware interpretation to show practical consequences for decision-making.
- Outcome
- Synthesized best-practice recommendations and constructed a draft AI Risk Report Card template for communicating risk to stakeholders.
Implications for AI Economics
- Improved decision-making under uncertainty
- Economic decisions about AI adoption (procurement, deployment, remediation investments) require expected-risk calculations; omitting uncertainty biases expected-cost/benefit analyses.
- Explicit uncertainty allows better estimation of expected harms (E[Harm] = ∑ probability × severity) and tail risks that matter for welfare and liability.
- Better policy and regulation design
- Regulators and labor-market policymakers need measures of both likelihood and severity to set thresholds, compliance requirements, and enforcement priorities; uncertainty qualifiers inform precautionary approaches and setting appropriate safety margins.
- Market mechanisms and contracting
- Insurance, indemnity clauses, and vendor contracting for algorithmic hiring can be priced more accurately when uncertainty and strength of evidence are reported; creates incentives for data collection and validation (value of information).
- Labor-market modeling and welfare analysis
- Models of technology-driven displacement, discrimination externalities, and redistribution depend on credible uncertainty estimates to forecast labor supply, earnings impacts, and social welfare under alternative mitigation strategies.
- Empirical research practices
- Economists studying AI effects should adopt: reporting of confidence intervals/bootstrap variability, Bayesian/interval-probability approaches for low-data settings, explicit separation of aleatory vs epistemic uncertainty, and strength-of-knowledge annotations to prioritize further data collection (value-of-information analysis).
- Transparency and market functioning
- Standardized risk reporting (AI Risk Report Card) could reduce information asymmetries between vendors, employers, workers, and regulators, improving market efficiency and reducing misallocation of resources due to over- or under-estimated risks.
- Research agenda suggestions
- Validate the report card’s utility in field/policy contexts and quantify how uncertainty-aware reporting changes economic outcomes (procurement choices, liability exposure, labor-market signals).
- Integrate risk-aware fairness metrics into economic models of AI adoption, insurance pricing, and regulatory impact assessments.
If you’d like, I can: - Extract the specific list of the ~22 fairness metrics the paper analyzed. - Produce a one-page AI Risk Report Card template customized for hiring-tool procurement (fields, examples). - Translate the paper’s case-study results into an economic decision example (cost-benefit under uncertainty).
Assessment
Claims (8)
| Claim | Direction | Outcome | Confidence & Evidence | Details |
|---|---|---|---|---|
| Most AI fairness evaluations of hiring and employment systems characterize the severity of bias- or fairness-related consequences but do not characterize the uncertainty surrounding either the occurrence of those consequences or the estimates of their severity. Ai Safety And Ethics | negative | Characterization and reporting of fairness-related harms and their uncertainty |
Reading fidelity
high
Study strength
medium
|
n=21
|
| Only one of the 21 fairness metrics commonly used to evaluate hiring tools can quantify probabilistic uncertainty. Ai Safety And Ethics | negative | Capacity of fairness metrics to quantify probabilistic uncertainty |
Reading fidelity
high
Study strength
medium
|
n=21
1 out of 21 metrics
|
| Fairness evaluation results were reported without variability estimates 75% of the time. Ai Safety And Ethics | negative | Reporting of variability or uncertainty estimates in fairness evaluations |
Reading fidelity
high
Study strength
medium
|
75% of the time
|
| Considering both consequence severity and uncertainty or variability is essential for understanding and controlling or mitigating risk in AI fairness evaluations. Ai Safety And Ethics | positive | Completeness of AI fairness risk characterization |
Reading fidelity
high
Study strength
high
|
not reported
|
| In the disparate-impact example, when expected harm severity is 0.85, increasing variability from 0.5 to 0.8 raises the probability of a harmful outcome from 15.89% to 76.93%. Ai Safety And Ethics | negative | Probability of a harmful disparate-impact outcome |
Reading fidelity
high
Study strength
high
|
increase from 15.89% to 76.93%
|
| The 0.8 disparate-impact threshold is based on legal precedent and tradition, but there is currently no empirical evidence that it represents unfairness better than alternative thresholds such as 0.7 or 0.9. Governance And Regulation | mixed | Empirical support for the fairness threshold used to define disparate impact |
Reading fidelity
high
Study strength
medium
|
not reported
|
| The authors' case study of AI-mediated resume screening demonstrates that risk assessments should incorporate both consequence-severity assessments and uncertainty assessments to characterize the risk landscape accurately and transparently. Ai Safety And Ethics | positive | Accuracy and transparency of AI fairness risk characterization |
Reading fidelity
high
Study strength
medium
|
not reported
|
| The AI Risk Report Card is proposed as a standardized tool for reporting AI evaluation results that incorporates key risk-science principles and is intended to make results more complete, understandable, and useful to diverse stakeholders. Governance And Regulation | positive | Completeness, understandability, and usefulness of AI risk reporting |
Reading fidelity
high
Study strength
speculative
|
not reported
|