Liability aimed at curbing algorithmic disparity can paradoxically limit disadvantaged patients’ access to AI and distort firm investment; higher legal exposure may later restore access by forcing firms to improve accuracy, and blanket equal-accuracy mandates can worsen outcomes by skewing incentives.

Main Finding

Liability rules that assign clinicians responsibility for harms caused by disparate clinical algorithms can produce unintended equilibrium effects across the AI lifecycle. Specifically, disparity-contingent liability can (i) reduce physician use of AI for disadvantaged patients (creating disparate access), (ii) have a non-monotone effect as liability increases—initially deterring use but, beyond a threshold, inducing firms to reallocate development toward the disadvantaged group or to adopt equal-accuracy designs which restore or increase use—and (iii) make a priori mandates that force equal measured accuracy across groups potentially welfare-reducing for both groups by distorting firm investment and downstream clinical deployment.

Key Points

• Setting and actors

  • Two-stage game: (1) an AI firm chooses per-group accuracies ρx (advantaged group) and ρy (disadvantaged group) at quadratic costs κx < κy; (2) a physician (the accountable decision-maker) decides whether to deploy AI for a case and whether to follow its recommendation.
  • Patient types t ∈ {x,y} are observable; the physician has case-specific uncertainty α ∼ U[0,1] and updates via Bayes on the AI signal with accuracy ρt ∈ (1/2,1).
  • Physician faces patient benefit b from correct treatment, per-use patient cost c, per-use physician benefit r (reimbursement/other returns), and liability ℓ when she follows a disparate AI’s incorrect recommendation for a disadvantaged (type-y) patient. Physician places weight θ ≥ 0 on nonclinical objectives (so θ > 0 is the main case).
  • Firm receives per-use payment f and incurs development cost κt(ρt − 1/2)^2 for each group t; κy > κx captures higher cost of improving performance for the disadvantaged group (data sparsity, measurement friction).

• Core mechanism and results

• Disparate use induced by liability: Holding algorithmic accuracy fixed, liability tied to disparate errors raises the physician’s expected downstream exposure for disadvantaged patients. This acts like a shadow cost of reliance and shrinks the set of patients for whom the physician consults AI—i.e., AI use declines more for disadvantaged patients, causing disparate access even if the algorithm itself is unchanged.
• Non-monotone effect of increasing liability: For small increases in ℓ, deterrence dominates and AI use for disadvantaged patients falls. As ℓ increases further, the firm optimally reallocates investment toward improving ρy (or switches to an equal-accuracy design), which lowers physician exposure and can restore or increase AI use for disadvantaged patients relative to the intermediate-liability level. Thus the physician’s equilibrium use for disadvantaged patients is non-monotone in liability.

• Perverse effects of equal-accuracy mandates: For κy > κx, mandating equal measured accuracy can induce the firm to reallocate resources away from the low-cost (advantaged) group toward the high-cost group in ways that reduce overall welfare: advantaged-group accuracy can fall substantially while disadvantaged-group accuracy rises only modestly. Combined with per-use reimbursements (r, f), this can incentivize inappropriate or excessive AI use and harm both groups. Equalizing measured accuracy therefore does not guarantee equalized or improved realized access/outcomes.

• Additional modeling features

  • Liability modeled as applied to clinicians (consistent with CMS §92.210 enforcement at point of use); when algorithm is equal-accuracy, the disparity-triggered liability does not apply.
  • Physician knows ρt and uses Bayesian updating (posteriors Q1|1, Q1|2).
  • The model abstracts from malpractice and other legal channels to isolate the disparity-contingent liability mechanism.
  • Extensions in the paper (summarized) include endogenous pricing and type-dependent priors; main qualitative insights are robust.

Data & Methods

• Type of study: Analytical theoretical model (game-theoretic, two-stage principal–agent style model linking product design and intermediary adoption).
• Methods and tools:
  • Bayesian updating of physician beliefs given AI signals.
  • Profit-maximizing firm choosing continuous accuracy choices ρx, ρy under quadratic development costs κt(·).
  • Physician decision rule trading off expected patient benefit, per-use cost c, reimbursement r, and expected liability ℓ weighted by θ.
  • Comparative statics and equilibrium characterization to derive how firm design and physician adoption respond to parameter changes (ℓ, κy/κx, r, f, θ, β).
  • Welfare comparisons across policy regimes (no liability, disparity-contingent liability, equal-accuracy mandate).
• Data: None—no empirical dataset; the contribution is conceptual and analytical.
• Key assumptions and limitations:
  • Patient type is observable and priors are type-independent (α ∼ U[0,1]) in the base model to isolate algorithmic disparity effects.
  • Development cost separability and quadratic cost form; κy > κx is central.
  • Liability is applied only when an incorrect recommendation from a disparate algorithm harms a disadvantaged patient; other liability channels (e.g., malpractice) are abstracted away.
  • Model is static (no dynamic firm learning from usage) and assumes fixed historical training datasets.

Implications for AI Economics

• Policy design: Point-of-use liability can change both downstream adoption and upstream product design. Regulators should expect non-obvious equilibrium responses:
  • Weak-to-moderate disparity-triggered liability may paradoxically reduce disadvantaged patients’ access to AI (greater deterrence for clinicians).
  • Stronger liability can be effective if it credibly forces firms to internalize disparity costs and invest in disadvantaged-data collection or equalize performance.
  • Mandated equal measured accuracy is not a panacea; without attention to asymmetric improvement costs and downstream incentives, it can reduce welfare and produce perverse deployment patterns.
• Regulatory complements:
  • Combine disparity-sensitive liability with incentives/subsidies for collecting and labeling disadvantaged-group data (lower κy) so firms can improve ρy without degrading ρx.
  • Align reimbursement and payment structures (f, r) to avoid incentivizing excessive or inappropriate AI use; consider tying reimbursement to demonstrated clinical benefit, not just per-use deployment.
  • Use monitoring, outcome-based evaluation, and deployment guidelines to ensure that measured parity translates into realized, equitable benefit (not just measured equality).
  • Consider reallocation of legal accountability (for example, partial responsibility for developers) and transparent reporting of per-group performance to more directly internalize upstream design externalities.
• Research directions for AI economics:
  • Empirical tests: exploit staggered rollouts or policy changes (e.g., CMS §92.210 implementation, CPT/NTAP reimbursement changes) to measure (a) clinician AI-use changes by patient group, and (b) firm-level changes in model design or retraining focused on disadvantaged groups.
  • Dynamic learning extensions: model how downstream usage generates data that can lower κy over time and how that interacts with liability and deployment.
  • Heterogeneous physicians: extend to variation in θ, risk aversion, or information precision to study distributional impacts within providers.
  • Alternative liability assignments: model comparative effects of assigning liability to firms versus clinicians, and hybrid schemes that condition liability on monitoring/reporting practices.
• Practical takeaway for scholars and policymakers: regulating algorithmic fairness at the point of use alters market incentives across design and deployment. Effective policy should be multi-instrument: pair liability that penalizes harmful disparate use with instruments that reduce firms’ costs of improving disadvantaged-group accuracy and that align deployment incentives (reimbursement, monitoring) with equitable health outcomes.

Paper: Luan, Singh, Dai (2026), "Algorithm Design and Physician Liability" (arXiv:2608.13618).