Clinician overrides are valuable preference signals, not noise; jointly learning a reward model and clinician capability from longitudinal override and outcome data can align clinical AI to patient trajectories and prevent the systematic suppression of correct-but-difficult recommendations. Value-based chronic care—with dense longitudinal outcomes and natural capability variation—offers especially favorable conditions for this approach.

Main Finding

Clinician overrides of clinical AI recommendations should be treated as structured, high-value preference signals—not noise. In value-based chronic care, override data can simultaneously train (1) a reward model that captures which clinical actions maximize patient-outcome–plus–economic objectives (conditioned on patient state s and contract context c), and (2) a capability model that captures which clinicians can reliably evaluate and execute those recommendations (κ). A dual-learning architecture that alternates reward and capability updates and weights clinician feedback by capability (β(κ)) avoids suppression bias, where correct-but-difficult recommendations vanish because clinicians lack execution skill.

Key Points

• Reframe overrides: Overrides are labeled preference pairs (in context) analogous to RLHF but richer—domain experts, real consequences, and observable downstream outcomes.
• Conditioning matters: Preferences must be modeled as P(prefer a' over r | s, c, κ). Contract context c (e.g., fee-for-service vs. outcome-based payment) fundamentally changes the reward for the same clinical action.
• Five-category override taxonomy (distinct update targets):
• Context — clinician has private state; expand state representation S.
• Judgment — genuine disagreement about clinical value; update reward model Rθ.
• Workflow — pragmatic constraints (delay/ordering); filter as noise for reward updates.
• Protocol — institutional rules; expand context c to encode protocols.
• Capability — clinician lacks skill/confidence; update capability model κφ and trigger scaffolding.
• Capability-weighted preference model: introduce clinician capability κd_k(t) ∈ [0,1] per clinician, per domain, time-varying. Use a capability-dependent temperature β(κ) (paper uses β(κ) = β0 + β1·κ) so overrides from high-κ clinicians have larger influence on the reward model.
• Modify action is especially informative: unlike binary accept/reject, modify(a') provides (i) a direct preference pair, (ii) a proximity signal in action space, and (iii) a direction for improvement — all valuable for reward learning.
• Dual learning & alternating optimization:
  • Reward model loss LR(θ) aggregates clinician feedback weighted by β(κ).
  • Capability model loss Lκ(φ) measures how well clinician behavior matches the current reward model.
  • Alternate fixing one model and optimizing the other; this disentangles low capability from low reward and prevents suppression bias.
• Value-based chronic care is a favorable environment: longitudinal density (many repeated interactions), outcome observability (follow-up outcomes), concentrated decision space, and natural variation in clinician capability make κ identifiable and reward learning feasible.
• Operational provenance: framework emerged from real-world deployment under outcome-based contracts where naive treatment of overrides led to disappearance of correct recommendations and other pathologies.

Data & Methods

• Interaction record: each event It = (st, rt, δt ∈ {accept, modify(a′), reject}, at, clinician k, context c, observed outcome ot+Δ).
• Reward model: parameterized Rθ(s, a, c). Critical novelty: reward is contract-conditioned so the same action a can have different value under different payment/incentive regimes.
• Preference formulation: Bradley–Terry style ordinal model extended with s, c, and κ. For example,
  • P(a′ ≻ r | s, c, κ) ∝ σ(β(κ) · [R(s, a′, c) − R(s, r, c)]).
  • β(κ) = β0 + β1·κ (linear form used for tractability; alternatives possible).
• Capability model: κφ(k, d, t) is a learned, per-clinician, per-domain, time-varying competence scalar. Conceptually decomposed into κexec (execution skill) and κalign (ability to judge alignment), though the framework primarily operationalizes a scalar for weighting.
• Losses and optimization:
  • LR(θ) = Σ_it β(κ_dt_kt) · log P(δt | st, rt, at, c; Rθ) — reward updates weighted by capability.
  • Lκ(φ) = Σ_it log P(δt | st, rt, κφ(kt, dt, t); Rθ*) — capability estimated conditional on current reward model.
  • Alternating updates: estimate κ given fixed R, then update R given estimated κ.
• Inference of override type: override types are latent and must be inferred from patterning across patients, clinicians, time, and outcome consequences (e.g., consistent referrals across clinicians in same site → protocol; single clinician consistent referrals for a domain → capability).
• Data requirements and challenges:
  • Requires longitudinal patient trajectories with outcome labels and clinician identifiers.
  • Needs structured logging of action as executed (not just accept/reject).
  • Causal attribution is nontrivial—outcome-following behavior may be confounded by unobserved state; careful causal inference and potential instrumental-variable or design-based strategies are needed.
  • Estimating κ and disentangling it from unobserved patient heterogeneity requires cross-clinician, repeated interactions on overlapping patient types.
• Practical system components described: telemetry for overrides and modifies, scaffolding/interventions for capability deficits, outcome collection pipelines, and model-training infrastructure to close the loop and produce a compounding flywheel.

Implications for AI Economics

• Incentives shape learning: outcome-based payment environments create the contract-conditioned signals necessary to learn rewards aligned with long-term patient trajectory rather than encounter economics. Without aligned financial incentives, learned rewards risk tracking short-term, billable actions.
• Investment trade-offs: Platforms should invest in clinician capability-building (training, scaffolding, supervision) because capability improvements both (a) improve patient outcomes and (b) increase the informativeness of feedback used to refine the reward model — producing compounding returns on platform quality.
• Risk of perverse automation if capability ignored: Naive preference learning can cause suppression bias — correct but difficult recommendations disappear because many clinicians override them for capability reasons. This can institutionalize lower-quality care in models and distort contract pricing and risk adjustment.
• Provider heterogeneity matters economically: Measuring κ enables more granular valuation of clinician labor (e.g., where to allocate higher-skilled clinicians, where to deploy decision support or specialist consults) and supports targeted upskilling investments with an economic ROI calculable from model-implied outcome improvements.
• Contract design & policy: Payor and regulator choices (e.g., encouraging outcome-based contracts, enabling outcome measurement) materially affect what AI systems can learn and thus the welfare gains from clinical AI. Policies that expand longitudinal outcome measurement and align reimbursement with outcomes widen the signal horizon for aligned AI.
• Platform competition and flywheel effects: Platforms deployed in VBC settings that successfully learn capability-weighted rewards can both improve outcomes and reduce costs, creating a data-driven advantage (more informative overrides, better reward models, better scaffolding), which has implications for market concentration and contract negotiation power between providers, payors, and platforms.
• Measurement & evaluation: Economic evaluations of clinical AI must account for the dual learning loop — returns to model improvement depend on capability investments and contract alignment; naive A/B evaluations that ignore clinician capability heterogeneity may under- or over-estimate benefits.
• Cautionary notes for governance:
  • Accountability and transparency: capability estimates and how they weight clinician input should be interpretable and auditable to avoid unfair penalization of clinicians working in resource-limited settings.
  • Distributional concerns: If models preferentially recommend actions easier for already-high-κ clinicians, care disparities could widen unless capability-building is prioritized.
  • Data sufficiency: The approach depends on rich longitudinal data and identifiable clinician variation; small or fragmented practices may not generate the necessary signal without coordinated measurement.

Overall, the paper argues that integrating clinician capability and payment context into preference learning turns clinician overrides from nuisance into a high-value signal. For AI economics, this reframing implies that payment models and investments in clinician capability are economic levers that determine both what AI systems can learn and the downstream value those systems create.