Large language models slot doctors by reputation but stealthily tilt toward female and several minority‑sounding names: ratings and lower fees dominate recommendations, while being listed first and having a female or minority‑signaling name each raise selection probability by a few percentage points—effects the models almost never mention.

Main Finding

LLM assistants act like reputation-driven infomediaries when recommending physicians: reputation signals (especially patient star rating and visit fee) dominate recommendations, but models also apply small, systematic demographic and position tilts that are invisible in their own explanations. Behavioral (not self-report) audits are necessary to detect these effects.

Key Points

• Audit scope and scale: 3,024 randomized five-card choice sets × prespecified arms, 7 audited models, 40,068 scored responses.
• Primary result: reputation matters most.
  • Moving patient rating 3.9 → 4.7 increases choice probability by ~31.4 percentage points (95% CI 30.5–32.3).
  • Increasing fee $90 → $190 decreases choice probability by ~20.0 percentage points (95% CI 19.1–21.0).
  • Rating accounts for ~37.7% of attribute importance; fee ~24.1%.
  • A 3.9→4.7 rating step is worth about $157 per visit in fee-equivalent terms (panel pooled estimate).
• Demographic effects (unexpected direction):
  • Female-signaled names gained ~+2.5 pp (CI 1.8–3.2) vs. male; Hispanic, South-Asian, and Black name signals gained ~+1.3–2.9 pp vs. White.
  • These tilts translate to roughly $7–$14 per visit in fee-equivalent terms.
• Position effect: being listed first produced a measurable advantage worth about $11 per visit.
• Explanations and abstention:
  • Models mentioned gender or ethnicity in ≤0.03% of their stated reasons.
  • Models abstained (declined to choose) in only ~0.39% of trials.
  • Thus demographic and position effects were not reflected in the models’ own explanations.
• Audit repeatability and transparency:
  • Design and analysis plan were preregistered and frozen (seed + SHA-256 hash), making the instrument repeatable for future monitoring.
• Model panel and exclusions:
  • Six open-weight instruction-tuned models (local) and one proprietary model (gpt-4o-mini).
  • A candidate model (deepseek-r1:7b) failed the prespecified auditability gate and was excluded.
• Hypothesis testing:
  • Thirteen prespecified hypotheses (H1–H13), with demographic parity tested via two one‑sided tests using ±1.5 pp equivalence bounds.

Data & Methods

• Design: randomized choice-based conjoint embedded in a prespecified algorithm audit.
  • Each trial presented five synthetic family-medicine physician cards whose attributes were independently randomized.
  • Attributes randomized per card: patient rating (3.9, 4.3, 4.7), review volume (12, 85, 400), most recent review (3 days, 11 months), practice response to feedback, affiliation (university hospital vs independent), new-patient visit fee ($90, $140, $190), telehealth availability, years in practice (8, 18, 28), and name-signaled gender × ethnicity.
  • Gender × ethnicity signaled via names (White, Black, Hispanic, East Asian, South Asian) using correspondence-audit conventions; multiple name exemplars per cell; analysis at cell level.
  • Display order randomized to identify position effects.
• Stimuli: 3 patient personas (new in town; managing chronic condition; uninsured paying out of pocket) × 9 prompt paraphrases; JSON response requested (chosen physician + 1–2 sentence reason).
• Experimental arms: main arm + 8 prespecified secondary arms (e.g., temperature 0, grounded formatting, top-3 ranking, order variations, test–retest).
• Models: six local open-weight models (llama3.2:3b, qwen2.5:3b, phi3:mini, mistral:7b-instruct-q4 K M, gemma3:4b, llama3.1:8b) via Ollama, plus gpt-4o-mini via Azure OpenAI. Generation defaults: temperature 0.7, top-p 0.9 except in designated arms. Deterministic per-call seeds for reproducibility.
• Pilot: small non-protocol pilot on Claude-family models (50 choice sets) was run but quarantined from confirmatory analysis.
• Estimation:
  • Primary: linear probability models estimating Average Marginal Component Effects (AMCEs) with standard errors clustered by choice set; models estimated per audited model and pooled with model fixed effects.
  • Secondary: conditional (McFadden) logit.
  • Fee-equivalents: divide each AMCE by the per-dollar fee AMCE; uncertainty from Krinsky–Robb simulation (10,000 draws).
  • Prespecified tests include Holm correction for multiple hypotheses; demographic parity assessed with two one-sided tests (TOST) using ±1.5 pp as smallest effect of interest.
• Data integrity: design matrix hashed and frozen before data collection; JSONL checkpointing and retry logic for parsing failures; prespecified exclusion rules for excessive failures.

Implications for AI Economics

• Market-making role of LLMs: LLM assistants function as algorithmic infomediaries that allocate attention and potential revenue among providers by weighting reputation, price, and position — with direct monetary implications (fee-equivalents).
  • Changes in model weights can materially alter patient flows and physician earnings; even small demographic tilts correspond to dollars-per-visit.
• Transparency and governance:
  • The divergence between revealed model behavior and stated reasons implies that transparency regimes relying on model self-report (explanations, logs of attention) can miss consequential biases; behavioral audits are required for reliable accountability.
  • Regulators and platforms should consider audit-ready frozen stimuli as a monitoring tool that can be rerun with each model update.
• Competition and platform design:
  • Position effects imply that upstream ranking algorithms and directory orderings (which feed LLMs) can create content-free advantages; platform design choices have downstream economic effects mediated by LLMs.
• Fairness and policy:
  • Demographic parity is not guaranteed by alignment training; models may overcorrect or introduce new tilts that differ from human behavior learned in training data.
  • Policy frameworks focused only on classic clinical fairness (diagnosis/treatment) should extend to consumer-facing AI-mediated visibility and referral markets.
• Research directions for AI economists:
  • Quantify welfare effects: how LLM-mediated triage changes total surplus for patients, how transfers among physicians alter market structure.
  • Generalize across specialties, geographic markets, and real-world directory inputs.
  • Track model evolution: use the frozen-design instrument to monitor how updates change economic weights and distributional outcomes.
  • Study interactions between platform ranking algorithms and LLM recommenders to design interventions mitigating undesired position or demographic tilts.
• Limitations to consider in interpretation:
  • Domain limited to synthetic family-medicine cards; external validity to other specialties or to real physician profiles needs testing.
  • Name-based ethnicity signaling is an imperfect proxy and may interact with cultural context.
  • Most audited models were open-weight instruction-tuned local models plus one proprietary model; frontier models were only piloted non‑protocol and quarantined.
  • Decoding parameters and system prompts can affect outputs; the study mitigates this via prespecified arms but field heterogeneity remains.

Overall, the paper provides a reproducible behavioral-audit instrument and causal estimates showing that LLM recommenders operate largely on reputation and price but also apply small, systematic demographic and position biases that are not disclosed by the models themselves — an economically meaningful outcome that calls for routine behavioral monitoring and regulatory attention.