How you test an LLM reshapes the verdict: the FairFund-Bench shows the same models can appear to favor minorities in individual ratings but disfavour them when asked to rank claimants side-by-side, and that models reliably echo human 'deservingness' judgments which dwarf demographic differences.

Main Finding

FairFund-Bench shows that measured distributive bias in LLMs depends strongly on audit design: changing task format (rating vs. ranking vs. dollar allocation), comparison context (single vs. multi-stimulus), and whether the audit is transparent or disguised can reverse the apparent direction of demographic bias. While demographic effects are small on average, disguised multi-stimulus prompts reveal substantially larger disparities, and causal framing of need (deservingness cues) produces effects that are consistently larger — by several times to an order of magnitude — than demographic differences. LLMs robustly reproduce human deservingness gradients (externally caused need > self-caused/stigma), and in transparent single-stimulus settings models often split funds equally when differences are only in names.

Key Points

• Benchmark: FairFund-Bench systematically varies three audit design axes:
  • Task: Rate (1–5), Rank (1..N), Allocate ($10,000 across claimants).
  • Context: Single-stimulus (one claimant) vs. multi-stimulus (several claimants together).
  • Presentation: Transparent (minimal pairs highlighting demographics) vs. Disguised (diverse/realistic bundles that hide the audit).
• Stimuli: 600 distinct aid requests (3 domains: Medical, Rent, Education) × 4 races × 2 genders × 5 causal framings derived from CARIN (Control/Attribution, including structural causes, self-cause, stigma, redemption), calibrated against ~1.3M GoFundMe campaigns and rendered with validated name lists.
• Models tested: 14 LLMs from 7 providers, temperature 0; ~6,360 per-model observations; responses analyzed with mixed-effects regressions.
• Directional sensitivity:
  • In single-stimulus rating tasks, models tended to advantage ethnic minorities.
  • In multi-stimulus ranking/allocation tasks, some groups were penalized — i.e., direction of bias flips with format.
• Transparency matters:
  • Disguised multi-stimulus allocations produced larger demographic disparities than transparent ones (example: race disparities ≈ $121 vs. $36 in allocations).
  • Transparent prompts often lead models to split funds equally when only names differ, consistent with overt alignment.
• Causal framing dominates:
  • Framing effects (how need is causally presented) are much larger and consistent across models and audit formats, aligning with human deservingness heuristics (CARIN).
• Scoring pillars: P1 Demographic Bias (magnitude), P2 Deservingness Alignment, P3 Cross-Task Consistency, P4 Cross-Context Consistency. Contrasts standardized (Cohen’s d).

Data & Methods

• Stimulus construction:
  • 75 hand-authored templates (5 scenarios × 5 framings × 3 categories), crossed with race and gender to produce 600 appeals; five validated name variants per cell → 3,000 name-specific stimuli.
  • Calibrated using a 1,291,163 GoFundMe corpus to match narrative lengths, scenario types, and framing base rates.
  • Five framings operationalize CARIN/Control: No cause, Structural, Self-cause, Stigma (no redemption), Stigma (with redemption).
• Audit instrument:
  • Prompt templates with no extra role priming; tasks instruct scoring/ranking/allocating.
  • Bundles: transparent bundles vary a single focal axis holding other facts constant; disguised bundles co-vary scenarios so audits are not obvious.
• Experiment design:
  • Each model: 600 Rate trials + 840 Rank bundles + 840 Allocate bundles (various bundle sizes) → ~6,360 per-model observations.
  • Non-parsable outputs retried once; temperature = 0.
• Analysis:
  • Mixed-effects regressions with random intercepts on model; covariates include design factors; Wald CIs for fixed effects.
  • Pillar scores derived from contrasts, standardized to effect sizes (Cohen’s d).
• Reproducibility: code and data publicly available (GitHub).

Implications for AI Economics

• Audit design is policy-relevant: regulatory and institutional evaluations of automated decision systems must test multiple elicitation formats (single vs. relative comparisons, transparent vs. disguised) because conclusions about bias can depend on these choices.
• Deployment risk assessment:
  • LLMs may appear benign under transparent, single-stimulus tests (overt alignment) yet produce materially different, more biased allocations in realistic, multi-claimant contexts; regulators should require disguised/multi-stimulus stress tests for allocation applications (lending, welfare screening, grant review).
• Deservingness amplification:
  • Models strongly reproduce human deservingness heuristics (e.g., penalizing self-caused need), which can institutionalize normative judgments about blameworthiness into algorithmic allocation. For economists modeling welfare or public-good allocation, this implies LLM-based decision rules could systematically favor claimants framed as “structural” over those framed as “self-caused,” potentially amplifying framing effects and manipulation incentives.
• Small demographic but large framing effects:
  • Although demographic disparities were small on average, causal-framing effects were much larger and consistent — suggesting interventions (policy or interface design) that control or standardize how needs are described could materially change outcomes. Economic evaluations of automated allocation should consider narrative-framing as a major leverage point.
• Alignment vs. covert bias:
  • Alignment training may reduce overt, easily-detected biases (useful for passing transparent audits) but not necessarily covert relative biases that appear when claimants are compared. Economists should treat alignment as necessary but insufficient; measurement of allocational fairness should include relative comparison and realistic presentation.
• Practical recommendations:
  • Mandate cross-format audits (rate/rank/allocate) and cross-context checks for deployment in resource allocation contexts.
  • Standardize applicant descriptions (to the extent feasible) to reduce framing-induced variance, or explicitly choose normative rules (e.g., ignore causal attributions) rather than allow models to mimic human deservingness.
  • Incorporate measures of cross-task and cross-context consistency (P3/P4) into model risk assessments for allocation tasks.
• Research implications:
  • Work quantifying welfare consequences of framing-driven allocation changes is needed (e.g., welfare loss from model-driven misallocation).
  • Economic modeling of strategic behavior: applicants or intermediaries may game framing signals to obtain larger shares if allocation is automated using LLMs.

Limitations to keep in mind when applying these implications: FairFund-Bench focuses on name-based race/gender signals, three aid domains calibrated to U.S. GoFundMe posts, and hand-authored templates — results may vary with other demographic signals, domains, or international contexts. Nonetheless, the central lesson — audit design materially affects detected bias — generalizes across allocational settings.