Major League umpires favor high-performing pitchers: pitches outside the strike zone are more likely to be called strikes when the pitcher had stronger prior-season performance, and this reputational anchoring shifts with game context—rising when run expectancy is high but falling later in games and when batters already have strikes.

Main Finding

Umpires' ball-and-strike calls show anchoring-based performance-evaluation bias (PEB): pitchers with stronger prior-season performance receive more "gifted" strikes (calls for strikes on pitches objectively outside the strike zone). Situational factors alter this bias—some attenuate it and others intensify it—so fatigue and decision stakes do not have uniform effects on evaluators' reliance on prior-performance anchors.

Key Points

• Data: 1,452,180 ball-and-strike calls by Major League Baseball umpires (2015–2018) matched to objective pitch-location data.
• Core result: Higher prior-season pitcher performance (measured by WAR) predicts a greater likelihood that umpires call out-of-zone pitches strikes (anchoring-based PEB).
• Moderators tested:
  • Inning (proxy for fatigue): contrary to expectation, PEB decreases in later innings.
  • Run expectancy (proxy for stakes): PEB increases when run expectancy is higher.
  • Strike count (batter has 1 or 2 strikes): PEB decreases when the batter already has strikes.
  • Batter All-Star status (reputation/status): no significant moderating effect.
• Interpretation: Anchoring on prior performance is a central cognitive mechanism driving evaluative bias among even highly trained professionals, but situational context (types of stakes, temporal factors) can either amplify or reduce reliance on that anchor.
• Contribution: Extends status/reputation literature by demonstrating prior-performance effects against an objective benchmark and offering a context-sensitive account of PEB.

Data & Methods

• Sample: 1,452,180 pitch calls from MLB games, 2015–2018.
• Benchmark: Computerized pitch-location data determine whether a pitch was objectively inside or outside the strike zone.
• Dependent variable: Gifted strike indicator — umpire called a strike on an objectively out-of-zone pitch.
• Key independent variable: Pitcher prior-season performance (Wins Above Replacement, WAR).
• Moderators/controls: Inning (time in game), run expectancy (game-state stakes), current pitch/at-bat strike count, batter All-Star appearances (status), and other standard controls for pitch/game context.
• Statistical approach: Generalized linear models (logistic-type models) testing the association between prior-season performance and gifted strikes and interactions with situational moderators.
• Robustness: Large-sample analysis with objective ground truth allows precise estimation of bias patterns among professional evaluators.

Implications for AI Economics

• Human evaluators anchor on historical performance even when objective, moment-level data are available. In AI/economics contexts (e.g., model evaluation, forecasting, human-in-the-loop systems), anchoring on prior model or agent reputation can bias assessments of current outputs.
• Context matters: Different forms of decision stakes (e.g., economic cost vs. time pressure) and temporal factors (fatigue) can have opposing effects on bias. Policy and system design should not assume uniform moderation by “stakes” or workload.
• Design recommendations:
  • Use objective benchmarks and automated metrics where feasible to reduce reliance on reputational anchors.
  • Randomize or withhold prior-performance signals during evaluations when unbiased assessment of current outputs is needed.
  • Monitor and mitigate contextual drivers of bias—e.g., adjust oversight or review protocols under high run-expectancy/ high-stakes situations where anchoring may increase.
  • Incorporate debiasing training and decision aids for human evaluators to reduce anchoring (e.g., structured checklists, anonymized evaluations).
  • Model human-in-the-loop behavior: include anchoring priors and context-dependent moderator effects when simulating evaluator decisions or designing incentive schemes.
• Research implications:
  • Replicate in other high-stakes evaluation settings (hiring, credit underwriting, peer review) to map generalizability.
  • Test interventions (signal removal, debiasing nudges, automated adjudication) and measure effects across different stake types and temporal loads.
  • Incorporate measured fatigue and richer stake metrics to better predict when anchoring will intensify or attenuate.
• Economic modeling: When designing markets or platforms that rely on reputation signals, account for asymmetric and context-dependent evaluator biases; reputational advantages can persist or distort allocation of opportunities even with objective performance information available.