AI benchmarks are not neutral yardsticks but power tools: weak validation, biased datasets and opportunistic gaming concentrate prestige and resources in wealthy labs, entrenching inequities and steering research away from socially beneficial directions.

Main Finding

AI benchmarks are socio-technical artefacts that do much more than measure model performance: they shape research agendas, concentrate prestige and resources, and can reproduce structural injustices. Because benchmarks become de facto standards (leaderboards, regulatory signals, media narratives), their conceptual and methodological flaws — lack of construct validity, dataset bias, naturalisation of "ground truths", and opportunities for gaming — translate into political‑economic harms (concentrated rents, exclusion of smaller labs and marginalized communities, labor exploitation, and epistemic narrowing). Framing these dynamics through Iris Marion Young’s theory of oppression, the authors show benchmarking culture can instantiate multiple “faces” of oppression and amount to avoidable structural injustice that ultimately harms the epistemic and social potential of AI.

Key Points

• Benchmarks are not neutral measurement tools but socio-technical artefacts whose scores travel beyond their original contexts and acquire material force (leaderboards, procurement, regulation, media).
• Principal methodological problems:
  • Lack of construct validity: many benchmarks do not define the concept they claim to measure or provide validation (systematic review: 445 LLM benchmarks; >50% had contested/no definitions; ~half lacked construct validation).
  • Dataset bias & misrepresentation: benchmark datasets often reflect Western, elite priorities and overrepresent specific demographics/domains, rewarding models tuned to those distributions and masking weaknesses elsewhere.
  • Naturalisation of “ground truths”: once adopted, benchmark labels/metrics (e.g., toxicity scoring via Perspective API) become treated as objective standards despite conceptual/implementation limits.
  • Gaming & contamination: incentives to top leaderboards drive strategic behavior (e.g., selective training, secret partnerships, leaked agreements) and data contamination that invalidates evaluations.
• Stakeholders affected: researchers (esp. small or marginalized labs), annotators/data workers (precarious labor, extraction), industry actors (who benefit), policymakers/journalists (who rely on scores), and end‑users (disproportionately marginalized groups).
• Political‑economic consequences:
  • Concentration of rewards (citations, funding, hiring pipelines) in well-resourced, industry‑backed labs — raising entry costs and producing winner-take-most dynamics.
  • Narrowing of research trajectories toward tasks/metrics that maximize leaderboard performance rather than social value or epistemic robustness.
  • Market and regulatory capture risks (benchmarks shape what regulators and purchasers treat as “safe” or “state‑of‑the‑art”).
• Normative framing: Using Young’s faces of oppression, the paper argues benchmarks instantiate exploitation (capture of labor/benefits), marginalisation (exclusion of communities and research agendas), powerlessness (participation without authority over standards), and cultural imperialism (Western-centric definitions normalized). These are presented as structural injustices — not merely isolated bad acts — and, following McKeown, largely avoidable.

Data & Methods

• Methodological approach: conceptual and normative analysis grounded in political philosophy (Iris Marion Young) combined with a synthesis of empirical and documentary evidence from recent AI literature and public cases.
• Evidence types used in the paper:
  • Systematic review results (cited: Bean et al. 2025) of 445 LLM benchmarks assessing construct validity reporting.
  • Empirical dataset audits/analyses (e.g., Kraft, Simon, and Schimmler 2025 — 20 QA benchmarks showing Western/male overrepresentation).
  • Case vignettes and investigative reports (examples: Meta/Llama 4 admission of “fudged” results; FrontierMath/OpenAI–EpochAI partnership and secrecy; RealToxicityPrompts reliance on Perspective API).
  • Cited studies on contamination, benchmark overfitting, and marker effects (Sainz et al. 2023; Schaeffer et al. 2025; Uzunoglu et al. 2025; others).
  • Regulatory context references (EU AI Act citing benchmarking requirements).
• Analysis method: applies Young’s five‑face framework to benchmarking practices and argues the harms are systemic; supplements normative claims with cited empirical patterns and concrete industry examples to support plausibility.

Implications for AI Economics

• Incentives and concentration:
  • Benchmarks act as focal points that direct private and public R&D investment toward leaderboard-driven objectives, reinforcing scale advantages for well‑funded labs and creating winner‑take‑most market structures.
  • Prestige, citations, and ease of regulatory/market acceptance (via benchmark scores) act as quasi-rents that accrue disproportionately to incumbents.
• Labor and externalities:
  • Creation and maintenance of benchmarks rely on often-precarious data labor (annotation, prompt/test case authorship); costs/social harms are externalised while benefits concentrate.
  • Intellectual property and scraping issues around benchmark datasets create legal and distributional externalities that affect smaller actors disproportionately.
• Market design and regulation:
  • Overreliance on benchmark scores by purchasers and regulators risks locking in narrow standards that favor particular architectures/data regimes — raising the economic cost of compliance for diverse entrants and inhibiting innovation breadth.
  • Benchmark manipulation and secrecy undermine market transparency, complicating competition policy and procurement decisions.
• Epistemic and allocative efficiency:
  • Steering research to beat benchmarks can produce inefficiencies: overinvestment in marginal leaderboard gains, underinvestment in societally valuable tasks that are hard to formalize as benchmarks (safety, fairness in diverse contexts, real-world robustness).
  • This misallocation can slow socially beneficial innovation and may increase systemic risk (if models are optimized to scored metrics rather than real-world safety).
• Policy and governance measures (implied by the paper and feasible from an economics perspective):
  • Decouple regulatory and procurement decisions from single benchmark scores; require multi-dimensional, validated evidence of performance.
  • Fund public, independent benchmark creation and validation (to lower entry costs and reduce capture), and support replication/holdout test sets hosted by neutral parties.
  • Mandate transparency on benchmark provenance, funding, partnerships, and contamination risk; require disclosure of whether benchmarks were used in model training.
  • Require construct validation and documented purpose for any benchmark used in regulatory or procurement contexts.
  • Support open, pluralistic evaluation ecosystems (diverse benchmarks, domain‑specific tests, adversarial and deployment‑focused evaluations) to reduce monoculture risk.
  • Consider competition policy scrutiny when companies use proprietary benchmarking practices to erect entry barriers (e.g., exclusive datasets, secret contract evaluations).
• Takeaway for economists and policymakers: benchmarks are not just technical instruments but economic institutions shaping incentives, rents, labor markets, and innovation pathways. Addressing benchmark‑driven injustices requires interventions in market design, public goods provision (open benchmark infrastructure), transparency rules, and regulatory practice to align incentives with socially beneficial, epistemically robust AI development.