AI can keep grant pipelines open while destroying public trust: a game-theoretic model finds automated review may create ‘zombie’ funding agencies that process grants without legitimacy, and small early shocks or policy signals (like emergency reviewer pay) can tip identical institutions toward recovery or collapse.

Main Finding

A co-evolutionary evolutionary-game model shows that automating scientific peer review with AI can preserve operational capacity while simultaneously destroying—or failing to restore—public legitimacy. Capacity (the ability to process proposals) and legitimacy (public acceptance of funding decisions) follow separate dynamics and can therefore fail independently. The system is path-dependent and stochastic: small early differences or modeling choices can push identical institutions to very different outcomes (full recovery, joint failure, or “zombie” agencies that process grants but lack legitimacy). Policy levers such as reviewer pay and emergency AI deployment can stabilize capacity but may erode legitimacy or backfire under politicization and collective scientist resistance.

Key Points

• Model structure
  • Three interacting actors: Scientists (strategic population), Public (strategic population), and a Funding Agency (adaptive controller).
  • Agency controls two levers: AI adoption for reviews (φ ∈ [0,1]) and reviewer pay (ρ ∈ [0,1]) under a finite budget.
  • Scientists choose among four strategies: Normal participation (N), Refusal to review (R), Proposal intensification/flooding (P), or Both refusal+flooding (B).
  • Public chooses among Acceptance of AI (A), Public rejection (J), or Litigation (S).
  • Strategy updating follows pairwise-comparison Fermi (imitation) dynamics; agency adapts to backlog and budget constraints.
• Principal qualitative findings (from analysis and simulations)
• Capacity and legitimacy are analytically separable. AI can restore throughput without restoring or even while undermining legitimacy.
• Legitimacy dynamics are bistable: outcomes depend on public acceptance of AI; multiple equilibria exist.
• Resistance cascades are easy to ignite because career costs for refusing to review are often low; collective-action benefits can make refusal self-reinforcing.
• Increasing reviewer pay helps only within strict budget-solvency limits; sudden “emergency” pay increases can be interpreted by the public as bribes and accelerate legitimacy loss.
• Finite-population (stochastic, individual-based) dynamics show high path dependence: identical starting conditions can produce divergent fates depending on the noise model. Under a detailed individual-based model authors report nearly universal backlog formation; the key stochastic variable is whether public trust recovers. Under coarser approximations, about half recover and ~1/6 become “zombie” agencies.
• Additional mechanisms highlighted
  • Proposal flooding (scientists retaliating by submitting many proposals) can outpace AI capacity and collapse the pipeline.
  • Litigation by the public can rollback AI deployment and affect budgets and legitimacy.
  • Politicization (exogenous parameter G) lowers the threshold for collective resistance and amplifies risks.

Data & Methods

• Type of study: Theoretical/modeling — no primary empirical dataset.
• Formalism:
  • Evolutionary game-theoretic model with co-evolution of two strategic populations (Scientists, Public) and an adaptive controller (Funding Agency).
  • Strategy frequencies evolve via Fermi (pairwise-comparison) imitation dynamics; weak-selection approximations (replicator-like) derived analytically; full Fermi dynamics used for numerical integration.
  • Funding Agency computes required AI deployment φ_req based on human-reviewer deficit and proposal load: φ_req = clip[(r + χ p) / β_unit, 0, φ_max] (schematic from text), and adjusts φ and reviewer pay ρ subject to budget and litigation; χ is proposal-load multiplier, β_unit is review capacity per unit AI.
  • Payoff functions for scientists and public are specified (see Table 1 in paper): scientists balance career baseline, pay from reviewing, costs for refusal, collective-action benefits of resistance, and reputational/legitimacy terms; public payoffs depend on institutional legitimacy L, AI-review legitimacy LA, politicization G, visible scientist resistance r, and costs of rejection/litigation.
• Analyses:
  • Analytical results (Appendix I) identify thresholds, separability of capacity vs legitimacy, and conditions for bistability.
  • Numerical simulations of deterministic ODEs and finite-population stochastic individual-based models explore dynamic outcomes across parameter sweeps (e.g., politicization, refusal cost, reviewer pay, AI capacity).
• Key parameters of interest (examples): politicization G, proposal multiplier χ (default 1.5), AI technical ceiling φ_max, AI per-unit capacity β, reviewer-pay responsiveness, selection strength λ in Fermi update, litigation cost c_S, collective-action benefits ω.
• Limitations noted by authors:
  • Stylized payoff functions and simplified actor types; many real-world institutional details abstracted away.
  • Parameters not empirically calibrated — model intended to identify qualitative regimes and priorities for empirical work.
  • Funding agency modeled as a non-strategic adaptive controller rather than as a full strategic actor embedded in political economy.

Implications for AI Economics

• Distinct value channels: efficiency gains from automation (reduced processing time, lower marginal review cost) do not automatically translate into social value if legitimacy falls; economic appraisal of AI should include legitimacy externalities and trust capital as first-order effects.
• Public goods & collective-action externalities: peer review is a public-good sustained by volunteer effort; economists should model how automation changes incentives for voluntary contributions and how retaliation/flooding externalities affect overall welfare and budget dynamics.
• Path dependence and multiple equilibria: policy interventions (timing, transparency, incentive design) matter critically. Early small differences can produce large welfare divergences — making precautionary governance and early legitimacy-building high-value investments.
• Cost-benefit rethinking of reviewer pay: paying reviewers to sustain capacity may look beneficial on capacity metrics but can create negative signaling (perceived bribery) that destroys legitimacy and reduces long-term social returns. Optimal incentive design must internalize signaling effects.
• Stochastic fragility and regulation: finite-population stochasticity implies that probability distributions over outcomes (not just expected values) must be considered in regulatory and budgetary planning; tail risks include zombie institutions that process funds but lose societal authority.
• Design and governance recommendations (for AI economics and policy design)
  • Include legitimacy metrics in economic evaluations of AI in public institutions; quantify willingness-to-pay for legitimacy preservation.
  • Prioritize transparent, explainable, and participatory AI deployment (ex ante engagement with scientists and publics) to shift bistable legitimacy basins toward acceptance.
  • Simulate dynamic scenarios with endogenous strategic agents (scientists, citizens) when assessing automation policy — static cost-efficiency analyses will miss key externalities.
  • Prepare contingency plans for “proposal flooding” and litigation (throttling submission rates, triage mechanisms) and model their fiscal and welfare impacts.
  • Empirically estimate key parameters (politicization, cost of refusal, public sensitivity to pay signals, AI per-unit capacity) to calibrate dynamic models and inform robust policy design.
• Research priorities for AI economics
  • Empirical calibration: measure how public legitimacy responds to varying degrees and types of automation and to reviewer-pay changes.
  • Welfare analysis: quantify long-run social welfare losses from legitimacy collapse, including effects on innovation outputs, private R&D investment, and human capital flows.
  • Mechanism design: explore incentive schemes and transparency protocols that sustain human reviewer participation while allowing safe, legitimacy-preserving AI assistance.
  • Stochastic policy evaluation: use agent-based and finite-population models to estimate probabilities of adverse regimes and optimal timing/scale of interventions under uncertainty.

Summary takeaway: automation can solve throughput problems but can produce large negative legitimacy externalities that conventional economic cost–benefit analyses miss. AI adoption in publicly funded peer review must be governed not just by technical capacity and budget calculus but by models that internalize collective-action, signaling, political polarization, and stochastic path dependence.