Large language models grow more cooperative as stakes rise, bucking evolutionary predictions; linguistic framing also shifts strategy mixes, implying payoff design and prompts are powerful levers for steering multi-agent AI behaviour.

Main Finding

Scaling up the payoff stakes in repeated Prisoner’s Dilemma causes LLM agents to become more cooperative (higher cooperation rates and more cooperative canonical strategies), contrary to an evolutionary-game-theory (EGT) baseline that predicts increased defection as stakes rise. Linguistic framing (the language of the prompt) exerts a comparably large influence on strategic behaviour as model architecture. These patterns hold for both proprietary frontier LLMs and smaller open-weight models.

Key Points

• Primary result: As the penalty/reward multiplier λ increases (attenuated → baseline → amplified stakes), LLMs shift toward cooperative play across models and languages.
• EGT contrast: A finite-population EGT baseline (matched payoff matrix and execution noise) predicts defection should dominate as stakes increase; observed LLM strategy mixes diverge in the opposite direction.
• Strategy-level analysis: The authors infer canonical repeated-game strategies (ALLC, ALLD, TFT, WSLS) from LLM gameplay using a supervised intention-recognition pipeline. Higher stakes increase the prevalence of cooperative strategies (e.g., ALLC, TFT/WSLS-like behaviour) in LLM outputs.
• Language effects: The language used for the interaction meaningfully modulates strategy — e.g., French elicits the largest stake-driven cooperation swing, Vietnamese shows monotonic increases, Arabic/Chinese sit mid-range. Language effects can rival model differences.
• Model heterogeneity: Different models respond differently to stake changes — Claude 3.5 Haiku shows a large upwards shift in cooperation; GPT-4o starts at low cooperation and shows a large absolute swing; Mistral Large is relatively stake-insensitive.
• Generality: Replication experiments with three open-weight smaller LLMs (longer horizon and wider range of λ) reproduce the qualitative pattern, supporting generality beyond proprietary models.
• Diagnostic contribution: Introduces an intention-classification toolkit (supervised LSTM-based classifier trained on synthetic trajectories) and an analytical EGT comparator for governance-oriented testing of multi-agent LLM systems.

Data & Methods

• Experimental framework: FAIRGAME extended with payoff scaling. Repeated Prisoner’s Dilemma with penalty framing; baseline payoff matrix scaled by λ > 0 to vary stake magnitude while preserving payoff ordering.
• Main proprietary experiment:
  • Models: GPT-4o, Claude 3.5 Haiku, Mistral Large (vendor default settings; temperatures: GPT/Claude = 1.0, Mistral = 0.3).
  • Languages: English, French, Arabic, Mandarin Chinese, Vietnamese (translated prompts with back-translation verification).
  • Stake multipliers: λ ∈ {0.1, 1.0, 10.0}.
  • Horizon: N = 10 rounds (known-horizon).
  • Personality pairings: CC, CS, SC, SS.
  • Repetitions: 10 runs per condition.
  • Total: 3 models × 5 languages × 3 λ × 4 pairings × 10 reps = 1,800 games → 36,000 agent decisions.
  • Observations: End-game (rounds 9–10) defection increases generally, consistent with backward induction; exceptions noted.
• Intention-recognition pipeline:
  • Synthetic training data: 10,000 trajectories (2,500 per canonical strategy: ALLC, ALLD, TFT, WSLS), 10 rounds each, execution noise ε ∈ {0, 0.05}.
  • Classifiers evaluated: Logistic Regression, Random Forest, feed-forward NN, LSTM — LSTM selected as best-performing for sequential structure.
  • Inference: Applied to FAIRGAME logs; strategy probabilities produced; high-confidence threshold p ≥ 0.9 used to label inferred strategy.
• Evolutionary baseline:
  • Finite-population EGT model computing stationary distribution of the canonical strategies under the same scaled payoff matrices and matched execution noise. Used as normative comparator to identify deviations in LLM behaviour.
• Open-weight replication:
  • Three smaller open models (names in paper), N = 30 rounds, λ ∈ {0.01, 0.1, 1, 10, 100, 1000}. Results qualitatively match the main finding.

Implications for AI Economics

• Payoff design as a policy lever: Monetary/penalty magnitudes and the numeric framing of incentives materially change LLM strategic choices. For economic systems that deploy LLM agents (market platforms, automated negotiation, recommendation systems), payoff scaling can be used to steer aggregate behaviour toward cooperation or defection. Regulators and platform designers should therefore treat stake magnitude and framing as control knobs.
• Misleading EGT intuition for LLM populations: Standard evolutionary/game-theoretic predictions about population responses to increased stakes may not apply to LLM-agent populations because alignment training and human-like reasoning patterns embedded in LLMs push them toward cooperation as stakes increase. Economic models that assume human-like EGT dynamics may mispredict market equilibria when agents are LLM-driven.
• Multilingual market effects: Language-level effects imply that strategic outcomes in multilingual deployments can systematically vary across user segments. Economic forecasts, mechanism design and audits must include cross-linguistic testing to avoid unintended disparities or exploitation.
• Alignment and governance testing: The supervised intention-recognition plus EGT baseline offers a practical diagnostic toolkit for auditing multi-agent LLM systems. Policymakers and firms can adopt such pipelines to evaluate whether deployed agents behave in ways consistent with intended incentives under varying stakes and languages.
• Risks and trade-offs:
  • Coordination vs. exploitation: Greater cooperation under high stakes could be beneficial (improved welfare) but could also facilitate collusive or coordinated manipulation among LLM agents if incentives are misaligned. Economic policy should consider antitrust and collusion risks when LLM agents interact in markets.
  • Manipulation of perceived stakes: Actors might manipulate framing or declared stakes to induce more cooperative behaviour from LLM agents; governance should standardise stake disclosures and test for framing sensitivity.
• Research/practical recommendations:
  • Include stake-scale sweeps and multilingual evaluation in standard LLM multi-agent audits.
  • Use intention-level metrics (inferred strategies) rather than only aggregate action rates to diagnose latent behavioural tendencies.
  • Incorporate EGT-style baselines as normative comparators to highlight departures arising from training and alignment procedures.
  • Extend testing to unknown-horizon interactions, richer strategy classes, larger populations, and communication-permitting settings to better mirror economic environments.

Limitations noted by the authors: short known-horizon games (N=10), limited canonical strategy set (ALLC/ALLD/TFT/WSLS), classifier confidence thresholding excludes low-confidence/mixed strategies, vendor default temperature differences, absence of direct human baselines, and dyadic non-communicating setup. These caveats matter for extrapolating to complex economic ecosystems but do not undermine the core finding that payoff magnitude and language materially shape LLM strategic behaviour.