Autonomous learning bidders can learn to sustain supra-competitive prices in a stylised electricity market; multi-agent RL experiments show punish-and-forgive dynamics, sensitivity to agents' discounting/memory, and profitable unilateral deviations consistent with tacit collusion in specific scenarios. The effect appears robust across several synthetic environments but remains a plausibility result limited by the model's stylized scope and algorithmic choices.

Main Finding

Independent learning-based bidding agents (modeled with multi-agent reinforcement learning) can autonomously learn and sustain supra‑competitive outcomes in a stylized algorithmic electricity market. Using a multi-dimensional set of tests (beyond simple profit vs. Nash comparisons), the authors show that tacit collusion is a realistic risk in electricity markets under conditions that commonly occur in practice (e.g., transmission constraints, certain demand and cost regimes).

Key Points

• Hypothesis: Tacit collusion can emerge when electricity market participants use autonomous learning agents to set bids.
• Environment: A repeated, single‑timeslot market cleared by DC optimal power flow (DC‑OPF) with locational marginal prices (LMPs). Agents are four strategic generators in a 7‑bus stylized PJM‑derived test case.
• Agents’ information: after each clearing, each agent observes its own dispatch, profit, and the vector of nodal LMPs — a public signal that does not reveal competitors’ actions (imperfect public monitoring).
• Collusion identification: authors argue supra‑competitive profits alone are insufficient to claim tacit collusion and therefore adopt a multi‑criterion approach:
• Punishment of Deviation — does a unilateral forced competitive deviation get met by a punish‑and‑forgive response from others?
• Unsustainability under Shortsight — if agents are made short‑sighted (low discount factor and truncated memory) do collusive outcomes break down?
• Short‑term Profitability of Deviations — can an agent gain in the short run by deviating, and would iterated best responses (IBR) then drive profits below the collusive level?
• Experimental evidence: in some scenarios agents learned to sustain supra‑competitive prices and exhibited behaviors consistent with collusive patterns (e.g., punishment dynamics and profitability tradeoffs that discourage unilateral deviation), even though agents had no explicit collusion instructions or communication.
• Drivers: market characteristics (transmission constraints vs. copper‑plate, demand level, and heterogeneity of marginal costs) materially affect the likelihood and strength of emergent tacit collusion.

Data & Methods

• Market model:
  • Single timeslot DC‑OPF clearing over N buses with line flow and phase‑angle constraints; generators have piecewise‑linear segments and submit per‑segment bids.
  • Stage profit = LMP at generator bus × dispatched quantity − true generation cost.
• Agents and game formulation:
  • Players: strategic generators choose per‑segment bid prices each stage.
  • Repeated game with imperfect monitoring; each agent accumulates a history of its recent actions/observations and updates policy to maximize cumulative profit.
  • Learning framework: multi‑agent reinforcement learning (MARL) to model co‑evolving strategies (details of the specific MARL algorithm are provided in the paper).
• Experimental design:
  • Stylized 7‑bus test system (adapted from prior literature) with 4 strategic generators.
  • Full factorial: Grid vs NoGrid (transmission constraints enforced or relaxed), three demand levels (Low/Medium/High), three marginal‑cost heterogeneity cases (Low/Medium/High) → 18 scenarios.
  • All exogenous fundamentals fixed within scenarios (no temporal stochasticity); dynamics arise solely from strategic learning.
• Collusion tests and counterfactuals:
  • Punishment test: after MARL convergence, force one agent to play a short‑term profit‑maximizing bid (solve a bi‑level problem via KKT/MILP conversion) while others continue learning, and observe whether competitors respond with a punish‑and‑forgive pattern.
  • Shortsight test: re-run MARL with very low discount factor and truncated memory to check whether previously sustained supra‑competitive behavior collapses.
  • Short‑term profitability (IBR) test: run Iterative Best Response (solving the same bi‑level optimization for each agent sequentially) to check if unilateral deviations are initially profitable but lead, via competitors’ responses, to lower profits than the MARL outcome.
• Numerical tools: market clearing solved via DC‑OPF; bi‑level deviations transformed to single‑level MILP via KKT and binary variables (standard in the literature) for deviation and IBR analyses.

Implications for AI Economics

• Practical risk: Algorithmic bidding using learning agents can generate tacitly collusive outcomes even without explicit coordination. Market outcomes can be materially altered by the learning dynamics of algorithms.
• Multi‑criterion detection: Regulators and market monitors should not rely solely on elevated prices or profits vs. Nash benchmarks to flag collusion. A richer set of behavioral tests (e.g., punishment responses, sensitivity to horizon/memory, IBR dynamics) is needed to distinguish collusion from other causes of high prices.
• Monitoring and stress testing: Market operators should incorporate MARL‑based simulators and the authors’ diagnostic tests into routine stress tests and ex‑ante evaluations of new algorithmic trading strategies.
• Market design levers:
  • Increase competitive pressure where feasible (more active participants, reduce barriers to entry).
  • Reduce conditions that facilitate tacit coordination: less persistent repeated interactions, reduced observability of public signals that enable conditioning, or design rules that make punish‑and‑forgive strategies harder to sustain.
  • Consider regulatory instruments: auditing/validation of deployed automated bidding systems, transparency requirements, randomized clearing features, or transaction rules that disrupt simple collusive dynamics.
• Research directions: extend analysis to richer, stochastic multi‑period markets (unit commitment, storage, renewables), larger markets with more heterogeneous participants, alternative learning architectures, and empirical validation on real market data. Limitations of current study include the stylized 7‑bus setup, fixed exogenous conditions, and four strategic generators — all useful for controlled experiments but requiring broader validation for general policy conclusions.