Reinforcement‑learning employers on online labor platforms can tacitly collude on wages — but that risk is concentrated when few firms compete and disappears if common training techniques like experience replay are used.

Main Finding

When firms on online labor platforms use state-of-the-art reinforcement learning (a deep Q-network, DQN) to set posted wages, they can learn tacitly collusive wage-setting behavior that harms workers — but this outcome is conditional. Collusive (non-competitive) wage outcomes emerge robustly in simulation when the number of competing firms is small and when a common DQN training technique (experience replay) is not used. The qualitative result holds across many variants of platform and model design.

Key Points

• Setting: repeated matching on online digital labor platforms where firms post wages and heterogeneous workers apply based on posted information.
• Strategic learning: firms independently update posted wages using reinforcement learning (DQN); there is no explicit coordination or communication.
• Main mechanism: with small numbers of competitors and without experience replay, the learning dynamics converge to tacitly collusive wage policies (i.e., joint outcomes different from the competitive benchmark).
• Role of experience replay: including experience replay in DQN training disrupts the temporal correlations that facilitate collusive dynamics and prevents the emergence of collusion in the authors’ simulations.
• Competition intensity matters: as the number of competitors increases, collusive outcomes are less likely; the effect depends on market structure.
• Robustness: the collusion result is robust to many modeled features, including different platform designs and worker heterogeneity/search behavior.

Data & Methods

• Approach: agent-based simulation / multi-agent reinforcement learning framework (no empirical field data).
• Agents:
  • Firms: use reinforcement learning (deep Q-network, DQN) to choose posted wage offers over repeated rounds.
  • Workers: heterogeneous in characteristics and application behavior; they respond to posted wages and other posted information in the matching process.
• Experiments:
  • Vary number of competing firms.
  • Toggle DQN features (notably experience replay vs. no replay).
  • Vary platform design parameters and worker-side primitives (heterogeneity, search frictions).
  • Measure outcomes such as equilibrium wages, firm profits, worker application patterns, and convergence properties.
• Diagnostics: analyze learned policies and dynamics to identify when tacit collusion emerges and under what algorithmic/design conditions it fails to appear.

Implications for AI Economics

• Algorithmic tacit collusion extends beyond product pricing: employer-side use of RL can generate anti-competitive wage outcomes in labor markets.
• Algorithm design choices matter for market outcomes: low-level ML implementation details (e.g., experience replay) can be policy-relevant levers that either facilitate or prevent collusive dynamics.
• Regulatory and platform policy:
  • Antitrust and labor regulators should consider algorithmic learning dynamics as potential sources of coordinated (tacit) employer behavior.
  • Platforms could mitigate risk by encouraging or requiring algorithmic features that break temporal correlations (e.g., experience replay, enforced randomization) or by auditing posted-wage dynamics.
  • Increasing market competitiveness (more employers) and transparency of posting practices may reduce the likelihood of collusion.
• Research directions:
  • Empirical validation on real platforms to detect signatures of algorithmic collusion in posted wages.
  • Study of other algorithmic architectures, information structures, and multi-agent learning protocols.
  • Designing regulation-friendly algorithmic designs and platform rules that preserve market efficiency and worker welfare.