Randomising when pricing algorithms update can break algorithmic collusion in simulations, but access to competitors' live prices preserves the risk; regulators should note that enforced synchrony and high market transparency may amplify collusion incentives.

Main Finding

Asynchrony in price-update timing substantially reduces the emergence of algorithmic collusion in a Bertrand duopoly where firms use Q-learning. The effect is strongest for stateless algorithms (which rely on spontaneous synchronization) and weaker when algorithms condition on competitors’ current prices. Access to real-time opponent prices makes collusion robust to asynchrony; access only to aggregated past prices requires high update synchrony for collusion to emerge.

Key Points

• Timing model: Each firm updates prices at times following a Poisson clock; when one updates the other updates with probability q ∈ [0,1]. q controls synchrony (q=0 independent clocks, q=1 perfectly synchronous).
• Three algorithmic specifications:
  • NOSTATE (stateless Q-learning): no conditioning on opponent history.
  • AVGPRICE: state contains the opponent’s average price since the firm’s last update (aggregate transparency).
  • CURRENTPRICE: state includes the opponent’s current price (high transparency / real-time observation).
• Two collusion measures:
  • Standard collusion index (CI): normalized increase in average joint payoff relative to Nash and monopoly payoffs.
  • Automatic detection of reward–punishment schemes: record algorithm reactions to unilateral price cuts and quantify pattern similarity using Dynamic Time Warping (DTW) plus DBSCAN clustering.
• Main empirical patterns:
  • Increasing q (more synchrony) increases collusion for the two stateful Q-learning setups; for AVGPRICE high q is needed for collusion to appear.
  • Stateless algorithms display strong collusion only at high q; collusion largely disappears when updates are asynchronous.
  • When algorithms observe competitors’ current prices (CURRENTPRICE), collusion is robust across a wide range of q (i.e., persists despite asynchrony).
  • Exploration rate (ε) has a non-monotonic effect on collusion in stateless case: low ε supports coordination on collusive patterns; very high ε increases average prices mechanically (not learned collusion).
• Policy-relevant takeaways: adding randomness to update timing can be an effective anti-collusion tool; limiting access to rivals’ current prices reduces collusion risk; mandated synchronous update policies (e.g., FuelWatch) may unintentionally increase collusion when paired with market transparency.

Data & Methods

• Economic environment:
  • Two-firm Bertrand duopoly with differentiated products and logistic demand: D_i(p_i,p_j) = exp((a - p_i)/µ) / [exp((a - p_i)/µ) + exp((a - p_j)/µ) + 1]
  • Instantaneous profit π_i = D_i (p_i − c).
  • Collusion index CI = (average joint profit − Nash profit) / (monopoly profit − Nash profit), scaled so CI=0 at Nash, CI=1 at monopoly.
• Timing:
  • Each firm’s update times are driven by Poisson clocks (parameterization: each clock with rate λ/2 and when one updates the other updates with prob q); interpretations and extremes discussed.
• Learning algorithms:
  • Q-learning with standard update Q ← (1−α)Q + α[reward + γ max Q], updating only the chosen action / state pair at each update.
  • NOSTATE: Q over prices only; AVGPRICE: state = (own previous price, opponent’s average price since last update, discretized to grid); CURRENTPRICE: state = (own previous price, opponent’s current price).
  • Action selection: ε-greedy. Stateless runs use constant ε; stateful runs use decaying ε_t = exp(−βt).
• Experiments:
  • Extensive numerical simulations across grids of q and exploration parameters; baseline numeric values reported for an example: a=2, µ=0.25, c=1, n=20 prices over [pN−ξ, pM+ξ] with ξ=0.1; 100 independent runs for stateless experiments (others similarly repeated).
  • Metrics: long-run average payoffs → CI; reaction profiles to unilateral price cuts recorded at end of training.
  • Pattern detection: compare reaction time series to unilateral price cuts against pre-training (random) behavior using DTW distance and cluster with DBSCAN to infer reward–punishment structure.
• Robustness checks and diagnostics:
  • Compare measured CI to discretization-induced Nash CI (found ≈ 0.043) to distinguish true collusion from discretization artifacts.
  • Analysis of ε effects and two-price simplified intuition for synchronization mechanism in stateless case.

Implications for AI Economics

• Market-design & regulation:
  • Asynchrony/noise in update timing is a promising, low-cost intervention to reduce algorithmic collusion risk — regulators could encourage or require randomized update schedules or otherwise prevent enforced simultaneity.
  • Policies that increase transparency of competitors’ current prices (real-time feeds, scraping-enabled visibility) raise collusion risk because they enable robust reward–punishment strategies even under asynchronous updates. Limiting fine-grained competitor observability is a potential mitigation.
  • Mandates that cause simultaneous updates (e.g., FuelWatch-style rules) can unintentionally facilitate algorithmic tacit collusion; the combination of transparency and enforced synchrony is especially risky.
• Detection & enforcement:
  • The DTW + clustering approach offers an automated way to detect learned reward–punishment dynamics from algorithm reactions to unilateral deviations; this may supplement output-based tests but should be validated in richer settings.
  • Enforcement strategies should account for timing structure: standard output tests calibrated on synchronous-update simulations may overstate collusion risk if real-world updating is asynchronous.
• Research directions:
  • Extend to richer algorithm classes (deep RL, policy-gradient methods), larger oligopolies (>2 firms), endogenous or strategic timing, asymmetric update rates, and noisy/partial observability.
  • Empirical validation using field data on update timestamps and price trajectories to measure real-world asynchrony and its correlation with supracompetitive pricing.
  • Study interactions between timing, market transparency policy, and platforms/providers that bundle pricing tools (which can create de facto synchrony).
• Limitations to keep in mind:
  • Results are simulation-based for a two-firm Bertrand model with Q-learning; generalization to other demand systems, cost structures, multi-firm markets, or more advanced learning algorithms is not guaranteed.
  • Specific parameter choices (grid size, learning rates, decay schedules) can affect quantitative outcomes; qualitative pattern (asynchrony reduces coordination absent real-time observations) is robust in their experiments but requires broader testing.

Summary takeaway: update-timing matters. Preventing or randomizing synchronous price updates and limiting real-time observability of rivals’ prices are pragmatic levers to reduce the risk that learning algorithms develop tacit collusion.