0 cumulative citations
View corpus contextA calibrated agent-based simulator reproduces Bitcoin market microstructure and impact, enabling realistic tests of reinforcement-learning market makers; comparative experiments show clear trade-offs across RL architectures and state-action designs that matter for market-making performance.
Adaptive market making using reinforcement learning in a multi-agent market simulation
openalex
descriptive
medium evidence
7/10 relevance
Summary only
summary available; pdf_status=error
DOI
Source
PDF
The thesis develops an empirically calibrated ABIDES extension (PRIME) that reproduces Binance Bitcoin market microstructure and market-impact patterns and uses this environment to benchmark reinforcement-learning market-makers, revealing architecture-dependent trade-offs in learning and performance.
Citation observations
Cumulative provider counts captured on specific dates; providers are never combined.
Financial markets are amongst the most complex and well-studied multi-agent systems. Over the past several decades, researchers have devoted considerable effort to understanding their structural and behavioural properties. However, building a simulated market environment that can replicate realistic interactive behaviour remains an ongoing challenge. This limitation restricts the scope and validity of experiments aimed at testing new trading strategies or hypotheses, particularly when simplifying assumptions, such as negligible market impact or the absence of market feedback, fail to hold. This thesis addresses this challenge by building on the Agent-Based Interactive Discrete Event Simulation (ABIDES) simulation platform to create an empirically grounded and reproducible environment for studying cryptocurrency markets. First, we develop a methodology for tuning simulation parameters to replicate the stylised facts observed in Binance following the 2017 cryptocurrency boom. Second, we introduce Price-Reverting Impact Model of a cryptocurrency Exchange (PRIME), a novel configuration of the ABIDES simulator designed to produce realistic market responses to agent actions whilst retaining the ability to track an external price series. Third, we use the PRIME framework to conduct the first controlled comparison of Reinforcement Learning (RL) based market makers across various architectures in a realistic, interactive environment, establishing a benchmark for future research. Our findings demonstrate that a carefully calibrated agent population comprising Zero Intelligence, Momentum, and Mean-Reversion agents can reproduce key statistical properties observed in the Bitcoin market. Furthermore, the PRIME framework successfully captures market impact behaviours described in existing and novel empirical observations, allowing for controlled but realistic experimentation of market agents in this environment. Finally, our reinforcement learning market-makers deployed on this platform reveal important insights into the trade-offs involved in state-action space design and the learning dynamics of various RL architectures. This setup also allows for a meaningful comparative evaluation of market-makers with previous work. Collectively, these contributions provide a reproducible foundation for advancing market simulation frameworks and for developing and evaluating novel reinforcement learning approaches to the market-making problem.
Summary
Main Finding
A carefully calibrated, agent-based extension of the ABIDES simulator—organized as the Price-Reverting Impact Model of a cryptocurrency Exchange (PRIME)—can reproduce key statistical properties of the post-2017 Binance Bitcoin market and generate realistic market-impact behaviour. PRIME provides a reproducible, empirically grounded environment that supports controlled, interactive experiments, including the first systematic comparison of reinforcement-learning (RL) market-makers across multiple architectures, yielding practical insights into state–action design and learning dynamics for market-making agents.
Key Points
- Contribution 1: Methodology to tune agent-based simulation parameters so the simulated market reproduces stylised facts observed on Binance after the 2017 cryptocurrency boom.
- Contribution 2: PRIME, a new configuration of ABIDES, that (a) produces realistic market responses to agent actions (market impact) and (b) retains the ability to follow an external price series for controlled experiments.
- Contribution 3: A controlled benchmark study of RL-based market-makers across different architectures in an interactive, realistic environment, enabling meaningful comparative evaluation against prior work.
- Empirical result: A population composed of Zero Intelligence, Momentum, and Mean-Reversion agents—when carefully calibrated—can replicate several statistical properties of the Bitcoin market.
- PRIME reproduces known and newly observed market-impact behaviours, allowing experiments that balance realism with experimental control.
- RL experiments identify trade-offs in state and action-space design and reveal characteristic learning dynamics across architectures, informing future RL market-making research.
Data & Methods
- Platform: Extended the Agent-Based Interactive Discrete Event Simulation (ABIDES) environment to implement PRIME and additional agent types.
- Empirical grounding: Simulation parameters were calibrated against Binance market data from the period following the 2017 cryptocurrency boom (order book and trade dynamics used to match stylised facts).
- Agents: Population includes Zero Intelligence agents, Momentum agents, and Mean-Reversion agents to produce endogenous market dynamics.
- PRIME design: A configuration that models price reversion/impact so actions by simulated agents produce realistic, observable effects on the limit order book while allowing an exogenous price series to be tracked when needed.
- Experiments: Controlled, interactive simulations comparing RL market-makers across multiple RL architectures (policy/value-based and actor-critic families) and varying state–action representations; evaluation focused on market-making performance and interaction-driven effects.
- Reproducibility: Emphasis on an empirically tuned, documented setup intended as a benchmark and baseline for future research.
Implications for AI Economics
- Better testbeds for multi-agent market research: PRIME supplies a more realistic, reproducible simulation environment for studying strategic interaction, emergent market structure, and algorithmic trading behaviours.
- Robust evaluation of AI market agents: Interactive, impact-aware simulations reduce false positives from simplifying assumptions (e.g., negligible market impact), improving external validity of RL trading agent research.
- Insights into RL design for markets: Observed trade-offs in state/action representation and learning dynamics guide architecture choices and training regimes for automated market-makers.
- Benchmarking and comparability: A reproducible PRIME configuration enables head-to-head comparisons across RL approaches and fosters cumulative progress in AI-for-markets research.
- Policy and safety: More realistic simulation frameworks can help regulators and exchanges stress-test algorithmic strategies and explore systemic risks arising from automated agents.
- Limitations and next steps: Simulation realism depends on calibration fidelity and agent heterogeneity; field validation and incorporation of additional market microstructure features (e.g., latency, fee structures, cross-exchange flows) will further strengthen external validity.
Assessment
Paper Typedescriptive
Evidence Strengthmedium — The work provides internally valid, reproducible simulation evidence: the simulator is empirically calibrated to real Binance market microstructure and reproduces known stylised facts and impact patterns, and controlled experiments compare RL architectures. However, it remains simulation-based (no live-market deployment), is calibrated to a specific asset/exchange/time period, and thus has limited external validity for real-world economic impact.
Methods Rigormedium — Methodological contributions appear systematic — extension of ABIDES, explicit calibration procedure to match empirical stylised facts, and structured benchmarking of multiple RL architectures — indicating careful design. Rigor is constrained by choices in agent population design, parameter tuning (risk of overfitting), scope of validation (single exchange/time window), and potential simplifications (e.g., omitted institutional details, latency, margining, and strategic adaptation by real counterparties).
SampleHistorical limit-order-book and trade data from Binance covering the period following the 2017 cryptocurrency boom were used to derive stylised facts and calibrate simulator parameters; simulated experiments use ABIDES extended with the PRIME impact model and an agent population of Zero Intelligence, Momentum, Mean-Reversion agents plus various reinforcement-learning market-maker agents evaluated across multiple architectures and state-action designs.
Themesinnovation adoption
IdentificationNo causal identification in the sense of observational causal inference; instead uses a controlled, agent-based simulation calibrated to Binance limit order book data (post-2017 boom) to reproduce stylised facts, and then performs within-simulation counterfactual experiments comparing RL market-maker architectures. The PRIME simulator enforces a price-reverting impact model while allowing an external reference price series so that agent actions and resulting market responses can be isolated and compared.
GeneralizabilityCalibrated to Binance Bitcoin market post-2017 — results may not generalize to other exchanges, assets, or different market regimes., Simulation abstractions (chosen agent types, matching rules, latency, margining, institutional features) may omit key real-world mechanisms affecting market impact and agent behaviour., Parameter tuning risks overfitting to the calibration period; out-of-sample robustness across regimes not established., RL agents trained in simulation may not transfer to live markets due to adversarial participants, transaction costs, regulatory constraints, and implementation frictions., Market participant adaptation (learning and strategic changes in the wild) is not fully modeled, limiting long-run external validity.
Claims (8)
| Claim | Direction | Outcome | Confidence & Evidence | Details |
|---|---|---|---|---|
| We develop a methodology for tuning simulation parameters to replicate the stylised facts observed in Binance following the 2017 cryptocurrency boom. Output Quality | positive | replication of stylised facts (statistical properties) of Binance market data |
Reading fidelity
high
Study strength
medium
|
not reported
|
| We introduce PRIME (Price-Reverting Impact Model of a cryptocurrency Exchange), a novel configuration of the ABIDES simulator designed to produce realistic market responses to agent actions while retaining the ability to track an external price series. Market Structure | positive | realism of market response to agent actions and capacity to follow an external price series |
Reading fidelity
high
Study strength
medium
|
not reported
|
| A carefully calibrated agent population comprising Zero Intelligence, Momentum, and Mean-Reversion agents can reproduce key statistical properties observed in the Bitcoin market. Output Quality | positive | match between simulated and observed statistical properties of the Bitcoin market (stylised facts) |
Reading fidelity
high
Study strength
medium
|
not reported
|
| The PRIME framework successfully captures market impact behaviours described in existing and novel empirical observations. Market Structure | positive | market impact behaviour (response of prices/liquidity to agent actions) |
Reading fidelity
high
Study strength
medium
|
not reported
|
| We use the PRIME framework to conduct the first controlled comparison of Reinforcement Learning (RL) based market makers across various architectures in a realistic, interactive environment, establishing a benchmark for future research. Research Productivity | positive | comparative performance of different RL market-making architectures |
Reading fidelity
high
Study strength
medium
|
not reported
|
| Our reinforcement learning market-makers deployed on this platform reveal important insights into the trade-offs involved in state-action space design and the learning dynamics of various RL architectures. Research Productivity | positive | effects of state-action space design on RL learning dynamics and market-making performance |
Reading fidelity
high
Study strength
medium
|
not reported
|
| This setup allows for a meaningful comparative evaluation of market-makers with previous work. Research Productivity | positive | comparability/benchmarking utility of the PRIME simulation setup |
Reading fidelity
high
Study strength
medium
|
not reported
|
| Collectively, these contributions provide a reproducible foundation for advancing market simulation frameworks and for developing and evaluating novel reinforcement learning approaches to the market-making problem. Research Productivity | positive | reproducibility and utility of the contributed simulation framework for future research |
Reading fidelity
high
Study strength
medium
|
not reported
|