0 cumulative citations
View corpus contextSimulated financial agent societies become fragile in larger populations: the harmful share needed to trigger systemic collapse falls sharply with society size even as the absolute number of malicious agents at the threshold rises, and broader network reach — not stronger individual conformity — chiefly drives the increased fragility.
WolfSociety: Understanding Collective Risk from Harmful-Agent Scaling in Financial Agent Societies
In simulated financial agent societies, collective collapse shows a nonlinear finite-size scaling: larger societies reach a 50% collapse probability at a smaller harmful fraction (αc falls from ~4.7% at N=100 to ~2.2% at N=2000) while the absolute number of harmful agents required increases.
Citation observations
Cumulative provider counts captured on specific dates; providers are never combined.
Safety evaluations typically focus on individual agents, but interacting agents can spread harmful information and influence the environment in which later decisions are made. We study how collective failure changes with harmful-agent fraction and society size in a controlled financial agent society, where agents communicate over a social network and trade in a shared market. In the primary financial scenario, collective failure requires broad harmful diffusion together with severe price dislocation or liquidity stress. Across all tested society sizes, failure remains rare at low harmful fractions but rises sharply over a narrow range. As society size grows from N=100 to N=2000, the harmful fraction associated with a 50% failure probability decreases from 4.7% to 2.2%, while the corresponding number of harmful agents increases from approximately 5 to 44. In contrast, when the number of harmful agents is held fixed, their impact becomes weaker as the society grows. Controlled interventions further show that broader network reach shifts the collapse boundary toward lower harmful fractions, whereas stronger conformity alone has little effect. To characterize these effects, we introduce Agent Society Dynamics, a finite-size framework for relating harmful-agent fraction, society size, and interaction structure to collective failure. Overall, our results reveal a nonlinear, size-dependent collapse transition in financial agent societies, showing that collective failure depends not only on the prevalence of harmful agents but also on the size and interaction structure of the surrounding society. Code is available at https://github.com/SAIL-Research-Lab/WolfSociety.
Summary
Main Finding
Larger simulated financial "agent societies" become fragile at smaller harmful-agent fractions even though more harmful agents are required in absolute terms. In a controlled multi-agent market+social network environment (WolfSociety / WolfBench), the harmful fraction α at which the probability of a society-level collapse reaches 50% falls with population size N (αc(100)=4.7% → αc(2000)=2.2%), while the harmful count at that boundary rises (Kc(100)≈5 → Kc(2000)≈44). The authors formalize this as Agent Society Dynamics and empirically estimate a finite-size boundary exponent ν ≈ 0.222, i.e. αc(N) ∝ N−ν and Kc(N) ∝ N1−ν.
Key Points
- Environment and collapse criterion
- Agents communicate over a social network and trade in a shared market. Market outcomes (price dislocation, liquidity stress) feed back into later agent decisions.
- Primary manipulation (S1) models pump-and-dump style harmful behavior. Collapse requires broad harmful diffusion together with either severe price dislocation or severe liquidity stress (operational thresholds specified in the simulator).
- Main empirical pattern (finite-size scaling)
- At each N tested (100 → 2000), collapse probability is low at small harmful fractions then rises sharply over a narrow α range.
- Collapse boundary αc(N) decreases with N, while Kc(N)=Nαc(N) increases with N. Estimated exponent ν ≈ 0.222 (0<ν<1 explains simultaneous fraction decrease and count increase).
- Fixed-count analysis
- When the number of harmful agents K is held fixed, the same harmful minority causes less severe diffusion and market disruption as N increases (i.e., a negative size coefficient bN in severity regressions).
- This decline in fixed-count severity is robust to different liquidity scaling rules for the market maker: fixed total liquidity, square-root scaling, and per-capita scaling.
- Interaction structure vs conformity
- Increasing individual conformity (agents weight social evidence more strongly) raises how much actions reflect social signals but has little effect on the collapse boundary.
- Increasing network reach (making harmful information accessible to more agents) shifts the collapse boundary to lower harmful fractions — i.e., reach matters more than per-agent conformity for system fragility.
- Joint changes to multiple interaction properties produce larger boundary shifts.
- Robustness
- Primary scaling pattern held across different controller realizations and five LLM backbones.
- Of three additional manipulation scenarios (S2–S4), two show nonlinear transitions similar to S1; one did not reach the collapse criterion in the tested range.
- Tools and contributions
- WolfBench: the packaged environment, four manipulation scenarios, evaluation interfaces, and code (repo cited in paper) for studying society-level failure in financial agent societies.
- Agent Society Dynamics: conceptual finite-size framework linking initial harmful input scaling and interaction-driven amplification to the collapse boundary.
Data & Methods
- Simulation design
- Agent-based simulation of N agents (tested N ∈ {100, 500, 1000, 1500, 2000} in main experiments) over T = 30 trading days.
- Harmful fraction α targeted; K = round(αN) harmful agents per episode. Agents have heterogeneous roles, social-message channels, and trading behaviors.
- Market maker provides liquidity; experiments vary liquidity scaling (fixed, √N, per-capita).
- Collapse and severity metrics
- Daily severity SS_t combines measures: social diffusion reach q_t, price dislocation d_t, liquidity stress L_t via a prespecified formula; episode severity RS = max_t SS_t. Collapse occurs if RS ≥ 1 (preset thresholds for q, d, L).
- Formalization
- Define collapse probability p(N, α) and collapse boundary αc(N) = inf{α : p ≥ 1/2}.
- Agent response model summary: local input ∆U = direct harmful input + θ·private evidence + γ·social evidence; action xi+1 = tanh(β/2 · ∆U). Societal interaction response summarized as χ+ ∝ N^ζ; initial harmful input as α N^δ, so ν = δ + ζ and αc ∝ N−ν.
- Information-based measure DN = I(A; M | V,H,R) / [I(A; M | V,H,R) + I(A; V | M,H,R)] to quantify how much actions reflect social vs private evidence.
- Estimation
- Collapse boundaries estimated by interpolation across sampled α grid; finite-size exponent ν estimated from αc(N) scaling. Fixed-count severity fit via log RS = a + bN log N + bK log K + ε.
- Robustness checks
- Sensitivity to threshold choices, liquidity rules, controller types, LLM backbones, and multiple manipulation scenarios.
Implications for AI Economics
- Systemic risk depends on both fraction and absolute count, and scales nonlinearly with market population size
- Larger trading populations are vulnerable to lower harmful fractions; economic risk assessments should therefore consider fraction-based thresholds that move with market size rather than fixed-fraction or fixed-count heuristics alone.
- Policy design and stress testing should jointly vary N and α: a fixed small minority that is inconsequential in a small market can become consequential (in fraction terms) as markets grow.
- Network structure and propagation channels matter more than per-agent conformity
- Interventions that reduce breadth of information propagation (limit broadcast/reach, message routing, or exposure) are likely more effective at reducing society-level collapse risk than measures that only reduce the weight individuals place on social signals.
- Market/platform design should prioritize controls on reach (rate-limiting promotions, limiting repeat amplification, provenance signals) in addition to agent-level behavior constraints.
- Liquidity provision mitigates but does not eliminate size effects
- Scaling liquidity (per-capita or √N) changes but does not fully remove the decline in fixed-count severity with N. Market microstructure rules matter; economic models should incorporate liquidity scaling when predicting impact of manipulative agents.
- Operational monitoring and regulation
- Surveillance metrics should track both harmful-agent counts and their network reach; thresholds used for intervention could be size-dependent (derived from finite-size scaling estimates) rather than absolute.
- Consider limiting the number of agents allowed to run similar automated trading/manipulative strategies or imposing caps on message propagation from low-provenance sources.
- Modeling and forecasting
- Macroeconomic and market models of AI-driven agent populations should incorporate finite-size scaling and interaction-mediated amplification (the δ + ζ decomposition) rather than assuming linear or size-invariant effects.
- Agent-based stress testing (e.g., with WolfBench-like environments) is a practical approach to explore regime shifts and tipping points produced by information propagation in markets.
- Caution and limitations for economic application
- Results are simulation-based and depend on modeling choices (network topology, role mix, LLM controllers, collapse thresholds). The collapse thresholds are operational choices for the simulator—not regulatory definitions.
- Further empirical validation and sensitivity analysis across realistic market microstructures and real-world communication channels are needed before applying quantitative thresholds in policy.
- Research agenda for AI economics
- Empirically estimate δ and ζ (initial input vs interaction amplification) in real or higher-fidelity market+social settings.
- Study trade-offs between openness/efficiency (broad information flow) and systemic fragility; design mechanism-level mitigations (liquidity rules, exposure limits, provenance/credibility mechanisms).
- Extend finite-size analyses to mixed markets with human and AI participants and to other economic domains (auctions, advertising markets, decentralized finance).
Repository and artifacts: authors provide WolfBench code and scenarios (repo referenced in paper) for reproducibility and further experimentation.
Assessment
Paper Typedescriptive
Evidence Strengthmedium — The paper presents internally consistent, reproducible, and systematically varied simulation evidence with multiple robustness checks (different liquidity rules, intervention experiments, alternative LLM backbones, bootstrapping and threshold sensitivity). However, the evidence is entirely from a stylized simulator: external validity to real-world markets and deployed multi-agent systems is limited by modeling assumptions (agent controllers, message/state abstractions, market microstructure), arbitrary collapse cutoffs, and the gap between simulated LLM agents and production systems.
Methods Rigorhigh — The authors use a well-structured experimental design (grid over N and α), report collapse boundaries via interpolation and bootstrap uncertainty, perform fixed-count analyses and multiple liquidity-scaling variants, conduct controlled interventions (network reach, conformity), provide sensitivity analyses and alternative controller/backbone checks, and introduce a finite-size scaling framework tying observed patterns to conceptual parameters; shortcomings are mainly model-specification choices rather than experimental execution.
SampleAgent-based financial market simulation with heterogeneous agent roles (N varied from 100 to 2000), T=30 day episodes, harmful-agent fraction α set per run with K = round(αN) harmful agents, four manipulation scenarios (S1 pump-and-dump primary, S2 scalping, S3 spoofing/layering, S4 wash trading), three liquidity rules (fixed, sqrt(N), per-capita), multiple LLM/backbone controller realizations, and repeated runs per parameter grid with bootstrap estimation; key readouts include collapse indicator combining harmful diffusion reach, price dislocation and liquidity stress, episode severity RS1, and information measures (DN).
Themesgovernance org_design
IdentificationControlled agent-based simulation experiments: harmful agents are randomly assigned across many runs while systematically varying society size N and harmful fraction α (and holding other protocol elements constant); causal statements derive from in-silico randomization and controlled interventions (fixed-count runs, liquidity scaling, and controlled changes to network reach and conformity), plus finite-size scaling analysis to summarize how collapse probability changes with N and α.
GeneralizabilityResults derive from a stylized simulator and may not map quantitatively to real markets or deployed multi-agent systems., Agent behaviors/controllers (LLM backbones, rule-based elements) are simplified/assumed and may misrepresent production AI agents., Market microstructure and liquidity rules are simplified abstractions; real-world venues have richer structure and institutional constraints., Collapse definition uses prespecified thresholds (operational cutoffs) that are not direct regulatory or empirical benchmarks., Network topology, attention/trust models, and parameter choices (e.g., conformity strength) may materially affect scaling exponents.
Claims (10)
| Claim | Direction | Outcome | Confidence & Evidence | Details |
|---|---|---|---|---|
| In the primary financial-agent scenario, collective failure is rare at low harmful-agent fractions and then rises sharply over a narrow range of harmful fractions. Market Structure | positive | Probability of society-level collective failure |
Reading fidelity
high
Study strength
medium
|
Sharp rise over a narrow harmful-fraction range
|
| As society size increases from N = 100 to N = 2000, the harmful-agent fraction associated with a 50% probability of collective failure decreases from 4.7% to 2.2%. Ai Safety And Ethics | negative | Harmful-agent fraction at the 50% collapse-probability boundary |
Reading fidelity
high
Study strength
medium
|
n=6
Decrease from 4.7% to 2.2%
|
| Although the harmful fraction required for collapse decreases with society size, the corresponding absolute number of harmful agents increases from approximately 5 to 44. Ai Safety And Ethics | positive | Number of harmful agents at the collapse boundary |
Reading fidelity
high
Study strength
medium
|
n=6
Increase from approximately 5 to 44 harmful agents
|
| The estimated finite-size scaling exponent for the collapse boundary is ν̂ = 0.222. Ai Safety And Ethics | negative | Scaling of the harmful-fraction collapse boundary with society size |
Reading fidelity
high
Study strength
medium
|
n=6
ν̂ = 0.222
|
| When the number of harmful agents is held fixed, the same harmful minority produces less severe harmful diffusion and market disruption as society size increases. Market Structure | negative | Episode severity, including harmful diffusion and market disruption |
Reading fidelity
high
Study strength
medium
|
not reported
|
| The decline in fixed-count severity persists under fixed-liquidity, square-root liquidity scaling, and per-capita liquidity scaling. Market Structure | negative | Severity of harmful diffusion and market disruption at a fixed harmful-agent count |
Reading fidelity
high
Study strength
medium
|
n=3
Persists across three liquidity rules
|
| Increasing conformity substantially increases the dependence of individual actions on social evidence but produces little change in the collapse boundary. Ai Safety And Ethics | mixed | Dependence of individual actions on social evidence and the harmful-fraction collapse boundary |
Reading fidelity
high
Study strength
medium
|
Substantial increase in social-evidence dependence; little change in collapse boundary
|
| Increasing network reach allows harmful information to spread to more agents and shifts the collapse boundary toward lower harmful fractions. Ai Safety And Ethics | positive | Harmful-information propagation and harmful-fraction collapse boundary |
Reading fidelity
high
Study strength
medium
|
Shift toward lower harmful fractions
|
| The primary size-dependent scaling trend remains consistent across rule-based and hybrid controller realizations and across five alternative LLM backbones. Ai Safety And Ethics | positive | Robustness of the size-dependent collapse-scaling relationship |
Reading fidelity
high
Study strength
medium
|
n=5
Consistent across five alternative LLM backbones
|
| Among the three additional manipulation scenarios, two exhibit nonlinear transitions, while the third does not reach the collapse criterion within the tested range. Market Structure | mixed | Nonlinear collapse transitions under alternative manipulation mechanisms |
Reading fidelity
high
Study strength
medium
|
n=3
Two of three scenarios show nonlinear transitions
|