California's 2020 cashless-bail reform produced no measurable change in reoffending in matched-arrestee comparisons; by contrast, large language models become more cooperative under responsibility framing—opposite human responses—and prompts shift models' internal reasoning even when choices stay the same.

Main Finding

Across three studies, rules and language shape choices in ways that are often invisible if researchers only measure final decisions. (1) California’s 2020 cashless-bail reform produced no detectable change in subsequent offending in the administrative data studied. (2) Large language models (LLMs) respond to responsibility framing opposite to humans in a Stag‑Hare coordination game: telling a model it “bears responsibility” increases cooperative (riskier “stag”) choices. (3) Prompt framing reliably changes the internal narrative and “embodiment” of model reasoning (first‑person, participation vs observation) but usually does not change the final decision. The shared methodological point: measuring only the outward decision can miss the important, policy‑relevant variation hidden in the mechanisms and generated narratives.

Key Points

• Bail reform study
  • Used propensity score matching on pre/post arrestee records.
  • No significant increase in offending after eliminating cash bail; no measurable improvement for released defendants either.
  • The result undermines both the claim that cashless bail spiked crime and the claim that it delivered measurable benefits (in the outcomes examined).
• LLM coordination study
  • Implemented a Stag‑Hare coordination game with LLMs as players.
  • Responsibility framing that reduces risk‑taking in humans causes the opposite in models: models cooperate (choose stag) more when told they bear responsibility.
  • The pattern holds across several model families and prompt variants.
• Embodiment‑scoring study
  • Introduces a rubric to score LLM outputs on first‑person perspective and whether reasoning invokes participation vs observation.
  • Prompt framing reliably shifts the model’s reasoning mode (more first‑person/participatory vs third‑person/observational) but has little effect on the final choice.
  • For researchers using LLMs as synthetic subjects, the observable decision can be stable while the model’s internal narrative changes—important when that narrative is the dependent variable.

Data & Methods

• Study 1 — Cashless bail reform (with Kiran John)
  • Data: administrative arrestee records from windows before and after California’s 2020 reform.
  • Identification strategy: propensity score matching to build comparable treated and control samples across implementation windows; comparisons of subsequent offending over multiple follow‑up windows.
  • Outcome: subsequent offending (recidivism/arrest metrics) — matched estimates produced small, statistically insignificant differences across windows.
• Study 2 — LLM Stag‑Hare coordination (with Gregory DeAngelo and Bryan C. McCannon)
  • Task: Stag‑Hare coordination game (safe option vs cooperative/riskier option; cooperative payoff dominates only if both choose it).
  • Subjects: multiple LLM families (several model types and sizes) and multiple prompt variants.
  • Intervention: responsibility framing prompts (telling the model it bears responsibility for consequences) vs control framings.
  • Measurement: frequency of cooperative (“stag”) choices; robustness checks across models and prompt phrasings.
• Study 3 — Embodiment‑scoring framework
  • Tasks: decision problems drawn from experimental economics (standard lab tasks used to elicit preferences/strategic reasoning).
  • Method: develop a coding rubric to score outputs on (a) degree of first‑person perspective/embodiment and (b) whether reasoning language frames the model as a participant or an observer.
  • Experiment: apply different prompt framings; score changes in reasoning mode and final decision; analyze alignment/decoupling between narration and choice.

Implications for AI Economics

• For policy evaluation and causal inference
  • Null effects are informative: rigorous pre/post and matching analyses can cut through polarized policy narratives (e.g., bail reform debates).
  • Administrative data and careful identification remain essential; absence of an effect in available outcomes should temper strong policy claims.
• For experiments that use LLMs as synthetic human subjects
  • Do not rely solely on observed choices. Prompts can change the reasoning or internal narrative without altering decisions—this matters if the study’s object is explanations, stated motives, or imagined perspective taking.
  • Report both surface behavior and measures of internal narrative/embodiment; preregister prompt variants and scoring rubrics.
• For deployment and governance of LLMs
  • Responsibility framing can shift model behavior in non‑human ways (e.g., increasing risk‑taking/cooperation), so behavioral responses to normative framing are model‑dependent and potentially counterintuitive.
  • Designers and regulators should test how normative language or accountability cues interact with model families and tasks before relying on them in high‑stakes contexts.
• For research practice and theory
  • The decision ≠ mechanism: researchers should measure intermediate representations (reasoning text, perspective, expressed incentives) in addition to final actions.
  • Develop and adopt standardized metrics (like the embodiment scores proposed) to compare narrative/representational modes across models and prompts.
  • When comparing LLMs to humans, be explicit that similar surface actions can be generated from different internal processes; avoid inferring human‑like motives from matching behavior alone.

Practical recommendations - When using LLMs as agents: include protocols that elicit and code reasoning (first‑person vs observational) and report both behavior and narrative metrics. - When evaluating policy interventions with observational data: use matching and multiple follow‑up windows; present null results transparently and test sensitivity to covariate balance and timing windows. - When designing prompts for normative effects: validate across model families and treat responsibility/accountability framings as potentially model‑specific interventions rather than universal nudges.