Randomized A/B-style interventions on predictions can reveal when predictions causally change outcomes, but the paper shows detection often requires large sample sizes unless performative effects are large or homogeneous — creating practical regions of indistinguishability where detection is infeasible.

Main Finding

The paper introduces OPAB (Outcome Performativity A/B Detection), an offline intervention-testing method that detects whether model predictions causally affect observed outcomes (Outcome Performativity) by randomly assigning predictions (do-interventions) and statistically testing for distributional differences in outcomes between assigned groups. The authors derive sample-complexity bounds for detecting performativity under several assumption classes, empirically validate those bounds on multiple datasets, and identify practical “regions of indistinguishability” where available interventions are insufficient to detect performativity.

Key Points

• Problem: Predictions can causally influence the outcomes they predict (Outcome Performativity), e.g., credit decisions affecting default or clinical predictions affecting care decisions. Undetected performativity can produce biased datasets and degrade future model performance.
• Method (OPAB): For an offline labeling phase, randomly assign predictions ˆy ∼ Bernoulli(0.5) to instances (a do-intervention), observe outcomes y, and test whether P(Y | do(ˆY=0)) and P(Y | do(ˆY=1)) differ significantly (e.g., Chi-squared or Fisher’s exact test). If so, conclude Outcome Performativity.
• Key theoretical result (Simple Outcome Performativity): when outcomes depend only on prediction (P(y=1 | ˆy=c) = α_c), the minimum per-group sample size N (for a Chi-squared test at significance δ) to detect a difference is N = χ^2_δ * [2(α0 + α1) − (α0 + α1)^2] / [2(α0 − α1)^2]. Consequence: required N → ∞ as α0 ≈ α1; small differences can be practically undetectable.
• Model-based extension: allowing dependence on features via a model f_θ (p*(x,ˆy) = f_θ(x + β_{ˆy}·θ)) yields an analogous bound where α_c is replaced by the observed group proportion p_{β_c} = P(Y=1 | do(ˆY=c), θ). Behavior is qualitatively similar: detection easier when per-group outcome rates diverge.
• Misclassification-based dynamics (as in prior work) also analyzed; the paper derives and validates sample complexity bounds under that assumption class as well (details and empirical validation included).
• Regions of indistinguishability: For a given budgeted per-group sample size \bar N and assumption-class parameters a, define the set of parameterizations where \bar N < N_{δ,a}. These parameter regions show where OPAB cannot reliably detect performativity for practical sample budgets.
• Empirical validation: simulations and experiments on multiple binary classification datasets (breast cancer, diabetes, adult census, Kickstarter, Titanic, loan) show the derived bounds predict detection behavior and reveal large indistinguishable regions when per-group effects are small. A case study on the Open Bandits dataset is also provided.
• Practical assumptions & caveats: the main operational simplification in much of the analysis is assuming uniform performative effect across the feature space so that group-level outcome distributions can be directly compared. When feature-dependent effects are present, conditional-estimation methods (T-learner, Double ML) are needed but are more sensitive to modeling choices and intervention imbalance.

Data & Methods

• OPAB algorithm:
  • For each instance x in a labeling set, randomly assign a prediction ˆy ∼ Bernoulli(0.5).
  • Observe the resulting outcome y after that prediction is acted on.
  • Accumulate counts for groups ˆy=0 and ˆy=1 and run a statistical test (Chi-squared or Fisher exact) comparing outcome distributions.
  • Reject null (declare performativity) if test p-value ≤ δ.
• Statistical testing: primary tests used are Chi-squared (δ = 0.05) and Fisher’s exact; robustness checks with other tests reported in appendix.
• Sample-complexity derivations: closed-form N expressions are derived for Simple and Model-based assumption classes (proofs in appendix). Misclassification-based bounds are also derived for that dynamics class.
• Empirical evaluation:
  • Synthetic imputation of performativity on existing non-performative datasets by manipulating outcomes according to the assumed dynamics (simple, model-based, misclassification).
  • Repeated simulations (typically averaged over many replicates) to estimate OPAB detection rates across parameter grids (e.g., α0, α1 or β0, β1).
  • Datasets used: breast cancer, diabetes, adult census, Kickstarter, Titanic, loan; plus an Open Bandits case study.
  • Typical setup: perform interventions on a subset (e.g., 10%) of dataset; measure detection probability as function of per-group sample size N and effect parameters.

Implications for AI Economics

• Cost and feasibility of detecting performativity:
  • The derived sample-complexity formulas allow practitioners and regulators to estimate labeling/intervention costs required to reliably detect outcome feedback. In many economically important settings (credit, insurance, marketplaces), required sample sizes can be very large when performative effects are modest, potentially making pre-deployment detection cost-prohibitive.
• Investment and monitoring decisions:
  • Firms should weigh the upfront cost of intervention-based testing (OPAB) against downstream risks: biased training data, degraded model performance, legal/regulatory liabilities, and negative welfare externalities. The presence of regions of indistinguishability suggests that in some parameter regimes it may be optimal to invest in continuous online monitoring rather than expensive offline testing.
• Market and strategic behavior:
  • Predictions that affect outcomes can create moral hazard and strategic responses. Detecting performativity early (pre-deployment) helps anticipate such feedback loops and design economic incentives (e.g., conditional pricing, underwriting rules) to mitigate adverse incentives.
• Policy and regulation:
  • Regulators could require evidence of non-performativity (or certified testing for performativity) before approving high-impact automated decision systems. OPAB provides a concrete, interpretable protocol and sample-size guidance for such audits—but the protocol’s assumptions (particularly uniformity across features) must be explicitly considered in regulatory standards.
• Pricing of data and labels:
  • In domains where interventions are costly (ethical or monetary), the sample-complexity bounds inform the price of obtaining labels under randomized predictions. This can affect budgeting for data acquisition, and competitive dynamics if only some firms can afford robust performativity testing.
• Design of pre-deployment experiments:
  • From an economic-design perspective, OPAB suggests that randomized-assignment experiments on outcomes are an efficient first step to detect performative risk before larger investment in conditional modeling. When sample budgets are tight, prioritize detecting large-effect performativity (where detection is feasible) and plan for alternate mitigation when effects are small but economically consequential.
• Limitations to apply in economic settings:
  • Uniform-effect assumption across X simplifies implementation but may not hold in heterogeneous economic populations (e.g., borrowers with varying resilience to denial). When heterogeneity exists, conditional methods are needed but increase sample requirements and model sensitivity — key economic trade-offs between precision and cost.

Practical recommendations for economists and practitioners - Use OPAB during labeling/label-acquisition stages when ethically and legally permissible as a low-complexity test for performativity. - Compute required per-group sample size via the provided formulas given plausible effect-size ranges; if required N exceeds budget, expect non-detectability and plan alternate monitoring/mitigation. - If heterogeneity across subpopulations is likely, either stratify OPAB experiments by key covariates or apply conditional estimators while accounting for larger sample needs and modeling risk. - Consider the social and regulatory costs of false negatives (undetected performativity) when setting detection budgets and thresholds; conservative policy may mandate testing even when OPAB cannot detect small effects.

Limitations noted by authors - Main analyses assume uniform performative effects across feature space; this makes OPAB simple but restricts applicability. - Detectability is sensitive to sample size; small but economically important effects may be practically undetectable. - Conditional estimators (if used) bring sensitivity to model and hyperparameter choices and to intervention imbalance.

Overall, OPAB provides a conceptually straightforward and analytically tractable framework for pre-deployment detection of outcome-level performativity, with explicit sample-size guidance and clear economic trade-offs between cost of testing and risk of undetected feedback.