A single foundation model can learn to produce asymptotically tight bounds for partially identifiable causal effects by training on a canonical full‑support prior over discrete structural causal models; toy experiments show correct coverage and much faster inference than a Gibbs baseline, though real‑world scalability remains untested.

Main Finding

The paper introduces a causal foundation-model approach for partial identification: train a Prior-Data Fitted Network (PFN) on synthetic SCMs drawn from a canonical prior that has full support over discrete structural causal models (SCMs). Under this setup the PFN’s predictive distribution converges (with increasing data) to the Bayesian posterior induced by the canonical prior conditional on the true observational distribution, and the posterior support converges to the true identified set. As a result, a single amortized model can produce asymptotically valid and tight bounds for arbitrary counterfactual/interventional queries from observational data (and optional structural constraints).

Key Points

• Problem targeted: partial causal identification (when causal queries are not point-identified due to unobserved confounding). Goal is to produce tight bounds (identified sets) rather than single estimates.
• Canonical SCMs: any SCM with discrete observables can be represented by a canonical SCM parameterized by finite categorical exogenous distributions; this yields a finite, maximally expressive parameterization for the discrete case.
• Canonical prior: put a prior with full support over the canonical SCM family by combining an Erdős–Rényi edge prior over graphs and Dirichlet priors over categorical exogenous distributions. Proposition: this prior gives positive mass to all SCMs (full support).
• PFN / amortization: train a transformer-based PFN to map (observational dataset, optional diagram) → distribution over a target causal query Ψ (binned in [0,1]), using synthetic examples generated from the canonical prior.
• Theoretical guarantees:
  • Consistency (Thm. 1): as sample size grows, PFN predictive distribution → Π0(Ψ | P(V)), i.e., the prior restricted to SCMs that generate the observed distribution.
  • Frequentist coverage (Thm. 2): Bayesian credible sets from the PFN are asymptotically valid frequentist confidence sets for the identified set.
  • Universality (Cor. 1): 100% Bayesian credible interval of the PFN converges to the true identified set — the PFN acts as a universal solver for partial identification.
• Practical extensions: can condition the PFN on partial/full causal diagrams (train on tuples (G, D, Ψ)), so domain knowledge is incorporable.
• Empirical demonstration: binary two-variable examples where analytical bounds exist. PFN achieves near-perfect coverage, comparable interval widths, and orders-of-magnitude faster inference than a Gibbs baseline over graphs.

Data & Methods

• Data generation:
  • Synthetic datasets are drawn by sampling canonical SCMs from the canonical prior (Erdős–Rényi graph prior with edge probabilities qD and qB; Dirichlet(α) priors for exogenous categorical distributions).
  • For each sampled SCM, generate n i.i.d. observational samples and compute the exact counterfactual/interventional ground truth Ψ(M).
• Model architecture:
  • Prior-Data Fitted Network (transformer-based), with input embeddings encoding variable values, types (observational vs interventional), and correlations.
  • Alternating attention layers: sample-wise attention (across dataset rows) and feature-wise attention (within-sample variable dependencies).
  • Linear head decodes a discrete (binned) predictive distribution over Ψ ∈ [0,1].
• Training objective: PFN is trained to predict the posterior predictive distribution over binned Ψ given the observational dataset, using synthetic (Dn, Ψ) pairs drawn from the canonical prior.
• Theoretical analysis:
  • Proved full support of canonical prior (so all SCMs compatible with observed distribution are a priori possible).
  • Show PFN converges to the prior conditioned on the observational distribution; credible sets attain asymptotic frequentist coverage; posterior support converges to identified set.
• Baselines / experiments:
  • Compared to a Gibbs sampler baseline (which requires graph input, so they took union over all 2-variable diagrams).
  • Metrics: coverage, mean interval width, and end-to-end inference time as function of dataset size n. Results: PFN (CFM) achieved ~100% coverage, similar widths, and much faster inference (milliseconds vs ~1s+ for Gibbs).

Implications for AI Economics

• Practical amortized solver for bounds under unobserved confounding:
  • Economists often face partial identification problems (e.g., treatment effects with unmeasured confounding, imperfect compliance). A trained CFM can produce tight, valid bounds for many different queries and datasets without bespoke derivations for each structural assumption.
• Scalability and speed:
  • Once trained, inference is extremely fast relative to iterative sampling/optimization baselines. This enables large-scale policy evaluation, sensitivity analyses, and repeated scenario testing (valuable for policy design, counterfactual scenario simulations, and automated economic decision tools).
• Integrating limited domain knowledge:
  • The ability to condition on partial causal graphs (or other structural constraints) makes the approach flexible for economics settings where some structural relations are known (e.g., institutional constraints, timing, exclusion restrictions) but others are uncertain.
• Reliable uncertainty quantification:
  • Asymptotic frequentist coverage of PFN credible sets gives a formal guarantee that posterior-derived bounds are valid confidence sets for identified sets — helpful for communicating uncertainty in policy recommendations and for regulation-sensitive domains.
• Cautions and limitations for economic applications:
  • Discrete-observables assumption: the theoretical construction relies on finite discrete endogenous variables. Many economic variables are continuous or high-cardinality; extension to continuous outcomes is nontrivial and not solved here.
  • Finite-sample behavior and prior sensitivity: while asymptotic guarantees hold, finite-sample bounds depend on the choice of canonical prior (Erdős–Rényi/Dirichlet hyperparameters) and PFN capacity/training. Domain-informed priors or conditioning on partial graphs are advisable.
  • High-dimensional systems: canonical parameterization can lead to very large latent cardinalities as graph size grows; computational and sample requirements may grow rapidly in realistic economic models. Careful architecture scaling, dimension reduction, or structured priors will be needed.
  • Interpretability & validation: PFNs are black-box neural models. For policy-sensitive economic decisions, combining PFN outputs with analytic bounds, sensitivity analyses, and diagnostic checks remains important.
• Overall: the paper supplies a promising framework to amortize and scale partial-identification inference in economics, enabling fast, principled bounding of causal queries under unobserved confounding, provided either variables are discrete or appropriate extensions/approximations are used and finite-sample/prior choices are carefully handled.