Concentrated within-variation can break t-tests in saturated fixed-effect regressions, and no single critical value cures the boundary failure; constructing design-only contrasts (cycle-space sign-flips) gives finite-sample exact p-values under symmetric errors and delivers practical power via a cycle-packing algorithm.

Main Finding

When most treatment variation is absorbed by many fixed effects, conventional Gaussian (t-test) inference can fail because identification depends on how residualized treatment variation is concentrated across observations, not its aggregate size. Halkiewicz develops a finite-sample, design-based alternative: construct nuisance-annihilating contrasts (q with q′D = 0) from the design, use sign-flip randomization on disjoint supports to obtain exact inference under symmetric (and blockwise symmetric) errors, and—for two-way (worker–firm, firm–time) designs—shows these contrasts correspond exactly to the cycle space of the observation multigraph. He introduces an observable capture ratio κC (a Pitman-efficiency analogue) that governs power, provides cycle-packing algorithms to produce high-capture granular supports, and supplies diagnostics (λn, Hn, Neff, score-based analogues) and software. Conventional leverage-corrected tests can severely over-reject under concentrated identifying variation; the cycle/sign-flip tests are exact and competitive in power when appropriately packed.

Key Points

• Concentration parameter (design-only): λn = maxi exi^2 / Vn (where ex = MDx, Vn = ex′ex). λn → 0 is the usual negligibility condition for asymptotic normality; when λn is not small, the asymptotic null may be a convolution mixing few large-weighted errors and a Gaussian remainder (Proposition 2.1).
• Impossibility at the boundary (Proposition 2.3): At the fully concentrated boundary (one persistent weight), the limiting null law depends on the unknown symmetric error distribution, so no fixed critical value (tabulated test) is uniformly valid across symmetric equal-variance laws. Valid inference must adapt to the realized error law.
• Nuisance-annihilating contrasts: any q measurable from (x, D) with q′D = 0 extracts a linear combination free of fixed effects: q′(Y − xβ0) = q′ε under H0. If contrasts have disjoint supports and errors are blockwise independent and symmetric across those blocks (Assumption 2), flipping signs on contrasts yields exact finite-sample randomization tests (Theorem 3.3).
• Two-way designs ↔ cycle space: In bipartite observation graphs (edges = observations, vertices = firms/workers or firm/time), the space of annihilating contrasts equals the graph’s cycle space. The within-projection norm Vn = ||ΠZ x||^2 is the squared projection of treatment onto the cycle space (Proposition 4.1).
• Capture ratio κC: For a system of C disjoint supports with local normalized projections vc, capture is κC = (1/Vn) ∑c ||ΠZ_Ac x||^2 ∈ [0,1]. In diffuse regimes κC is the Pitman relative efficiency; κC^−1/2 is the multiplicative standard-error price (Theorem 3.9).
• Capture–granularity trade-off: Merging supports increases capture (can reach κC = 1 by taking one projection per biconnected block) but reduces the randomization group size (one sign per support), possibly wrecking power because the attainable minimal p-value grows (orbit-size floor). Fine, granular supports preserve randomization granularity but may lower capture. The paper formalizes and prices this trade-off.
• Cycle packing algorithms: constructs (digons, edge-disjoint firm-pair four-cycles, recursive contraction) that produce granular, high-capture supports. On public matched employer–employee extract, cycle packing achieves κC ≈ 0.51 (match-level treatment) vs 0.26 for naive greedy packing; for a time-varying covariate it achieves 0.91 vs 0.88.
• Empirical diagnostics & examples:
  • Grunfeld investment regression: realized score concentration λn = 0.739 (N_score_eff = 1.80); 32 valid supports capture κC = 0.627; exact 95% CI is [0.150, 0.450].
  • Monte Carlo: with concentrated design (λn = 0.29, effective sample 6.0), df-corrected t rejects up to 58.5% (size badly inflated), HC2 up to 33.4%; cycle test empirical size ~5% everywhere; cycle-test power often higher once oracle size correction applied to conventional tests.
• Software and reproducibility: implementations in PanelAdequacy.jl and an R companion (panelcert) are provided.

Data & Methods

• Model and conditioning: Linear model Yi = xiβ + di′γ + εi, conditional on the design (x, D). Focus is on inference for β with γ unrestricted nuisance.
• Key design objects:
  • ex = MDx (treatment residualized on fixed effects)
  • Vn = ex′ex
  • λn = maxi exi^2 / Vn (self-normalized leverage)
  • Herfindahl Hn = ∑(exi^2 / Vn)^2, Neff = 1/Hn
  • Score-based realized analogues using fitted residuals (bλscore, bNscore_eff) as warnings.
• Limit theory: Proposition 2.1 gives convolutional limits when a finite number of weights persist; Corollary 2.2 shows studentization cannot generally restore normality.
• Exact-test construction:
  • Build annihilating contrast system {q1,...,qC} with q′c D = 0 and pairwise disjoint supports Sc (measurable from design).
  • Under Assumption 2 (blocks of observations that are independent and centrally symmetric; supports unions of blocks), contrast scores Uc = vc′(Y − xβ0) are invariant to flipping signs on each support.
  • Define test-statistic T(U1,...,UC) (default T = |∑c bc Uc| with bc = vc′x) and compute randomization p-value by enumerating or Monte Carlo sampling sign patterns s ∈ {±1}^C (Theorem 3.3). Studentized variants are also exact.
• Two-way graph methods:
  • Observations are edges of bipartite multigraph. Annihilating contrasts correspond to ±1 alternations around cycles (including digons for repeated matches).
  • Projection of x onto cycle space gives Vn; local projections on supports give the captured portion.
  • Cycle packing: algorithmic constructions (digons, firm-pair 4-cycles, recursive contraction) to get many edge-disjoint or nearly edge-disjoint cycles to maximize κC with maintained granularity.
• Monte Carlo setups:
  • Simulation designs include concentrated and diffuse regimes, heteroskedastic symmetric errors, and dependence structures consistent with Assumption 2 at various granularities.
  • Benchmarks included df-corrected t, HC2, and oracle-adjusted versions.

Implications for AI Economics

• Many AI-economics settings have high-dimensional fixed effects and networked observations (platform-user, user-item, firm-worker, time-varying policy adoption). When controlling for those effects, remaining identifying variation for treatment or instrument often concentrates on few observations; this paper shows standard asymptotic t- or cluster-robust inference can be invalid in that regime.
• Practical diagnostics: report λn, Hn, Neff and the score-based analogues (bλscore, bNscore_eff). If effective sample size is small or λn large, conventional inference is suspect and the exact randomization methods are a principled alternative.
• Exact randomization tests adapt to the realized (unknown) error law and remain valid under arbitrary heteroskedasticity so long as blockwise symmetry/exchangeability assumptions are plausible at the support granularity chosen. This is useful for AI policy evaluation (e.g., A/B tests with fixed effects, algorithmic interventions with network spillovers) where dependence and heteroskedasticity are common.
• Design guidance: thinking in graph terms (cycles) suggests how to design data collection or treatment assignment to increase identifiability—pack cycles or create movement that produces cycle-space variation. The capture–granularity trade-off is directly relevant to clustering choices in practice: coarser clusters may increase captured variation but can destroy randomization granularity and power.
• Algorithmic/statistical complementarity: these exact, design-based tests provide a robustness layer that complements ML-based heterogeneity/exogeneity diagnostics. They can be used to validate inference from models that rely on asymptotic approximations, or as replacements when asymptotics fail due to concentration.
• Accessibility: the paper supplies code (PanelAdequacy.jl, panelcert) and practical algorithms for cycle-packing, so applied AI economists can compute diagnostics and run exact tests on large panels or matched datasets.
• Caveats: exactness requires blockwise symmetry (or exchangeability via antithetic pairing) at the support level—practitioners must judge plausibility given the error-generating process (e.g., residual interactive fixed effects may violate the assumption). The method handles random-effect-like within-col reinterpretations that lie in the fixed-effect column space without cost, but complex cross-block dependence remains outside its scope.

Suggested practical workflow for AI-econ applied work: 1. Compute design diagnostics (λn, Hn, Neff, and score-based analogues). 2. If concentration is substantial (high λn or small Neff), construct annihilating contrasts; in two-way settings, attempt cycle packing targeting high κC while keeping supports compatible with plausible error-blocking. 3. Run the sign-flip randomization test (exact or Monte Carlo) and invert for confidence intervals. 4. Report diagnostics, κC, chosen support granularity, and sensitivity to block partitioning (e.g., observation vs match vs firm-level blocks).

Overall, this paper supplies both a diagnostic lens and a practically implementable, finite-sample valid inference procedure for the concentrated-identifying-variation regime common in modern panel and networked datasets relevant to AI economics.