Most inference-time controls already exist in production, but they are fragile: 15 of 20 mechanisms have commercial substrates today, yet none withstand high-capability or state-level adversaries and many enforcement primitives are defeated by fine-tuning, implying regulators cannot rely on training-compute thresholds alone.

Main Finding

Training-focused compute governance is necessary but no longer sufficient: a substantial portion of AI capability now emerges at inference time (via inference scaling, agentic scaffolding, and compression to consumer hardware). Ansari develops a structured feasibility taxonomy of 20 inference-time governance mechanisms (monitoring, verification, enforcement), rates them against real-world vendor evidence on a four-point readiness scale, and evaluates them against a two-dimensional adversary model and four governance scenarios. While 15 of 20 mechanisms have commercial technical substrates in production today, their assurance and adversarial robustness are uneven: mechanisms generally work against cooperative deployers and low-to-medium-capability users but fail against high-capability, state-level deployers, and many enforcement primitives are undermined by fine-tuning.

Key Points

• Why inference matters

  • Capability can migrate to deployment through: (1) inference scaling (repeated sampling, longer chains of thought, ensembles), (2) agentic scaffolding (tool use, memory, external resources), and (3) compression to consumer hardware.
  • Training-compute thresholds (e.g., EU AI Act, US reporting regimes) are silent on inference and can be bypassed via inference-side techniques.

• Taxonomy and readiness

  • 20 inference-time governance mechanisms organized into Monitoring, Verification, and Enforcement clusters.
  • New verification primitive added: V7 — chain-of-thought monitorability.
  • Four-point readiness scale: 1) currently deployable (commercial substrate in production), 2) near-term, 3) requires R&D, 4) speculative.
  • Conservative rating rule: where evidence conflicts, the more conservative rating is used.

• Empirical readiness

  • 15 of 20 mechanisms have commercial technical substrates in production at major providers.
  • Readiness varies by adversary role and capability: most mechanisms are adequate only against cooperative deployers and low-to-medium-capability end users.
  • No mechanism in the taxonomy is rated adequate against a high-capability state-level deployer.
  • Fine-tuning and post-distribution modification can remove model-internal enforcement mechanisms; platform-external controls may still function.

• Adversary model & scenarios

  • Two-dimensional adversary model: 3 capability tiers × 4 adversary roles (developer, deployer/integrator, end user, fine-tuner/scaffold-builder).
  • Mechanisms mapped to four governance scenarios: domestic regulation, bilateral/multilateral coordination (treaties, shared certs), industry self-regulation, and compute-marketplace/platform governance.

• Substitution analysis

  • A convergence/substitution analysis links inference-stage mechanisms to hardware-stage mechanisms (from a companion hardware governance paper) and formulates a conditional substitution principle specifying when inference controls can substitute for hardware controls.

• Method reliability & scope

  • Ratings derived from a documented four-vendor evidence base (commercial providers); consumer-hosted/open-weight deployments are treated as coverage limitations.
  • Second-rater check on a random subset (7 mechanisms) yielded quadratic-weighted Cohen’s kappa = 0.74 (substantial agreement); unweighted kappa = 0.53 (moderate).
  • Full adversary and scenario matrices are provided in appendices.

Data & Methods

• Method type: feasibility taxonomy and qualitative-quantitative readiness assessment grounded in vendor documentation and academic literature.
• Mechanism set: 20 inference-time mechanisms classified into Monitoring, Verification, Enforcement (detailed definitions and per-mechanism descriptions in Section 3).
• Evidence base: documented vendor/provider sources from four major commercial providers (snapshots saved; listed in Appendix D).
• Readiness rating procedure:
  • Four-point scale (1 = currently deployable; 4 = speculative).
  • Conservative decision rule: when sources disagree, adopt more conservative rating; when academic sources silent on adversarial conditions, rate against adversarial (not cooperative) case.
• Adversary model: two-dimensional—capability tiers (low/medium/high) crossed with adversary role (developer, deployer, user, fine-tuner/scaffold-builder). Ratings are scoped by adversary surface.
• Validation: independent second rater assessed a random subset of mechanisms (7/20); inter-rater reliability measured with quadratic-weighted Cohen’s kappa = 0.74. Disagreements documented in appendices.
• Coverage limitations: focus on commercial-provider-mediated inference; consumer hardware and unfederated open-weight deployment are handled as cases where platform-mediated mechanisms lose force (flagged per mechanism).

Implications for AI Economics

• Regulatory coverage and market leakage

  • Training-centric regulation can create regulatory arbitrage: firms (or states) can achieve frontier capabilities cheaply via inference-side techniques or local (consumer) deployment. This weakens the efficacy of ex ante capital controls on training compute and shifts the emphasis to post-training controls and marketplace governance.
  • Open-weight models and consumer-hardware deployment reduce chokepoints, increasing the likelihood of spillovers and making market-based enforcement (platform intermediaries, KYC for API purchasers) less effective. Economically, this favors decentralization and may reduce marginal enforcement costs but increases negative externalities.

• Market structure and gatekeeper power

  • Platform- and marketplace-mediated inference controls (attestation, per-request monitoring, KYC) increase the regulatory leverage of major cloud providers and marketplaces, potentially reinforcing their market power. Policymakers will face trade-offs between relying on these intermediaries (effective enforcement) and preserving competition and innovation.
  • If regulators require platform-mediated controls to manage inference risk, new compliance costs and switching frictions could entrench incumbents or raise barriers to entry.

• Compliance costs, incentives, and strategic behavior

  • Firms will weigh the cost of building inference governance (monitoring infrastructure, attestations, secure enclaves) against the expected compliance and liability costs. Where enforcement is weak against high-capability adversaries, firms may face incentives to underinvest in governance, particularly for deployments aimed at opaque or adversarial actors.
  • Fine-tuning capability undermines many model-internal enforcement mechanisms. Economic actors could intentionally delegate risky modifications to downstream fine-tuners to evade obligations—creating moral hazard across the developer–deployer–user chain.

• International coordination and public-good problems

  • No single inference-time technical mechanism is adequate vs. high-capability state actors; effective mitigation requires international coordination (mutual recognition, treaty verification, export controls). This elevates collective-action and public-good problems: countries must invest in joint monitoring and enforcement to avoid a global race-to-the-bottom.
  • Economic sanctions, trade measures, or cross-border platform restrictions may become the practical tools to deter state-level misuse—each with significant geopolitical and economic costs.

• Innovation, investment, and R&D priorities

  • The finding that 15/20 mechanisms have commercial substrates suggests near-term feasibility for many governance tools, but the assurance and adversarial robustness gaps indicate R&D priorities: hardening verification (e.g., robust attestation, chain-of-thought monitorability), improving enforcement that survives fine-tuning, and privacy-preserving monitoring.
  • Markets (venture, corporate R&D) are likely to fund solutions where regulatory demand or liability risk creates commercial opportunity (e.g., attestation providers, secure inference enclaves, certified toolchains). Public R&D may be needed for mechanisms that are socially valuable but not privately profitable (robust verification vs. state-level adversaries).

• Liability, insurance, and pricing of inference

  • If inference governance is required, providers may price in monitoring/attestation services and KYC; risk-based pricing of API access (or tiered access) could emerge, with higher compliance costs for high-risk use cases.
  • Insurance markets may develop to underwrite inference-related harms, but insurers will demand measurable controls and auditability—driving standardization of verification mechanisms. The uneven robustness against powerful adversaries could make insurance coverage limited or expensive for high-risk deployments.

• Policy design tradeoffs

  • Regulators must choose layers to regulate: training compute, inference calls, marketplaces, or hardware. The paper’s conditional substitution principle implies economic trade-offs: hardware-layer controls can be more durable against some forms of circumvention but are costly and less feasible globally; inference-layer controls are implementable via intermediaries but vulnerable to decentralization and fine-tuning.
  • Differential regulatory instruments (e.g., mandatory attestation + export restrictions + targeted sanctions) may be needed to cover different adversary/scenario combinations, but this creates compliance complexity and potential uneven regulatory burdens across firms and countries.

Overall, the paper signals that economic analysis of AI governance must expand beyond training-compute-focused models to incorporate inference-stage dynamics: altered incentive structures, new gatekeepers, arbitrage paths, and public-good and international coordination challenges.