Vendors can satisfy visible AI-safety checks while keeping harms high: when customers are locked in and enforcement relies on vendor-controlled evidence, documentation and standardized tests become a compliance floor rather than a guarantee of mitigation; independent audits, portability and outcome-linked exposure are effective levers to restore accountability.

Main Finding

The paper shows that visible compliance (documentation, evaluations, meeting procurement floors) can coexist with low substantive harm mitigation when (i) enforcement depends on verifiable evidence, (ii) vendors control part of the evidence-generating infrastructure, and (iii) deployers face switching costs after integration. High switching costs weaken deployer monitoring, which lowers verifiable detection and therefore reduces the vendor’s incentive to invest in mitigation — producing a stable “proxy-compliance” equilibrium in which vendors stop at what is observable (the procurement floor) but under-mitigate relative to the social first best. Independent audit rights, portability (lower switching costs), incident-reporting channels, or outcome-linked liability can break this equilibrium by adding evidence channels the vendor cannot suppress or by making penalties independent of vendor-controlled detection.

Key Points

• Proxy-compliance mechanism (informal): s ↑ (lock-in) ⇒ deployer’s effective recourse falls ⇒ monitoring m falls ⇒ verifiable detection p(t,m) falls ⇒ vendor’s marginal exposure falls ⇒ vendor mitigation e falls ⇒ expected harm ↑.

• Formal definition: proxy compliance = equilibrium with t = t̄ (the procurement/compliance floor) but e < e_F B (social first-best mitigation).

• Core model ingredients

  • Three actors (vendor, deployer, regulator represented by enforcement parameters).
  • Timeline: vendor chooses auditability t and mitigation e; deployer observes and decides adoption and monitoring m; harm realized; evidence arises with probability p(t,m); regulator enforces if evidence exists.
  • Detection/mitigation primitives:
    • Harm probability q(θ,e) = max{0, θ − e} (mitigation reduces risk linearly).
    • Detection probability p(t,m) = 1 − exp(−α t − β m) (rises with auditability and monitoring).
    • Quadratic costs for t, e, m.
  • Switching cost s reduces deployer’s effective recourse δ_eff = δ/(1 + s).

• Main analytical results (intuitions & policy interpretations)

  • Prop. 4.4: With weak baseline independent detection (Δ = 0) and sufficiently high switching costs, proxy compliance exists. As s → ∞, monitoring → 0 and e* approaches a low level determined by vendor-controlled detection only.
    • Interpretation: lock-in can sustain visible-but-shallow compliance.
  • Prop. 4.5: Increasing independent detection Δ or lowering s raises mitigation e*. Portability reinforces the transition to accountability.
  • Thm. 5.1: For any fixed level of lock-in, there exists a finite independent-audit threshold Δ̄(s) such that Δ ≥ Δ̄(s) eliminates proxy compliance and attains the first-best mitigation. Audit rights are the most powerful single lever.
  • Prop. 5.2: Reducing switching costs (increasing portability, compatible tooling, exportable artifacts) raises deployer monitoring and thereby vendor mitigation.
  • Prop. 5.3: Effective incident reporting (coverage ρ and enforcement scale μ) creates an evidence channel that substitutes for vendor-gated detection and increases mitigation.
  • Prop. 5.4: Outcome-linked exposure (liability tied to realized harms) raises mitigation independent of detection technology because penalties do not depend on vendor-controlled evidence.

• Equilibrium structure: vendor optimally sets t = procurement floor t̄ (no extra auditability beyond the visible requirement) and chooses e to balance mitigation costs against expected enforcement exposure (which depends on Δ, p(t̄,m(e)), and the monitoring response). The deployer chooses m given e and s; m* falls as s increases.

• Worked numerical example and (s, Δ) diagram illustrate three regimes: proxy-dominant, mixed, and accountability (first-best).

Data & Methods

• Approach: theoretical political-economy model (sequential game) solved for subgame-perfect equilibrium. Comparative statics and policy counterfactuals are derived analytically; proofs and full derivatives are in appendices.

• Key functional assumptions (tractable specification used for comparative statics and examples):

  • q(θ,e) = max{0, θ − e} (harm probability).
  • p(t,m) = 1 − exp(−α t − β m) (evidence/detection probability).
  • Cost functions c_j(x) = k_j x^2 / 2 (quadratic costs for t, e, m).
  • Procurement/compliance floor t̄: vendors must meet at least t̄ to be adopted.
  • Enforcement and regulator summarized by parameters: λ_F (evidence-based penalty scale), Δ (independent audit capacity / audit rights), μρ (effective incident-reporting exposure), λ_OFO (outcome-linked exposure), and δ (recovery/exposure fraction), with switching cost s reducing δ to δ_eff = δ/(1 + s).

• Solution techniques:

  • Backwards induction: deployer monitoring best-response derived (unique interior solution).
  • Vendor maximizes adoption benefit minus costs and expected penalties after anticipating m(e), leading to closed-form expression for interior e in terms of enforcement exposure H(t̄,m*) + Δ.
  • Characterization of unique interior equilibrium and the corner (full mitigation) with comparative statics for institutional parameters.
  • Numerical worked example to illustrate magnitudes and regime boundaries.

• Empirical orientation: the paper maps model primitives to measurable institutional features (procurement floors, API/weight access, portability standards, reporting coverage, liability structures) and suggests testable implications for monitoring, mitigation, and the compliance–outcome gap.

Implications for AI Economics

• Explains a common empirical puzzle: widespread documentation, model cards, and standardized tests can coexist with recurring post-deployment harms because vendors rationally stop at visible compliance when enforcement hinges on vendor-controllable evidence and deployers are locked in.

• Policy priorities and their economic channels:

  • Strengthen independent audit rights (Δ): most direct route to raising vendor mitigation even under high lock-in — but requires genuine, enforceable independence (internal access, raw evaluation logs), not just nominal audit processes.
  • Increase portability / lower switching costs (s): restores deployer bargaining power and raises monitoring, creating an indirect but robust increase in vendor mitigation; standards and exportable tooling can achieve this.
  • Build robust incident-reporting channels (μρ): adds an evidence channel that vendors cannot gate, improving enforcement exposure even if audits remain partial.
  • Shift toward outcome-linked liability (λ_OFO): aligns vendor incentives with realized harms and can be effective even when detection is weak because penalties aren’t conditional on vendor-supplied evidence.
  • Procurement rules matter: if procurement floors t̄ are the main observable requirement, vendors will minimally satisfy them rather than exceed them to improve safety; procurement design should be paired with independent detection or outcome-based metrics.

• Market structure and concentration: platform/foundation-model concentration amplifies the lock-in effect; regulatory reforms that unintentionally raise compliance costs without addressing portability or independent detection may deepen concentration and worsen mitigation incentives.

• Empirical tests suggested by the model:

  • Cross-firm/regime comparisons: relate measures of vendor-controlled auditability (API restrictions, access to weights/data), deployer switching costs (integration depth, fine-tuning dependence), and independent audit capacity to observed mitigation actions and realized harms.
  • Natural experiments: procurement rules that raise t̄ without improving Δ should show increased visible compliance but little change in operational harms; reforms that increase Δ or lower s should reduce harms.
  • Measurement proxies: detection p can be proxied by third-party audit access and published red-team artifacts; mitigation e by internal safety interventions disclosed or inferred from product behavior; monitoring m by deployer incident logs or internal compliance spending; switching costs s by integration time/costs and downstream tooling lock-ins.

• Research & regulatory guidance:

  • Design audit regimes to give auditors access that is not vendor-gated (true Δ), and pair transparency requirements with enforcement capacity (λ_F) to make disclosure meaningful.
  • Complement documentation mandates with enforceable portability and reporting rules, and consider expanding liability models that do not depend solely on verifiable vendor evidence.
  • Be cautious that well-intentioned visible-compliance mandates (without parallel reform) can become cheap signals that reinforce staying at the procurement floor.

Limitations and scope - Scale (e.g., market concentration) is an exogenous setting in the paper; dynamic entry/competition is not modeled. - The regulator is summarized through parameters rather than modeled as a strategic actor; political constraints on resourcing and capture are not endogenized. - The functional forms (linear risk reduction, exponential detection, quadratic costs) are chosen for tractability; qualitative mechanisms are robust but quantitative thresholds depend on specification.

Overall, the paper delivers a concise political-economy explanation for why formal responsible-AI signals can fail to translate into reduced harms and identifies institutional levers (independent audits, portability, reporting, liability) that can restore substantive accountability.