Regulatory audit metrics can be gamed by routing content through equivalent variants, undermining safety claims; a provably minimal 'semantic-envelope' repair restores certificate guarantees in the authors' formal model and synthetic tests, though real-world validation remains needed.

Main Finding

Auditors who publish scalar safety metrics for recommender systems risk Goodhart-style gaming: platforms can rearrange which semantically equivalent content variant is served to reduce the measured score without reducing actual harm. The paper shows that (1) any metric that scores variants directly is manipulable whenever within-class variant scores differ; (2) a conservative repair—the semantic-envelope, which assigns each variant the maximum score observed in its semantic class—is the unique pointwise-minimal classwise-constant fix that restores invariance to within-class manipulation; and (3) under this envelope one can derive a class-stratified, strategy-agnostic certificate bounding true harmful exposure up to published coverage and annotation error terms. Synthetic and formal-method stress tests show the fragile (direct-scoring) metric routinely violates useful certificates, while the semantic-envelope does not in tested instances.

Key Points

• Policy context: Ofcom (UK) and EU/DSA guidance increasingly treat scalar exposure metrics as evidence of compliance for recommender systems; this converts metrics into optimization targets for platforms.
• Threat model (narrow, explicit): platform-side within-class representation choice only — the auditor publishes a transformation graph (candidate variant pairs, attribute checklist, validation scores, threshold) that induces semantic classes; platform can choose distribution over variants but cannot change the published protocol.
• Definitions:
  • Manipulation invariance: metric depends only on semantic-class mass, not on which variant within a class is served.
  • Certification: a (γ, β)-certificate guarantees H(x) ≤ γ M(x) + β for every admissible platform strategy x.
  • Class-coverage certificate: auditor publishes per-class lower bounds (P_c) and a strictness ε; useful when τ/ε + β < 1 (τ = audit budget).
• Fragility result: direct per-variant scoring (the “fragile metric”) fails manipulation invariance whenever two variants in the same class have different scores — i.e., trivial to game.
• Semantic-envelope repair: Env(m)(v) = max_{u in cl(v)} m(u). Properties:
  • Restores manipulation invariance (metric depends only on class mass).
  • Is the unique pointwise-minimal conservative classwise-constant repair (it raises scores only as much as necessary to guarantee classwise constancy and never above the observed class max).
• Certification theorem: given harm-pure classes and per-class coverage α̂ (min over harmful classes of the maximal observed score in each class), Env yields a certificate H*(x) ≤ (1/α̂) MEnv(m)(x) + η̄, where η̄ absorbs annotation and protocol disagreement. The paper extends this to imperfect (non-pure) classes via a disagreement mass Δ(x) that is published/upper-bounded.
• Protocol sensitivity: tightening validation threshold or pruning edges refines the partition and weakly decreases envelope scores pointwise; auditors should publish sensitivity analyses (e.g., max path length, weakest-edge confidence) so transitive merging costs are visible.
• Experiments & verification: conducted synthetic stress tests and cross-checked using exhaustive enumeration on finite-state mixed-strategy grids, SMT encodings in Z3 and cvc5, and bounded single-player MDPs in PRISM-games. Compared semantic-envelope to fragile (direct) scoring and a class-mean repair.
  • Empirical outcome: fragile metric repeatedly violated envelope-style certificates with large gaming gaps under fixed audit budgets; class-mean gave looser certificates; semantic-envelope showed no violations in tested instances.

Data & Methods

• Modeling:
  • Finite variant set V; published transformation family T0; attribute-preservation predicate A(v,u); human validation scores s(v,u); acceptance threshold ρ; admissible edge set E_ρ; semantic classes C = connected components of closure(E_ρ).
  • Platform strategy x ∈ Δ(V) (probability distribution over variants), utility u(v) arbitrary; auditor reports metric M_m(x) = Σ_v x_v m(v) and enforces budget M_m(x) ≤ τ.
  • Harm labels: ideal-case per-class harm h(c) (harm-pure classes) and more realistic latent per-variant harm h(v); disagreement mass Δ(x) = Σ_v x_v |h(v) − ĥ(cl(v))| captures labeling/protocol error.
• Theoretical results:
  • Propositions and theorems proving manipulability of direct scoring, optimality/minimality of the semantic-envelope among conservative classwise-constant repairs, derivation of class-stratified certification inequalities, and monotonicity under protocol refinement.
  • Extensions that quantify slack from imperfect annotation and transitive-closure risks (publishable η̄ and Δ(x) terms).
• Empirical / formal verification:
  • Reproducible synthetic catalogs with harmful classes containing variants with different detector scores.
  • Solved adversarial platform best-responses (LP/MDP) under audit budget constraints to produce worst-case violations.
  • Cross-checked encodings: exhaustive enumeration on finite grids, SMT in Z3 and cvc5, bounded-MDP via PRISM-games.

Implications for AI Economics

• Measurement-as-instrument creates incentives: treating a scalar metric as compliance evidence makes it an instrument that firms will optimize against, not just a passive report. This alters platform incentives and creates a regulatory-design externality that must be internalized by auditors and regulators.
• Audits are strategic games; measurement robustness matters economically:
  • Direct-scoring metrics lower audit enforcement effectiveness and can render regulatory thresholds vacuous, allowing platforms to maintain utility while masking harm via representation choices.
  • The semantic-envelope is economically attractive because it prevents simple low-cost gaming (variant switching) and yields stronger pre-declared certificates that regulators can credibly rely on.
• Trade-offs and costs:
  • Conservatism vs. looseness: envelope increases measured scores (worst-case per class), which tightens enforcement but may require lower audit budgets or higher reported exposure. That can impose greater compliance costs on platforms (or require more stringent thresholds to attain the same allowed exposures).
  • Audit design costs: building and validating transformation families, running human validation studies, and publishing sensitivity statistics (path lengths, weakest-edge confidences) are non-trivial. Regulators must budget for these institutional costs.
  • Enforcement/operational cost on platforms: platforms may invest in representation engineering to circumvent audits where possible (if protocols are weak), or conversely invest in reducing truly harmful exposure if audits are robust.
• Market structure and competition:
  • Platforms with superior control over representation (better paraphrasing, thumbnail selection, localized renderings) can game fragile metrics more easily, potentially distorting competition if regulators do not adopt robust repairs.
  • Robust auditing raises barriers to low-cost gaming and may favor platforms that actually reduce harm, shifting market incentives toward safer designs.
• Regulatory capture and protocol bargaining risk:
  • The model presumes a published protocol; in practice, platforms may lobby to shape T0, A, s, ρ. The paper explicitly excludes protocol renegotiation from its core model; economically, this is a key vulnerability—protocol design itself is a bargaining object with potential for capture.
• Policy recommendations with economic interpretation:
  • Treat audit metrics as security objects and publish the full protocol (transformations, validation scores, threshold) so that the audit surface is explicit and not manipulable by secrecy.
  • Use semantic-envelope (max-over-class) repairs for classwise invariance; publish per-class coverage α̂ and an upper bound on disagreement mass η̄ so certificates are meaningful and auditable.
  • Require auditors to publish sensitivity analyses (vary ρ, report weakest edge/confidence and max path length) so the marginal cost of transitive merging and annotation noise is transparent—this reduces hidden uncertainty that platforms could exploit.
• Open economic research directions:
  • Modeling multi-platform competition where platforms can strategically relocate harmful content across platforms or catalogs.
  • Endogenizing protocol design: studying regulator–platform bargaining over T0, A, s, ρ and implications for welfare, enforcement efficacy, and potential capture.
  • Quantifying costs of audit construction and human validation, and optimizing audit budgets τ versus statistical power/robustness.
  • Dynamic arms races: platforms may invest in automated paraphrase generation to produce low-scoring admissible variants; dynamic models of investment and enforcement would illuminate long-run equilibria.
• Practical takeaway for regulators and economists: scalar safety metrics must be hardened against obvious optimization channels; conservative, publishable repairs like the semantic-envelope provide a principled, minimally invasive way to restore manipulation invariance and produce credible predeclared certificates, but they change compliance incentives and impose measurable costs that regulators and policymakers should budget for and analyze.