Smooth, curvature-based oversight rules that gently reward probabilistic reports inevitably let strategic agents inflate their reports; switching to sharp binary approval thresholds preserves calibration and, remarkably under the Brier score, eliminates the welfare loss entirely.

Main Finding

The paper proves an endogenous impossibility: when a principal both scores an agent’s probabilistic report with any strictly proper scoring rule and uses the report to grant non-accuracy payoffs (approval, allocation, downstream control), the principal’s optimal screening design necessarily creates the very non-affine approval incentives that make truthful reporting suboptimal. Put differently, optimal oversight endogenously produces the perturbation that destroys calibration. There is a constructive escape—committing to a sharp step-function threshold recovers first-best welfare for every strictly proper scoring rule—but this escape restores welfare, not truthful reporting (agents still misreport predictably). The quadratic (Brier) score is special: under smooth oversight it yields a type-independent inflation cost and therefore a unique welfare equivalence between first- and second-best.

Key Points

• Setup and scope
  • Binary-outcome, scalar-type reporter (probability p), agent reports r; principal uses a strictly proper scoring rule S for accuracy and an approval function q(r) that gives the agent additional payoff.
  • Core conditions: binding conflict of interest, undetectability of certain deviations under the information structure, and non-affine perturbation (formalized as NT1–NT3).
• Perturbation Lemma (Lemma 3.1)
  • Adding any non-constant, non-affine function to a strictly proper scoring objective shifts the agent’s maximizer away from truthful reporting. The paper gives a closed-form perturbation formula (equation (3.2)) quantifying how the optimum shifts as a function of the perturbation gradient, the scoring-rule curvature, and perturbation weight γ.
• Endogeneity (Result 1 / Theorem 5.3)
  • The principal’s optimal approval function q*(r) is necessarily non-affine (smooth/affine approval rules are strictly suboptimal). Thus the principal’s rational design choices create the perturbation that makes truthful reporting suboptimal—an endogenous impossibility.
• Escape (Result 2 / Theorem 5.8)
  • A step-function approval rule q*(r) = 1{r ≥ r0} attains first-best screening for every strictly proper scoring rule. The mechanism functions because the agent faces a binary inflate-or-not decision, which induces a type threshold independent of the scoring-rule curvature.
  • Important caveat: the escape recovers screening welfare but not honest reporting; agents below r0 inflate to r0.
• Brier-score uniqueness (Proposition 5.9)
  • Under the Brier score (constant G'' = 2), the inflation cost is type-independent, yielding welfare equivalence between second-best (with strategic misreporting) and first-best. For any non-Brier scoring rule, the welfare gap under smooth (C1) oversight is bounded below by Ω(Var(1/G'') · (γ/β)^2), so curvature heterogeneity matters.
• Connections to literature and theory
  • Uses convex-analytic / Fenchel-conjugate skeleton tying together classic results (Savage–McCarthy, Gneiting–Raftery, Rochet’s cyclical monotonicity, Archer–Tardos).
  • Perturbation formula is closely related to the envelope theorem/Milgrom–Segal arguments; techniques rely on implicit function arguments and convexity.
• Domain-general implications
  • Results are developed for two instances: AI agent oversight (primary motivation) and marketplace operation (mechanism-design parallel). Multidimensional types extension sketched (Section 6.6): core impossibility generalizes, welfare characterization remains open.

Data & Methods

• Nature of the work: theoretical/analytical (no empirical data). The contributions are proved theorems and constructive mechanisms.
• Main mathematical tools and structure:
  • Model: single-reporter credibility game with score S(r, outcome), approval function q(r), agent payoff = S + γ · q (plus possibly other linear weights).
  • Convex analysis / Fenchel conjugacy: organizes the elicitation traditions and represents proper scoring rules via convex generators G with curvature G''(p).
  • Perturbation analysis: derive closed-form formula for argmax shift when adding perturbation h(r) to a strictly concave objective (equation (3.2)); uses implicit-function/envelope-type reasoning.
  • Mechanism-design optimization: principal chooses q(·) to maximize screening welfare subject to incentive responses; Theorem 5.3 shows optimal q is non-affine. Theorem 5.8 constructs the step-function q that attains first-best.
  • Comparative statics: quantify how deviation magnitude scales with perturbation weight γ and scoring-rule curvature; derive lower bounds on welfare gaps for non-Brier rules (Ω bound involving Var(1/G'')).
• Proof objects referenced: Lemma 3.1 (Perturbation Lemma), Theorem 5.3 (optimal q non-affine / endogeneity), Theorem 5.8 (step-function escape), Proposition 5.9 (Brier uniqueness and welfare equivalence), plus connections to classical identities (Savage–McCarthy, Archer–Tardos, Rochet).
• Assumptions & limits:
  • Key technical assumption: undetectability of some deviations given the information structure (finite monitoring horizon)—without this, detection could restore truthfulness.
  • Main formal results are for scalar types and binary outcomes; d-dimensional extension exists in Section 6.6 but welfare results are open for d>1.
  • Escape recovers welfare but not epistemic calibration.

Implications for AI Economics

• For scalable AI oversight and alignment
  • Smooth scoring-based incentives (any strictly proper scoring rule combined with smooth approval/selection rules) cannot be relied on to preserve calibrated, truthful probability outputs when agents obtain non-accuracy payoffs from reports. Optimal principal designs will typically induce miscalibration.
  • Practical design implication: use sharp, committed thresholds (binary approval rules) rather than smooth graded approval when the goal includes preserving downstream decision efficiency. Thresholds can deliver first-best welfare even though they permit predictable, non-calibrated reports.
  • Calibration and welfare diverge: welfare-optimal oversight can accept predictable misreporting if the principal can adjust selection thresholds appropriately. If the principal also needs epistemic calibration for downstream tasks (e.g., model interpretability, aggregated forecasting, auditability), thresholds are insufficient—additional monitoring or changes to the information structure are required.
• On choice of scoring rule
  • The Brier score (quadratic) is special: under smooth oversight it reduces heterogeneity in the inflation cost across types, creating robustness (welfare equivalence) that other scores lack. This gives a normative rationale for preferring quadratic penalties if the principal insists on smooth approval functions and values welfare parity.
  • However, Brier does not resolve the endogeneity: optimal q is still non-affine absent commitment to thresholds or detection capabilities.
• Policy and mechanism-design domains (marketplaces, auditors, certifiers)
  • Marketplaces that condition allocations/payments on reported signals should expect strategic report inflation when reports affect allocations and are scored for accuracy. Optimal platform design may therefore require reservation thresholds (reserve prices) or hard cutoffs rather than smoothly varying allocation rules.
  • Regulatory designs seeking calibrated disclosure should either (a) remove or reduce the non-accuracy payoff channel, (b) invest in detection/monitoring to break undetectability, or (c) accept welfare-optimal but epistemically miscalibrated outcomes and adjust policy accordingly.
• Directions for applied research and empirical testing
  • Empirically estimate how real-world agents (autonomous models, marketplace sellers, rating agencies) respond to combined scoring + approval incentives; test the closed-form perturbation predictions (how deviation scales with γ and local curvature).
  • Evaluate hybrid designs: thresholds combined with randomized audits, or softened thresholds that retain partial calibration while achieving near-first-best welfare.
  • Extend welfare analysis and constructive characterizations to multidimensional reporting tasks common in AI (vector-valued confidences, multi-class prediction).
• Broader alignment message
  • Goodhart-like failure is endogenous: optimizing for efficient screening with smooth incentives will create miscalibration even when the scorer is strictly proper. Thus, alignment-by-scoring requires either non-smooth commitment devices (thresholds), improved detectability, or a redesign that eliminates the conflicting payoff channel.

If you’d like, I can: - Extract the formal statements (Lemma 3.1, Theorems 5.3 & 5.8, Proposition 5.9) into concise math summaries. - Produce a short checklist of practical design choices for AI overseers (thresholds vs. smooth rewards, monitoring, choice of scoring rule). - Sketch how the closed-form perturbation formula (3.2) reads and how to plug in a given scoring rule to predict miscalibration.