When advisers may lie, the optimal response is a 'trust region'—accept recommendations only if the reported belief falls inside a calibrated band and otherwise clamp them to the nearest trusted boundary; beyond explicit alignment thresholds, even partially truthful advice strictly improves worst-case payoffs.

Main Finding

When an adviser (e.g., an AI) is truthful with known probability α and otherwise can send arbitrary messages, the agent’s optimal robust decision rule is a “trust-region” policy in belief space. The agent accepts reported beliefs at face value if they lie inside a compact trust region T, and otherwise clips the report to the nearest point on T’s boundary (nearest in the Bregman distance induced by the agent’s indirect payoff). This clipping limits how much any recommendation can move actions and therefore guards against worst-case misalignment. The paper gives a full characterization of these policies, proves a minimax (saddle-point) result establishing equilibrium existence, and derives sharp alignment thresholds (including closed-form solutions in binary-state and binary-action environments).

Key Points

• Model setup
  • Finite state space Ω (|Ω| = N) with prior µ0; adviser observes a signal s (posterior belief) and reports a message m ∈ ∆(Ω).
  • With probability α the adviser is aligned and truthfully reports his posterior; with probability 1 − α the adviser is misaligned and may send any message.
  • The agent has private type θ (private signal) independent of adviser signal conditional on state, chooses action a ∈ A, and evaluates strategies by the worst-case expected payoff: V(σ) = α E[id,σ][u] + (1 − α) infβ E[β,σ][u].
• Trust-region characterization
  • Any optimal robust strategy is equivalent to a trust-region strategy (TRS): there exists a compact T ⊂ ∆(Ω) such that
    • if reported m ∈ T, the agent plays a Bayes-optimal action for belief m;
    • if m ∉ T, the agent acts as if her belief were P(m) ∈ T, where P(m) minimizes the Bregman distance DU(m, m′) over m′ ∈ T (U is the agent’s indirect payoff).
  • Interpretation: moderate reports are trusted; extreme reports are clipped to a safe boundary point.
• Mathematical structure and tools
  • Indirect payoff U(µ) = maxˆσ E[u | belief µ] is convex; Bregman distance DU(·, ·) associated with U determines the “closest safe interpretation.”
  • A minimax theorem holds: there exists a saddle point (trust-region equilibrium) where the agent’s TRS and an adversarial misaligned strategy form mutual best responses. This yields a certificate: exhibit an adversarial reporting strategy to verify optimality.
• Threshold/phase-transition results
  • Minimal viable alignment: threshold α* above which the agent can strictly improve on relying only on private information.
  • General bound: if information is useful, α > 1/2 is sufficient for advice to be valuable; in binary-state problems this bound is tight.
  • In high-dimensional state spaces, α* can be much smaller (potentially as low as 1/N), so advice can be robustly valuable even at low alignment probabilities.
• Special cases
  • Binary-state: trust region is an interval around the prior; if α ≤ 1/2 the interval collapses to the prior (ignore advice); if α > 1/2 and information is sufficiently valuable, there is a unique interval expanding with α.
  • Binary-action and no private information: solution often all-or-nothing—either full trust (T = ∆(Ω)) or ignore adviser (T = {prior}). A threshold alignment probability determines the regime; typically α ≤ 1/2 means no robust benefit.
• Practical/formal implications
  • The agent never benefits from distorting her use of private information; robust rules restrict Bayes-optimal action rules.
  • Implementation does not require commitment: TRS arises endogenously as a saddle point.
  • The approach gives a principled justification for “extreme-belief aversion” (discounting extreme outputs).

Data & Methods

• Methodology
  • The paper is theoretical and analytical (no empirical data). It formulates a game-theoretic/worst-case information-design model and derives characterizations and comparative statics.
  • Robust criterion: alpha-max-min (Hurwicz-style) — agent maximizes the weighted average of the aligned-expectation and worst-case misaligned-expectation, with weight α.
• Key technical components
  • Definition and use of the indirect utility U(µ) (agent’s value from optimally using private information given interim belief µ).
  • Convex analysis of U and Bregman distances DU(m, m′) = U(m) − U(m′) − ∇U(m′)·(m − m′) to define distance-to-trust-region and clipping mapping P(m).
  • Minimax/saddle-point existence: proves that supσ infβ V(σ, β) = infβ supσ V(σ, β), yielding trust-region equilibria.
  • Special-case closed-form solutions: binary-state (interval trust region), binary-action/no-private-info (all-or-nothing threshold), and parametric examples (e.g., medical triage) illustrating computation of T.
• Assumptions (important for applicability)
  • Adviser truthfully reports with probability α; misaligned adviser can send arbitrary messages (modeled adversarially).
  • Agent’s private signal and adviser signal are conditionally independent given state.
  • State space finite (analysis allows continuous signal spaces); action and type spaces compact; utility continuous in actions.
  • U is convex; differentiability or uniqueness of Bayes-optimal actions used in deriving gradients/Bregman distances in some results.

Implications for AI Economics

• When to trust AI outputs
  • Use the trust-region principle: accept moderate/confident-but-typical model outputs; clip or discount highly extreme or unusual outputs to a boundary interpretation before taking high-stakes actions.
  • This gives a formal basis for interface designs that suppress or flag extreme confidence (e.g., cap probabilities, present bounded posterior ranges).
• Minimum alignment required for benefit
  • If α (probability of correct/truthful alignment) exceeds about 1/2, advice is generally worth using; in high-dimensional problems, lower α can suffice (possibly as low as 1/N). Thus, in complex, rich-state problems, even weakly aligned systems can add robust value.
  • For simple binary decisions without private information, α ≤ 1/2 typically implies no robust gain; designers should be cautious about deploying such advisers unless alignment is sufficiently high.
• Robust design and certification
  • The minimax/saddle-point result offers a certification recipe: to justify a decision policy, construct an adversarial reporting strategy that makes that policy a best response—this provides a provable guarantee under worst-case misalignment.
  • Trust-region policies do not require prior commitment and can be implemented reactively based on reported beliefs and pre-computed trust sets.
• Human-AI interaction and delegation
  • The framework formalizes safe delegation: the agent effectively delegates only within a trusted subset of the model’s outputs; outside it, the agent reinterprets outputs conservatively.
  • Guidance for what to reveal to AI: because agent’s private information is valuable and U is known, the agent should generally not disclose private signals to the adviser if misalignment is possible.
• Policy and product design recommendations
  • Estimation: estimate α (alignment probability) and adviser signal distribution τ to compute trust regions; if α uncertain, conservative (smaller) trust regions are warranted.
  • Interface defaults: clip extreme probabilities, show bounded confidence intervals, or convert extreme outputs into conservative recommendations.
  • Monitoring: focus testing on adversarial behaviors that would exploit sensitivity to extreme reports; use adversarial stress tests to identify effective trust-region boundaries.
  • Regulation: regulators can require provable robust guarantees (e.g., exhibit adversarial strategies that certify deployed trust-region policies) for high-stakes AI systems.
• Broader theoretical implications
  • Connects misalignment robustness to information-design and delegation literatures, and provides a formal microfoundation for behavioral phenomena like extreme-belief aversion.
  • Suggests that the dimensionality of decision problems materially changes alignment requirements—an important consideration in economic assessments of AI deployment risk.

If you want, I can (a) compute a trust region for a simple parametric example (e.g., binary-state medical triage) given payoff numbers and an α, or (b) outline an algorithm to numerically compute T given an arbitrary utility U(·) and adviser signal distribution. Which would be most useful?