Value alignment is a governance problem, not just an engineering puzzle: misalignment stems from objective-setting, information flows and whose interests count, so durable solutions require institutions and contested decision‑making rather than only model tweaks.

Main Finding

Value alignment for AI is primarily a structural governance problem, not a single technical or purely normative one. Using the principal–agent framework, the paper shows misalignment arises along three interacting axes—objectives, information, and principals—and therefore cannot be “solved” by model design alone. Alignment is pluralistic and context-dependent: resolving it requires ongoing institutional processes that negotiate which interests count, how objectives are set and verified, and what trade-offs are acceptable.

Key Points

• Structural definition: Value alignment problems are instances of a principal–agent problem in human→AI delegation that occur when (a) the AI agent’s objective function is mis‑specified, or (b) there are informational asymmetries between human principals and the AI agent.
• Three orthogonal but interacting axes:
  • Objectives axis: proxy mis‑specification, reward hacking, perverse incentives and the usual “outer alignment” issues.
  • Information axis: opacity, hidden actions, non‑verifiability, distributional shift, and other informational asymmetries that prevent principals from observing or verifying agent behaviour.
  • Principals axis: plurality and conflict among stakeholders (developers, users, affected communities, regulators, shareholders) so that “aligned” for whom becomes contested.
• Interaction effects: misalignment along one axis can exacerbate the others (e.g., biased proxies both misstate objectives and conceal harms from affected stakeholders).
• Human→AI special case: unlike human–human agency, AI agents have no intrinsic values; misalignment often stems from imperfect specification of objectives and institutional gaps rather than from independently motivated agents.
• Scaling hypothesis for value‑aligned AI: as model generality, deployment scope, and stakeholder diversity increase, informational asymmetries, value conflicts, and power imbalances systematically amplify, making alignment management harder.
• Practical conclusion: technical methods (RLHF, interpretability, robustness, benchmarks) matter but must be embedded in governance arrangements—contractual design, accountability, participatory processes, dispute and remediation mechanisms—to manage ongoing misalignment and trade‑offs.

Data & Methods

• Paper type: conceptual/theoretical analysis and literature synthesis (FAccT 2026 conference paper; arXiv preprint).
• Core method: apply and extend the principal–agent (agency) framework from economics to human–AI delegation.
  • Uses canonical agency concepts (moral hazard, adverse selection, non‑verifiability) to characterize the information axis.
  • Maps outer alignment / objective specification issues to the objectives axis.
  • Incorporates pluralistic and social‑choice insights to formalize the principals axis.
• Draws on and situates existing technical approaches (e.g., RLHF, CIRL, incomplete contracting, game‑theoretic methods) but does not present new empirical data or algorithmic experiments.
• Provides diagnostic examples (e.g., predictive policing) to illustrate interactions among axes and to motivate the scaling hypothesis.

Implications for AI Economics

• Research agenda:
  • Model multi‑principal delegation problems formally (multiple stakeholders with heterogeneous utilities) rather than single‑principal setups.
  • Extend contract theory and mechanism design to AI agents: design incentives, verification protocols, and enforceable contracts that account for informational asymmetries and pluralistic principals.
  • Incorporate externalities, public goods, and distributional impacts of misalignment into welfare analyses.
  • Study dynamics under the scaling hypothesis: how increases in model capability and scope change market equilibria, investment in safety, and systemic risk.
• Market and firm incentives:
  • Aligning commercial incentives (shareholder returns, product market competition) with broader stakeholder welfare requires institutional fixes (regulation, standards, liability rules), not only technical fixes.
  • Competitive pressures can create a “race” that under‑invests in governance; economists should model these strategic interactions and potential market failures.
• Information and verification markets:
  • There is economic value in verification, auditing, and monitoring services (third‑party audits, red teaming, certification); market design and regulation can promote supply of credible information.
  • Information provision (transparency, provenance, model cards) affects contracting costs and the feasible set of incentive schemes.
• Policy and regulation:
  • Design policy instruments oriented to governance: mandatory disclosure, auditability standards, stakeholder representation in procurement and public contracts, liability rules to internalize harms.
  • Use incomplete contracting insights: require minimum verification rights, contingency clauses, and remediation procedures when proxies fail.
• Distributional and welfare trade‑offs:
  • “Aligned enough” is inherently political and involves trade‑offs (efficiency vs equity, short‑run performance vs long‑term safety). Economists should quantify these trade‑offs and model how institutional rules allocate alignment costs across agents and stakeholders.
• Institutional innovations:
  • Develop mechanisms for participatory decision‑making and grievance redress that change who counts as a principal in practice (e.g., community oversight, stakeholder co‑design).
  • Consider insurance, indemnity, and funding instruments to manage residual alignment risk and externalities.
• Practical modelling implications:
  • When evaluating interventions (technical or policy), include informational frictions and multiple principals in counterfactuals.
  • Cost–benefit and welfare assessments should include governance costs of monitoring, contestation, and ongoing negotiation.

Taken together, the paper suggests AI economists should shift more attention from treating alignment as a property of models to analysing the institutional and market mechanisms that determine whose values get encoded, how objective proxies are chosen and monitored, and how information and power imbalances are corrected.