Agentic AI makes alignment an ongoing organizational cost: systems that generate open-ended actions, novel outputs, and evolving goals turn coordination into a continuous process of projection, monitoring and renegotiation rather than a one-off specification task; firms that can cheaply sustain persistent alignment gain a competitive edge.

Main Finding

Agentic AI — systems that plan, act, and revise goals over extended horizons — introduces structural uncertainty across action trajectories, generated representations, and governing regimes. Extending Team Situation Awareness (Team SA) to human–agentic AI teaming shows that while Team SA remains a useful integrative anchor, its stabilizing assumptions are strained. Under open-ended agency, processes that normally sustain alignment (relational interaction, iterative learning, coordination/control) can fail or reverse: legitimacy can rest on epistemic fragility, iterative updating can amplify divergence, and apparent shared awareness can coexist with substantive loss of oversight. The central problem becomes sustaining alignment over time as agents continuously generate, revise, enact, and govern futures.

Key Points

• Definition of agentic AI and the three forms of structural uncertainty:
  • Trajectory uncertainty — open-ended multi-step action selection and replanning that unfolds during execution.
  • Epistemic uncertainty — generative outputs (rationales, facts, artifacts) whose grounding and evidentiary status are contestable.
  • Regime uncertainty — non-stationary governing configurations (objectives, constraints, capabilities) that can change across episodes.
• Reciprocal modeling: agentic AI actively models human collaborators (preferences, likely responses) creating recursive dynamics that condition trajectories, outputs, and regime evolution.
• Why classic assumptions break:
  • Traditional Team SA assumes boundedness and relative stationarity so iterative updates converge; agentic AI violates those assumptions.
  • Social norms and institutional accountability that constrain human teammates are weaker or absent for agentic systems, producing different coordination dynamics.
• Team SA as integrative anchor:
  • Maps existing HAT theory families (evaluative attitudes, relational interaction, cognitive learning, explanatory guidance, collective coordination, operational control) onto SA levels (perception, comprehension, projection).
  • Reveals where each theory helps and where it leaves gaps under open-ended agency.
• Continuity vs tension:
  • Continuity: Team SA’s leveled structure (perception/comprehension/projection) remains useful to diagnose alignment problems and to organize interventions.
  • Tension: The dynamic assumptions (that communication, feedback, and role differentiation will stabilize teaming) are fragile when agents revise goals, generate contestable outputs, or change policy over time.
• Pathological outcomes identified:
  • Epistemic fragility legitimized: social legitimacy (trust, reliance) may be built on outputs that lack robust grounding.
  • Divergence amplification: iterative human–agent adjustments can feed mutual miscalibration rather than correction.
  • Oversight erosion: shared situational awareness can mask decline in meaningful human control as the agent’s regime drifts.
• Proposed research agenda highlights:
  • Reconceiving measurement (e.g., projection congruence across heterogeneous cognitive systems).
  • Studying dynamics (longitudinal experiments, field studies, simulations) to observe drift and misalignment over episodes.
  • Designing mechanisms for continuous alignment (transparent grounding, monitoring/auditing, interruptibility, governance protocols).
  • Investigating institutional and market responses (contracting, liability, standards, incentives).

Data & Methods

• Primary method: conceptual and theoretical analysis.
  • Comprehensive literature synthesis across human–AI teaming, team cognition (Team SA), human–automation interaction, organizational coordination, and explainable AI.
  • Analytical extension of Team SA to account for open-ended agency and three-dimensional structural uncertainty.
  • Mapping of extant theory families onto Team SA levels to identify continuities and boundary conditions.
• Empirical agenda (proposed, not executed in this paper):
  • Experimental lab studies and longitudinal field studies to observe alignment dynamics across episodes and model updates.
  • Behavioral experiments to measure projection congruence, trust dynamics, and iterative updating effects.
  • Computational/agent-based simulations to explore regime drift, recursive modeling effects, and coordination failure modes.
  • Design-probe evaluations of interface, transparency, and governance interventions (e.g., audit trails, constraint specification, interruption affordances).

Implications for AI Economics

• Valuation and product-market fit
  • Market value of agentic AI depends not only on immediate performance but on maintainable alignment over time; firms must price in the cost of continuous monitoring, governance, and potential regime drift.
  • Products that favor adaptability (open-ended agency) may command premium only if credible alignment mechanisms exist; otherwise buyers discount due to oversight and liability risk.
• Contracting and incentives
  • Standard principal–agent contracts assume fixed agent objectives; agentic AI undermines this, requiring dynamic contracts and incentive schemes robust to non-stationary agent behavior (performance metrics, re-negotiation clauses, verifiable audit logs).
  • Moral hazard and information asymmetries increase: providers may update agents in ways that benefit them but deviate from client interests; contracting must incorporate update governance and rollback rights.
• Organizational design and labor
  • Firms will need new roles and capabilities (continuous AI auditors, governance officers, situational monitors) and may restructure teams around persistent oversight rather than episodic supervision.
  • Labor demand shifts: higher demand for monitoring, interpretability, and governance skills; potential deskilling of routine tasks but increased coordination costs where oversight is costly.
• Competition and market structure
  • Competitive advantage may hinge on ability to maintain reliable alignment at scale (trustworthiness, explainability, certifiable governance), potentially favoring incumbents who can absorb monitoring costs.
  • Markets for complementary services (auditing, insurance, alignment-as-a-service, verification tools) will expand.
• Risk, insurance, and externalities
  • Regime uncertainty elevates tail risks (unanticipated objective drift); insurers, regulators, and firms will need new risk models, higher premiums, and capital buffers.
  • Systemic externalities: misaligned agentic AI in widely used infrastructure could propagate errors across firms and sectors, increasing systemic risk.
• Measurement and productivity accounting
  • Productivity gains are harder to attribute and may be less stable when agentic behavior evolves; macro and firm-level productivity statistics must account for temporal alignment losses and monitoring costs.
• Policy and regulation
  • Regulatory frameworks should mandate provenance, auditability, and update-disclosure requirements to reduce epistemic and regime uncertainty.
  • Liability regimes must address evolving-agent behaviors; policymakers may need to require fail-safe control points and human override guarantees for critical domains.
• Empirical economics research opportunities
  • Natural experiments around model updates, platform policy changes, or regulation rollouts can reveal causal effects of regime shifts on firm outcomes.
  • Structural and behavioral models can incorporate non-stationary agent policies to study incentives, innovation rates, and welfare impacts.
  • Market design research can evaluate trading platforms for agentic AI services, contracting primitives, and insurance markets for alignment risk.

Overall, the paper signals that economic models and organizational strategies must internalize the temporal and structural uncertainties of agentic AI—shifting focus from one-off alignment to sustained governance, monitoring, and incentive design over time.