LLM agents are built to answer, not to reason with people — a gap that undermines human-AI teams in high-stakes decisions; adopting a Collaborative Causal Sensemaking agenda would reorient training and evaluation to produce AI teammates that co-reason, surface uncertainty, and improve trust and complementarity.

Main Finding

The paper argues that the persistent "complementarity gap" — human–AI teams in high‑stakes, judgemental tasks often failing to outperform the best individual — stems from a mismatch between current LLM agents (trained as answer engines) and the collaborative, causal sensemaking process experts use. It proposes Collaborative Causal Sensemaking (CCS) as a research agenda: train agents to co-construct, critique, and revise shared causal and goal models with human partners, and to sustain sensemaking loops (discrepancy → hypothesis → test → joint update → action) over time so teams achieve true complementarity.

Key Points

• Complementarity gap: In domains with delayed, uncertain, value‑laden outcomes, human–AI teams often underperform the best individual because agents do not participate in the latent, co-evolving reasoning processes experts use.
• CCS definition: Joint construction, critique, and revision of shared causal and goal models between human and AI, with agents learning from joint decisions and persisting models across interactions.
• Conceptual formalization:
  • Cast interaction as a cooperative partially observable decision process where humans and agents have latent world models (W_H, W_A) and goal structures (G_H, G_A) that evolve.
  • Propose objectives that balance task reward with penalties for epistemic (d_W) and teleological (d_G) misalignment: J_CCS ≈ E[sum γ^t r_t] − λ_W E[d_W] − λ_G E[d_G].
  • Emphasize local, task‑specific alignment (subgraphs / factorized models) rather than full theory‑of‑mind distributions.
• Five research agendas:
• Formalise co‑evolving world and goal models (extend Dec‑POMDP/CIRL/Active Inference to latent W and G dynamics and productive disagreement).
• Measure shared understanding without direct access to latent models using behavioural/artefact proxies (causal graphs, counterfactual simulatability, verification cost, complementarity metrics, sycophancy stress tests).
• Create training ecologies (constructivist playworlds / discrepancy engines) that induce epistemic friction, produce sensemaking trajectories, and yield supervision for collaborative moves.
• Build architectures for persistent, structured models (neuro‑symbolic causal graphs, episodic sensemaking memory, explicit goal representations, theory‑of‑mind modules) so agents can persist and retrieve shared models across sessions.
• Develop principled policies for when to disagree, defer, or intervene using value‑of‑information, mixed‑initiative protocols, and teleological constraints to avoid sycophancy or manipulative behaviour.
• Safety and governance: CCS requires provenance, auditability, and constraints on endogenous goal formation to avoid manipulation and goal drift.
• Empirical evaluation emphasis: shift from static one‑shot benchmarks to team‑level, longitudinal measures (verification cost, calibrated trust, robustness under shift, whether team outperforms best individual).

Data & Methods

• Nature of the work: conceptual / research agenda paper (no primary empirical dataset).
• Methods and tools advocated:
  • MAS formalisms: extend Dec‑POMDP, CIRL, and Active Inference to model co‑evolving latent world/goal state.
  • Behavioural proxies and metrics: Structural Hamming Distance or graph edit distance for externalised causal graphs; counterfactual simulatability tasks; verification cost and complementarity metrics.
  • Training regimes:
    • Constructivist collaborative playworlds that induce epistemic friction via partial, biased views for agents/humans.
    • Interactive fine‑tuning that logs not only actions but why experts corrected or revised hypotheses.
    • Naturalistic logging (with governance) to capture goal evolution.
  • Architectures:
    • Neuro‑symbolic "causal twins" for editable shared models.
    • Episodic sensemaking memory (triplet records), lightweight reasoners and theory‑of‑mind modules.
    • Teleological representations such as reward machines to encode goal logic.
  • Evaluation protocols:
    • Stress tests for sycophancy and justified dissent.
    • Manipulation experiments that perturb shared models to test causal impact on performance and trust.
    • Shadow deployments and longitudinal logging to test transfer.
• Empirical work recommended but not reported: simulations in playworlds, human‑in‑the‑loop experiments, and deployment logging to validate CCS gains.

Implications for AI Economics

• Economics of complementarity: CCS reframes the value of AI not as raw predictive accuracy but as the ability to generate complementarities via sustained joint causal understanding. Measuring returns to AI investment should incorporate reductions in verification costs, increases in team performance relative to the best individual, and gains from better calibrated trust.
• Productivity and labor allocation:
  • Potential to shift expert labor from low‑value verification tasks to higher‑value judgment and oversight if CCS reduces the need for step‑by‑step checking.
  • Could change the effective skill premium: experts who are better at collaborating with CCS‑enabled agents (e.g., who externalise models, ask useful probes) may capture more productivity gains.
• Incentives and adoption:
  • Firms bear costs of building richer playworlds, structured memories, and audit trails; private returns depend on measurable team gains under realistic stakes. Economic research should estimate investment thresholds where CCS yields positive ROI.
  • Market differentiation: vendors offering CCS capabilities could command premiums in high‑stakes sectors (healthcare, law, education, finance) where verification costs are high and causal reasoning matters.
• Evaluation and procurement:
  • Procurement and regulation should move beyond test‑set accuracy to team‑level metrics (verification time, calibrated trust, complementarity) and longitudinal audits of goal drift.
  • Payment and contracting models could incorporate incentives for maintaining epistemic provenance and for achieving persistent alignment.
• Safety, externalities, and regulatory implications:
  • CCS introduces new failure modes (goal drift, manipulative disagreement); regulators need to consider requirements for provenance, audit logs, and limits on autonomous goal formation.
  • Externalities from training ecologies: large synthetic playworlds will be data‑intensive; coordination problems may arise over shared benchmarks, governance, and access.
• Research directions for economists:
  • Quantify economic gains from reduced verification burden and better calibrated trust via field experiments or quasi‑experimental deployments.
  • Study market structure impacts: does CCS raise entry costs (complex architectures, long training ecologies) and thus favor incumbents, or does it create new niches for specialized collaborative agents?
  • Design incentive mechanisms (contracts, pricing) that align firms' deployment incentives with long‑term epistemic safety and complementarity.
  • Model labor market effects: substitution vs. augmentation for experts, changes in task allocation, and potential changes to training and credentialing if CCS agents externalise and persist causal models.
• Measurement & policy: economists can contribute robust metrics and causal inference tools (e.g., difference‑in‑differences, instrumental variables) to estimate CCS effects on productivity, welfare, and distributional outcomes in real deployments.

Overall, CCS proposes shifting both technical focus and economic evaluation of AI from producing oracle answers to cultivating persistent, auditable teammates that co‑reason with humans — a shift with significant implications for how we measure, value, regulate, and deploy AI in judgment‑intensive sectors.