0 cumulative citations
View corpus contextDiagnostic multiple-choice probes expose hidden team misalignments but detection remains far from solved: top models score ~50% F1 on a new IoA-Suite benchmark, while a trained 8B prober uncovers nearly three unvoiced disagreements per real meeting and produces modest gains in multi-agent task performance.
Illusion of Alignment: Detecting Hidden Disagreement in Collaborative Dialogue
The paper introduces IoA-Suite and an MCQ-probing method to detect hidden disagreements in collaborative dialogue, finding that state-of-the-art models perform poorly (~50% F1) but a trained IoA-Prober-8B modestly improves detection and surfaces previously unvoiced disagreements in a small real-user study.
Citation observations
Cumulative provider counts captured on specific dates; providers are never combined.
Collaborative dialogue can end with apparent agreement while participants still differ on goals, assumptions, or execution plans, creating an \textbf{illusion of alignment (IoA)}. A real-user study across 18 meetings confirms that IoA arises routinely in human collaboration. Yet IoA poses a paradox: if participants were aware of such disagreements, they would already be explicit; if not, they cannot articulate them when asked, leaving IoA invisible to both participants and observers. In this work, we make IoA detectable by generating diagnostic multiple-choice questions whose divergent answers across participants provide direct behavioral evidence of hidden disagreement. We construct \textbf{IoA-Suite}, a dataset and evaluation protocol for detecting hidden disagreement, spanning five task types and six domains. We find that even the best model attains only 49.5\% F1, with the bottleneck traced to private context that the dialogue does not surface. We then train \textbf{IoA-Prober-8B} based on IoA-Suite, reaching 51.8\% F1 on IoA-Suite. Across the aforementioned 18 real meetings, it surfaces 2.89 hidden disagreements per meeting that participants confirm they had not voiced, transferring to live human dialogue. Further, in multi-agent collaboration, pairing IoA-Prober-8B with LLM agents improves downstream task performance on BigCodeBench-Hard and HiddenBench.
Summary
Main Finding
The paper defines and operationalizes the "Illusion of Alignment" (IoA): collaborative dialogues that end in apparent agreement while participants privately hold divergent goals, assumptions, or plans. IoA can be detected reliably by eliciting downstream behavior — specifically, generating diagnostic multiple-choice questions (MCQs) about the dialogue and checking for divergent answers across participants. The authors build IoA-Suite (a synthetic dataset and evaluation protocol), show that current LLMs struggle (best baseline F1 ≈ 49.5%), trace the core bottleneck to private participant context, and improve performance with a trained probe (IoA‑Prober‑8B, F1 ≈ 51.8%). In live meetings, the prober surfaced on average 2.89 previously unvoiced hidden disagreements per meeting; pairing the prober with LLM agents also improved multi-agent downstream task performance.
Key Points
- Illusion of Alignment (IoA): surface agreement in transcript but latent mismatches in post-dialogue construals (commitments) due to private agendas/assumptions.
- Detection reformulation: ask an external model to generate diagnostic MCQs from the transcript + public profiles; have each participant answer independently. Divergent answers are objective behavioral evidence of hidden disagreement.
- Validity filter: divergent MCQs are further filtered for meaningfulness, implicitness (not already debated), and non-redundancy; then attributed to planted latent disagreements where possible.
- Dataset & evaluation: IoA‑Suite (synthesized) with planted latent disagreements, 1,200 train / 120 val / 300 test dialogues spanning 5 task types and 6 domains; generators and judges are high-capacity LMs, with cross-checks by other models and human annotators.
- Empirical results:
- MCQ probing substantially outperforms direct prompting across nine frontier models (average F1 gain ≈ +13.5 points).
- Best model under MCQ probing (GPT‑5.4) reaches 49.5% F1 on IoA‑Suite — indicating IoA detection is far from solved.
- Revealing private context (participants' private agendas/assumptions and/or turn-level inner thoughts) to the detector yields large gains (e.g., GPT‑5.4: F1 rises from 49.5 → 82.0 when given private static context).
- IoA‑Prober‑8B trained via RL on IoA‑Suite reaches 51.8% F1 and transfers: in an 18‑meeting user study it surfaced 2.89 validated hidden disagreements per meeting.
- In multi-agent collaboration, using IoA‑Prober‑8B alongside LLM agents improved downstream task performance on benchmarks (BigCodeBench‑Hard, HiddenBench).
- Bottleneck diagnosis: the main limitation is the detector’s lack of access to each participant’s private context (agendas and tacit assumptions), which are precisely what conversations fail to surface.
Data & Methods
- Formalization:
- Each participant pi has public profile ri, private agenda gi, tacit assumptions ai. Transcript u(τ) is observable; post-dialogue construal ci(ι) = C(ι; ri, gi, ai, u(τ)). IoA is set M(τ) of issues ι where ci differ.
- MCQ probing pipeline:
- IoA Prober generates candidate MCQs Q from u(τ) and {ri}.
- Each participant answers Q from their own perspective; divergent answers identify candidate disagreements.
- Validity filter V (meaningful, implicit, non-redundant) removes spurious divergences and attempts to attribute remaining questions to ground-truth elements of M(τ).
- Precision/recall/F1 computed over attributed, valid MCQs.
- Dataset construction (IoA‑Suite):
- Two-stage synthesis with planted M(τ): (1) generate participant contexts + planted disagreements; (2) generate dialogue under constraint that planted disagreements remain latent (no explicit debate).
- Seeds grounded in real collaborative scenarios; content x instantiated via scholarly publications to anchor substantive discussion.
- Coverage: five task types (planning, design review, troubleshooting, coordination, brainstorming) × six domains (ML research, software engineering, cross-disciplinary, medicine, business, law).
- Split sizes: 1,200 train / 120 val / 300 test. Quality controlled via multi-model re-scoring and human annotation.
- Evaluation:
- Baselines: direct prompting vs MCQ probing across closed- and open-source LLMs (nine models).
- Simulated participants: role-playing simulators (using strong LMs) supply per-participant answers for automated evaluation; real-user validation used in live studies.
- Model improvements:
- Trained IoA‑Prober‑8B using an RL-style recipe on IoA‑Suite to optimize asking informative, concise MCQs (improves F1 to 51.8%).
Implications for AI Economics
- Organizational coordination and execution risk:
- IoA is a systematic source of costly execution failures (failed handoffs, wasted effort). Tools that surface latent disagreement can reduce coordination costs and increase effective productivity in teams, implying potential gains in firm value from deployment.
- Market for alignment-probing tools:
- There is commercial value in products that proactively detect hidden misalignment in meetings, design reviews, or code handoffs. Demand will be highest where asymmetric private contexts and high coordination costs exist (software engineering, healthcare, cross-disciplinary R&D).
- Private information and information asymmetry:
- The core technical bottleneck is private context (agendas/assumptions). Economically, this maps to information asymmetry problems: detection is costly because the most valuable signal is private. Firms may invest in structured public communication (templates, checklists, mandatory probes) to reduce latent disagreement, or in permissioned sharing of private intent data with mediating tools. Both strategies have trade-offs (privacy, cognitive load).
- Incentives and strategic behavior:
- Knowledge that meetings may be probed could change behavior: participants might conceal or hedge differently, or strategically craft public profiles to influence probe outputs. Designers must consider incentive-compatible probe designs and potential gaming.
- Cost–benefit and externalities:
- Probing imposes participant cost (time/attention) and possible privacy/exposure externalities. Economic adoption depends on whether expected reductions in coordination failure exceed these costs. Benchmarks like those in this paper help quantify benefits (e.g., ~3 hidden disagreements surfaced per meeting).
- Labor and task allocation:
- Better IoA detection could shift comparative advantage: more junior staff or remote collaborators may gain from automated prober assistance, potentially altering division of labor and wage dynamics in coordination-heavy roles.
- Multi-agent systems and automation economics:
- Improved IoA detection enhances multi-agent LLM collaboration, raising productivity of automated teams and increasing returns to deploying such systems for complex, interdependent tasks. This can accelerate automation adoption in coordination-intensive domains, affecting labor demand.
- Research and policy directions relevant to AI economics:
- Quantify the monetary value of surfaced disagreements in different domains (e.g., avoided rework in software vs. medical errors).
- Study optimal probing frequency and question budget under diminishing returns / attention costs.
- Explore mechanisms to elicit private-context signals without violating privacy or inducing strategic misreporting (mechanism design, contract incentives).
- Consider regulatory or governance implications where probing exposes tacit knowledge or sensitive assumptions (data protection, workplace surveillance).
Overall, the paper provides a practical, verifiable method to detect a costly, previously hidden coordination failure mode and demonstrates both technical limitations of current models and realistic gains from targeted probes. For AI economics, this highlights a clear area where AI tools can reduce coordination frictions, but also where information asymmetries and incentive problems will govern adoption and value.
Assessment
Paper Typedescriptive
Evidence Strengthmedium — The paper provides multiple empirical evaluations: a synthetic dataset (IoA-Suite) with held-out test splits, cross-family LLM benchmarking, a trained 8B probe (IoA-Prober-8B), and a real-user study (18 meetings, 43 participants) plus downstream multi-agent task tests. These together support the core claims that MCQ probing can elicit hidden disagreement and that a trained prober improves detection. However, the primary dataset is synthetic and generated/judged by large LLMs (risking self-enhancement and distributional bias), the real-user validation sample is small, and external generalizability beyond the seeded domains and English scholarly-seed scenarios is limited, so causal or large-scale productivity claims are not strongly established.
Methods Rigormedium — The authors use a careful, reproducible pipeline (seeded scenario generation, stage-wise automated judging, held-out splits, cross-family re-scoring, human annotator checks) and a clear, verifiable evaluation protocol (MCQ probing with an explicit validity filter and simulator-driven participant answers). Weaknesses: the dataset is synthesized with GPT-5.4 (generator and judge), creating potential self-enhancement and model-coupling bias despite cross-checks; participant behavior is simulated for much evaluation (relying on LLM simulators such as Gemini-3-Flash); the real-user study is small (18 meetings) and may not capture diverse real-world organisational contexts; downstream improvements are shown but effect sizes and robustness across settings are modest.
SampleIoA-Suite: synthesized dataset with 1,200 training, 120 validation, and 300 test dialogues spanning five collaborative task types (planning, design review, troubleshooting, coordination, brainstorming) and six domains (ML research, software engineering, cross-disciplinary work, medicine, business, law). Dialogues are generated via LLM pipeline (GPT-5.4 as generator and judge); participant-construal simulation uses Gemini-3-Flash. Benchmarking covers nine frontier models (closed- and open-source families). Real-user study: 18 meetings with 43 participants, where IoA-Prober-8B surfaced on average 2.89 previously unvoiced disagreements per meeting. Downstream evaluation: pairing the prober with LLM agents improves performance on multi-agent benchmarks (BigCodeBench-Hard, HiddenBench).
Themeshuman_ai_collab productivity
GeneralizabilityDataset is synthetically generated (GPT-5.4) and seeded from scholarly publications, which may not match naturalistic workplace conversations., Evaluation relies heavily on LLM-based simulators and judges, creating potential self-enhancement and model-family bias despite cross-checks., Real-user validation is small (18 meetings, 43 participants) and may not represent diverse industries, team sizes, languages, or cultures., The approach assumes participants will reliably answer MCQs; applicability when participants are unwilling or inattentive is uncertain., IoA definition emphasizes definitional/lexical divergences (e.g., 'Torch' = Lua vs PyTorch); other forms of misalignment may be underrepresented.
Claims (9)
| Claim | Direction | Outcome | Confidence & Evidence | Details |
|---|---|---|---|---|
| Illusion of alignment arises routinely in human collaboration: across 18 real meetings, participants confirmed an average of 2.89 hidden disagreements per meeting that they had not articulated during the original discussions. Team Performance | negative | Number of hidden, unvoiced disagreements per collaborative meeting |
Reading fidelity
high
Study strength
medium
|
n=43
2.89 hidden disagreements per meeting
|
| Multiple-choice question probing improves hidden-disagreement detection compared with directly prompting models to list hidden disagreements. Other | positive | F1 score for illusion-of-alignment detection |
Reading fidelity
high
Study strength
medium
|
n=9
F1 improvement of 5.7% to 17.9%, with a mean gain of 13.5%
|
| MCQ probing increases recall for every evaluated model, rather than improving F1 only by generating more questions. Other | positive | Recall of hidden-disagreement detection |
Reading fidelity
high
Study strength
medium
|
n=9
mean recall increase of 11.2%
|
| The strongest evaluated model achieves only 49.5% F1 on IoA-Suite, and no evaluated model exceeds 50.0% F1. Other | null_result | F1 score for hidden-disagreement detection |
Reading fidelity
high
Study strength
medium
|
n=9
49.5% F1
|
| Providing either participants’ inner thoughts or their static private context substantially improves IoA detection performance, increasing F1 by more than 20 percentage points for all three evaluated models. Other | positive | F1 score for hidden-disagreement detection under different context-access conditions |
Reading fidelity
high
Study strength
medium
|
n=3
more than 20.0 percentage points absolute F1 improvement for all three models
|
| For GPT-5.4, adding participants’ inner thoughts increases F1 from 49.5% to 74.1%, while adding static private context increases F1 to 82.0%; providing both increases F1 to 84.4%. Other | positive | GPT-5.4 F1 score for IoA detection |
Reading fidelity
high
Study strength
medium
|
49.5% to 84.4% F1 when both private-context sources are provided
|
| IoA-Prober-8B reaches 51.8% F1 on IoA-Suite. Other | positive | F1 score for hidden-disagreement detection |
Reading fidelity
high
Study strength
medium
|
n=300
51.8% F1
|
| Pairing IoA-Prober-8B with LLM agents improves downstream task performance in multi-agent collaboration on BigCodeBench-Hard and HiddenBench. Team Performance | positive | Downstream multi-agent task performance |
Reading fidelity
high
Study strength
low
|
not reported
|
| IoA-Suite contains 1,200 training dialogues, 120 validation dialogues, and 300 test dialogues spanning five collaborative task types and six domains. Other | positive | Availability and coverage of a hidden-disagreement detection benchmark |
Reading fidelity
high
Study strength
medium
|
n=1620
1,620 total dialogues
|