More collaborators can hurt as much as help: simulated human-AI teams often see performance drop when members are added without coordination rules, but adding shared group memory plus approval gates substantially improves outcomes—especially for three-person teams.

Searching for Synergy in Shared Workspace Human-AI Collaboration

Nachiket Kotalwar, Rohini Das, Carolyn Rose · June 16, 2026

arxiv quasi_experimental medium evidence 7/10 relevance Full text usable extracted full text Source PDF

In a simulated shared-workspace environment, adding relevant collaborators can reduce team performance when coordination structure is absent, while scaffolding (shared memory plus human-in-the-loop approval gates) raises mean performance—most strongly in three-person teams—by clarifying responsibilities and routing expertise.

Automated AI agents are increasingly capable, yet many scientific and professional tasks require human judgment and contextual expertise. We study shared-workspace human-AI teams, where AI agents and human collaborators must coordinate responsibilities before submitting a final answer. Using the Collaborative Gym environment with DiscoveryBench tasks, we examine when adding simulated human collaborators improves performance and when process loss turns additional collaborators into coordination overhead. Across 1,482 sessions, adding relevant collaborators can lower performance when teams lack structure to coordinate their contributions. We then evaluate scaffolding that combines shared group memory with simulated human-in-the-loop (HITL) gates, where selected actions require approval from a designated simulated participant. This scaffolding yields higher mean performance, most clearly in three-person teams, with clearer responsibility signals and stronger routing of expertise to team actions. Overall, how human-AI teams coordinate and integrate expertise matters as much as the capability available to them.

Summary

Main Finding

Adding simulated human collaborators with relevant expertise can reduce team performance in unstructured shared-workspace human–AI teams because of coordination (process) loss. Introducing simple coordination scaffolds — a pre-task shared group memory (who knows what, responsibilities, evidence criteria) combined with simulated human-in-the-loop (HITL) approval gates chosen by the team — raised mean performance, most clearly for three-person teams (AI + two distinct human profiles). The scaffold shifts how work is distributed (more messaging and clearer initiative routing) so distributed expertise is actually routed into supported final hypotheses.

Key Points

Counterintuitive baseline result: Default shared-workspace teams did not outperform the single-agent AI baseline; performance fell as more profiled collaborators were added.
- Single-agent Performance ≈ 0.71.
- Default-D (AI + data profile): 0.69; Default-R (AI + researcher): 0.68.
- Default-DR (AI + data + researcher): 0.63. This pattern is consistent with process loss: extra expertise increased interaction but not supported outcomes.
Scaffolded collaboration (shared group memory + team-chosen simulated HITL gates) increased mean Performance across compositions, most notably:
- Scaffolded-D: 0.72, Scaffolded-R: 0.73, Scaffolded-DR: 0.76 (Scaffolded-DR +0.13 vs Default-DR).
Process-level changes under scaffolding:
- Initiative became more evenly distributed. For DR teams, initiative entropy rose from ~0.54 to ~0.85 (normalized).
- Total non-message work changed little (7.6 → 7.9 actions in DR), but human work rose modestly (1.6 → 2.2) and team messages increased substantially (6.1 → 18), indicating more coordination communication rather than more raw work.
- Improvements in trace diagnostics: Workflow Coverage, Hypothesis Support, and Profile Alignment all increased under scaffolding (evidence handoffs and responsibility signals strengthened).
Diagnostic variants:
- Shared group memory alone and preassigned-gates were evaluated; the full scaffold (memory + team-chosen gates) yielded the clearest gains, suggesting complementarity between explicit shared knowledge and routed approvals.
Methodological note: important failures showed up in interaction traces (intermediate steps not carried into submissions), motivating trace-level evaluation in addition to final outcomes.

Data & Methods

Environment and tasks:
- Collaborative Gym extended to run DiscoveryBench archaeology subset (38 tasks). Tasks require variable mapping, temporal criteria, domain interpretation, and evidence construction.
- Total reported sessions: 1,482.
Participants and models:
- Each session had one AI agent (DeepSeek V3.2 with ReAct loop) and 0–2 simulated human collaborators with fixed private guidance profiles:
  - D: data-analysis specialist (variable mapping, computation).
  - R: researcher (query semantics, relation wording, evidence).
- Simulated collaborators reproduce behavioral patterns of real participants per prior work; they see only their own action history while the AI sees the team-level action history.
Collaboration structures:
- Default: original shared-workspace (no extra scaffolds).
- Scaffolded: pre-task shared group memory (expertise/responsibility/evidence map) + team-chosen simulated HITL gates (selected actions require designated approver).
- Diagnostic variants: shared group memory only; preassigned-gates (owners fixed by action type).
Evaluation:
- Outcome: normalized Task Performance (Collaborative Gym).
- Trace/process metrics: Human Work (non-message actions by simulated humans), Total Work, Team Messages, Initiative Entropy (normalized), Workflow Coverage (Cwf), Hypothesis Support (Shyp), Profile Alignment (Aprofile).
- Reference workflow graphs per task were used to judge coverage and support; an LLM judge labeled trace events against these graphs.
Experimental design:
- Results averaged across tasks and three random seeds per task; LLM and interface held constant across variants so differences reflect team composition and collaboration structure.

Implications for AI Economics

Coordination is a first-order cost when adding agents (human or AI). Simply increasing available expertise (adding agents/tools) can reduce productive output unless organizational/interaction design explicitly manages dependencies.
Product and platform design:
- Integrate coordination features (shared group memory / transactive-memory artifacts, role/expertise mapping) and selective approval/routing mechanisms (HITL gates) into multi-agent collaboration tools. These features can convert diverse capabilities into higher-value outcomes with modest extra work.
- Provide APIs and UI primitives for: (a) defining & sharing responsibility maps, (b) configuring approval routes for consequential actions, and (c) surfacing agreed evidence criteria.
Deployment and team design:
- For small multi-agent teams (notably three-party teams), invest in light-weight coordination scaffolds; they can unlock synergy and raise mean productivity.
- Beware of scaling without structure: adding agents without coordination protocols risks higher communication costs and lower-quality outputs (process loss).
Measurement and incentives:
- Evaluate collaborative AI systems with process-level metrics (initiative distribution, evidence handoff, workflow coverage) in addition to final-stage outcomes; these trace metrics reveal coordination failures that outcome-only metrics miss.
- When pricing or assessing multi-agent services, account for coordination overhead and savings from scaffolding — the net value of an extra expert/agent depends on the organizational support for routing and integration.
Policy and markets:
- Markets for collaborative AI tools should not only compete on model capability but also on coordination affordances (shared state, approval routing, role mapping).
- Regulatory/operational controls that require human sign-off for high-consequence actions align with the empirical finding that routed approvals help route expertise and improve supportable decisions.

Actionable recommendation for practitioners: before adding more specialized AI or human agents, implement (or experiment with) a brief pre-task shared expertise map and a small set of approval gates for decisions that change artifacts or conclusions. This modest investment in coordination structure is likely to reduce process loss and increase the realized value of diverse expertise.

Assessment

Paper Typequasi_experimental Evidence Strengthmedium — Large number of controlled sessions (1,482) provides good internal evidence that coordination structure affects team performance in this environment, but the use of simulated collaborators and lab-style tasks limits external validity to real-world human-AI teams and economic outcomes. Methods Rigormedium — The study uses a substantial sample and clearly defined interventions (team size, shared memory, HITL gates) and analyzes responsibility signals and routing of expertise, but key details about randomization, robustness checks, pre-registration, and sensitivity to model/hyperparameter choices are not specified; reliance on simulated human collaborators and a single benchmark suite constrains methodological rigor for real-world inference. Sample1,482 experimental sessions in the Collaborative Gym environment using DiscoveryBench tasks; teams composed of AI agents and simulated human collaborators with variations in team size (including two- and three-person teams), collaborator relevance, and scaffolding interventions (shared group memory and simulated human-in-the-loop approval gates); outcomes are task performance (accuracy/quality), responsibility signals, and routing of expertise. Themeshuman_ai_collab productivity org_design IdentificationControlled between-session experiments in a simulated Collaborative Gym environment: the authors vary team composition (number of simulated collaborators and their relevance) and the presence/absence of coordination scaffolds (shared group memory + simulated human-in-the-loop approval gates) and compare task performance across these conditions, holding task set (DiscoveryBench tasks) constant to infer the effect of team structure on outcomes. GeneralizabilitySimulated collaborators may not reflect real human behavior (communication, error patterns, incentives)., Tasks drawn from DiscoveryBench may not span the full range of scientific/professional work or industry contexts., Controlled environment omits organizational incentives, time pressures, and multi-day workflows present in real teams., Findings may depend on specific agent architectures/hyperparameters used and may not hold for other models., Team sizes tested are small (primarily two- and three-person), limiting extrapolation to larger teams or firms.

Claims (7)

Claim	Direction	Outcome	Confidence & Evidence	Details
We ran 1,482 sessions in our experiments. Other	null_result	number of experimental sessions	Reading fidelity high Study strength high	n=1482 1,482 sessions 0.8
Adding relevant collaborators can lower performance when teams lack structure to coordinate their contributions. Team Performance	negative	team performance (task success / accuracy)	Reading fidelity high Study strength medium	n=1482 0.48
A scaffolding that combines shared group memory with simulated human-in-the-loop (HITL) gates yields higher mean performance. Team Performance	positive	mean team performance	Reading fidelity high Study strength medium	not reported 0.48
The scaffolding's performance improvement is most clear in three-person teams. Team Performance	positive	team performance by team size	Reading fidelity high Study strength medium	not reported 0.48
The scaffolding produces clearer responsibility signals and stronger routing of expertise to team actions. Task Allocation	positive	responsibility signaling and routing of expertise to actions	Reading fidelity medium Study strength low	not reported 0.14
How human-AI teams coordinate and integrate expertise matters as much as the capability available to them. Organizational Efficiency	positive	relative impact of coordination/integration versus capability on team performance	Reading fidelity high Study strength medium	not reported 0.48
We study shared-workspace human-AI teams using the Collaborative Gym environment with DiscoveryBench tasks. Other	null_result	experimental platform and task set used	Reading fidelity high Study strength high	not reported 0.8