Prompting LLM agents to be less or more 'agreeable' changes how they communicate but only matters for unstructured tasks: low agreeableness substantially worsens outcomes in research-style collaboration and bargaining, while coding milestone completion is largely unchanged.

When Does Personality Composition Matter for Multi-Agent LLM Teams?

Aryan Keluskar, Amrita Bhattacharjee, Huan Liu · June 25, 2026

arxiv quasi_experimental medium evidence 7/10 relevance Full text usable extracted full text Source PDF

Manipulating LLM 'personality' strongly shifts agent communication but only degrades objective performance on open-ended collaboration and bargaining tasks, while structured coding milestone completion remains largely unaffected.

Personality prompting shapes how large language models communicate, yet whether these behavioral shifts affect objective task outcomes remains under-explored. Prior work shows that agents prompted with low agreeableness produce adversarial language, while those prompted with high agreeableness become cooperative, but the relationship between communication style and task performance has not been systematically examined across multiple domains. In this work, we investigate whether personality composition matters for multi-agent team performance by manipulating personality traits across frontier LLMs on three task domains: structured coding, open-ended research collaboration, and competitive bargaining. We find that personality effects depend critically on task structure. In coding tasks, low agreeableness leads to large communication shifts that have little effect on milestone completion. In open-ended collaboration and bargaining, the same manipulation substantially degrades performance. We discuss implications for multi-agent system design and the limits of personality manipulation.

Summary

Main Finding

Personality prompting (especially low agreeableness using Goldberg bipolar adjectives) reliably shifts multi-agent LLM communication toward exploration and disagreement (φ rising ≈0.44→0.93 across models), but whether that process-level change affects objective outcomes depends critically on task structure: structured artifact tasks (coding) largely buffer outcome degradation, while unstructured/cooperative (research) and competitive (bargaining) tasks suffer large performance losses (research milestones ↓ up to 66%; bargaining agreement rates collapse to ≈0%).

Key Points

Protocol & manipulation
- Personality shaped via Goldberg bipolar adjective pairs combined with 9-level qualifiers; primary conditions = low-A (level 2) and high-A (level 8).
- Agreeableness (low-A) produced the largest, consistent communication effect across models; conscientiousness and openness did not replicate this effect.
Communication effect
- Measured φ = fraction of communicative acts devoted to exploration (questions + disagreements + suggestions) vs. convergence (acknowledgments).
- Low-A drove φ from ~0.44 (baseline for some models) to ~0.93 for Claude Sonnet 4.5, GPT‑4o, Grok‑3 (DeepSeek rose via suggestions rather than disagreements).
- Disagreement rates rd ≈ 0.45–0.51 under low-A (for several models).
Outcome heterogeneity by domain
- Coding (cooperative, high artifact structure): large communication shifts but milestone completion largely unchanged for 3/4 models (weak φ↔milestone correlation r≈−0.19; planning quality degraded but artifact constraints buffered execution).
- Research (cooperative, low structure): low-A substantially reduced judged milestones (e.g., GPT‑4o 10.5→3.5, −66%; other models also large drops).
- Bargaining (competitive, low structure): low-A collapsed agreement rates to near 0% across evaluated models; high-A tended to increase agreement rates.
Model heterogeneity and caveats
- Claude showed more robustness in some research tasks (model-specific differences).
- Negative-valence adjective prompts may confound pure trait manipulation (safety/obedience effects); the authors flag this as a methodological limitation.
- Low-A agents in bargaining still made and revised offers but almost never accepted—indicating behavioral change not merely refusal to act.
Process vs. outcome dissociation
- The study documents a clear behavioral–outcome dissociation: strong, consistent shifts in communication do not always propagate to task success; whether they do is mediated by the structure and constraints of the output medium.

Data & Methods

Multi-agent benchmarks and tasks
- Coding: MultiAgentBench 3-agent software tasks (5 tasks). Cooperative, high-structure; deliverable = code file subject to syntax/semantic constraints.
- Research: MultiAgentBench 5-agent open-ended research idea generation (15 tasks; n≈30 per condition).
- Bargaining: Two-agent buyer–seller negotiation benchmark (50 tasks; n≈100 per condition).
Models evaluated
- Claude Sonnet 4.5, GPT‑4o, Grok‑3, DeepSeek V3.1 (selection of frontier LLMs).
Communication labelling
- Messages split into segments; each segment labeled (multi-label) for question, disagreement, suggestion, acknowledgment.
- Primary labeling via GPT‑4o‑mini (deterministic); validation with independent judge model Kimi K2.6 showed high agreement.
- Communication state φ = (cQ + cD + cS) / (cQ + cD + cS + cA).
Outcomes & metrics
- Coding/research: milestone completion counts and LLM-as-judge planning quality rubric.
- Bargaining: agreement rate and offer/accept behavior logs.
Statistical evidence
- Communication shifts under low-A: p < 0.003, large Cohen’s d (e.g., d from −1.17 to −12.39 across models for φ changes).
- Research milestone drops and bargaining collapses were statistically and practically large (e.g., GPT‑4o research milestones d≈1.41, bargaining agreements → ~0%).

Implications for AI Economics

System design and team composition
- Personality prompting is a lever that meaningfully changes agent discourse; designers should treat personality as an architecture-level parameter when engineering multi-agent workflows.
- For artifact-heavy tasks (coding, formal verification, structured pipelines), injecting dissenting or adversarial personalities may be low-risk: artifact constraints can “buffer” process-level dysfunction, limiting downstream economic harm.
- For open-ended collaboration and automated negotiations, personality composition matters economically: low agreeableness can sharply reduce productivity (fewer usable outputs) or destroy transaction completion (failed agreements), directly affecting value capture and user outcomes.
Productization & go-to-market risk
- Negotiation bots or automated sales/contract agents are vulnerable: a mis-specified personality prompt can eliminate deals. Firms should validate personality settings under realistic task structure and opposing-agent scenarios before deployment.
- Coding assistants that orchestrate sub-agents may be robust to adversarial-style prompts, suggesting safer pathways for deploying heterogeneous-agent teams in developer tools—but care is still needed because planning quality degraded even when milestones held.
Incentive engineering & policy
- When automating bargaining and market interactions, platform designers must consider incentive compatibility and personality-induced market failure modes (e.g., high disagreement rates causing durable deadlock).
- Regulation or standards might be warranted for deployed agents that negotiate on behalf of users (consumer protection against agents tuned to adversarial personalities).
Economic modeling of multi-agent systems
- The person-environment fit perspective translates to multi-agent LLM teams: the economic value of agent traits is task-dependent. Models of automation substitution, coordination costs, and multi-agent complementarities should include output-structure as a moderating variable.
- Cost–benefit analyses for introducing dissenting agents (to prevent premature convergence) should account for task structure: potential benefit in exploration may be realized in ideation tasks but may be neutral or harmful in competitive settings.
Research and evaluation recommendations
- Benchmarking for multi-agent economic performance should include both process metrics (communication patterns) and outcome metrics (task completion) and explicitly test structured vs. unstructured tasks.
- Developers and economists should examine heterogeneous teams (mixed personalities) and dynamic incentives, and control for prompt valence effects that may trigger non-trait safety behaviors.

Reference: Keluskar, A., Bhattacharjee, A., & Liu, H. (2026). When Does Personality Composition Matter for Multi-Agent LLM Teams? (preprint, under review).

Assessment

Paper Typequasi_experimental Evidence Strengthmedium — The paper uses a controlled manipulation that yields strong internal causal evidence linking personality prompts to communication and downstream outcomes within the simulated setting; however, evidence is limited to a small set of frontier LLMs, particular prompt recipes and task instances, and relies on synthetic multi-agent interactions rather than real-world human-AI teams, reducing external validity and robustness certainty. Methods Rigormedium — The study systematically examines multiple LLMs and three distinct task domains, which supports comparative inference, but details likely missing on sample size per condition, pre-registration, robustness checks, metric objectivity (especially for open-ended tasks), and sensitivity to alternative prompt formulations or model versions. SampleSimulated multi-agent experiments using several modern large language models (frontier LLMs) organized into teams; personality traits (e.g., high vs low agreeableness) induced by prompting; tasks covered three domains—structured coding (measured by milestone completion), open-ended research collaboration (qualitative/productivity metrics), and competitive bargaining (outcome metrics); multiple runs/seeds across conditions. Themeshuman_ai_collab productivity IdentificationExperimental manipulation of agent 'personality' via prompt engineering assigned across multi-agent LLM teams; causal inference comes from randomized/controlled comparisons of performance across personality conditions while holding model, task, and other settings constant. GeneralizabilityResults are limited to the specific LLM architectures and versions tested and may not hold for other models or future updates., Findings depend on the particular prompt-engineering approach used to induce personality and may not generalize to alternative prompting or fine-tuning methods., Experimental setting uses synthetic multi-agent interactions without human participants, limiting applicability to human-AI or mixed human-agent teams., Task selection covers three domains but does not represent the full range of real-world organizational tasks or industry workflows., Outcome measures for open-ended tasks may rely on subjective or proxy metrics, affecting comparability to real productivity measures.

Claims (5)

Claim	Direction	Outcome	Confidence & Evidence	Details
Personality prompting shapes how large language models communicate. Other	positive	communication style / behavior of LLMs	Reading fidelity high Study strength medium	not reported 0.48
Prior work shows that agents prompted with low agreeableness produce adversarial language, while those prompted with high agreeableness become cooperative. Other	mixed	communication tone (adversarial vs cooperative)	Reading fidelity high Study strength medium	not reported 0.48
Personality effects depend critically on task structure. Team Performance	mixed	variation in team performance by task structure	Reading fidelity high Study strength medium	not reported 0.48
In coding tasks, low agreeableness leads to large communication shifts that have little effect on milestone completion. Task Completion Time	mixed	milestone completion (task completion success)	Reading fidelity high Study strength medium	not reported 0.48
In open-ended collaboration and bargaining, the same manipulation substantially degrades performance. Team Performance	negative	team performance (quality/success in open-ended collaboration and bargaining)	Reading fidelity high Study strength medium	not reported 0.48