In real human-AI interactions, design beats disposition: AI attributes—above all transparency—drive outcomes in hiring and transactional scenarios more than user personality does, although purely simulated agents overstate the influence of human traits.
Imperfectly Cooperative Human-AI Interactions: Comparing the Impacts of Human and AI Attributes in Simulated and User Studies
Comparing 2,000 simulations and 290 human-subject interactions, the study finds that while simulated data emphasize the role of user personality, real human interactions are primarily shaped by AI design—especially transparency—across hiring and transactional scenarios.
AI design characteristics and human personality traits each impact the quality and outcomes of human-AI interactions. However, their relative and joint impacts are underexplored in imperfectly cooperative scenarios, where people and AI only have partially aligned goals and objectives. This study compares a purely simulated dataset comprising 2,000 simulations and a parallel human subjects experiment involving 290 human participants to investigate these effects across two scenario categories: (1) hiring negotiations between human job candidates and AI hiring agents; and (2) human-AI transactions wherein AI agents may conceal information to maximize internal goals. We examine user Extraversion and Agreeableness alongside AI design characteristics, including Adaptability, Expertise, and chain-of-thought Transparency. Our causal discovery analysis extends performance-focused evaluations by integrating scenario-based outcomes, communication analysis, and questionnaire measures. Results reveal divergences between purely simulated and human study datasets, and between scenario types. In simulation experiments, personality traits and AI attributes were comparatively influential. Yet, with actual human subjects, AI attributes -- particularly transparency -- were much more impactful. We discuss how these divergences vary across different interaction contexts, offering crucial insights for the future of human-centered AI agents.
Summary
Main Finding
When human–AI interactions are only partially cooperative, LLM-based simulations and parallel human-subject experiments give divergent accounts of what drives outcomes. In purely simulated dyads (2,000 Sotopia-S4 runs), simulated users’ personality traits—Extraversion and Agreeableness—were major determinants of negotiation- and deception-related outcomes. In the parallel user study (290 U.S. participants), AI design attributes—above all chain-of-thought-style transparency—dominated effects on user experience and scenario outcomes. The paper concludes that LLM simulations can model personality archetypes but understate humans’ sensitivity to observable AI characteristics, so human-in-the-loop validation is essential for AI design and policy in imperfectly cooperative settings.
Key Points
- Two complementary experimental streams:
- Simulation: 2,000 LLM-driven dyadic transcripts using Sotopia-S4 (GPT-4o) across factorial manipulations.
- User study: 290 Prolific participants interacting with identically configured AI agents.
- Scenarios: two thematic groups
- Hiring negotiations (high- and low-stakes) over salary and start date.
- AI-LieDar (partial-truthfulness) scenarios where AI agents face trade-offs between utility and truthfulness (Benefits, Public Image, Emotion).
- Manipulated AI attributes: Transparency (chain-of-thought “thinking tokens” shown vs hidden), Warmth, Expertise, Adaptability, Theory of Mind (ToM). Baseline = all five high; non-baseline ablations set one attribute low at a time.
- Simulated human personality: 2×2 factorial on Extraversion × Agreeableness (high/low), producing four archetypes; held constant in user study (measured only).
- Evaluation metrics: scenario outcomes (Deal Made, Points, Goal Achievement, Truthfulness/Secrets Kept), interaction-process metrics (Transactivity, Verbal Equity, Conversation Reliability), relational measures (Warmth, ToM), plus participant surveys. Causal discovery methods and LLM-based Sotopia-Eval scoring used.
- Main empirical contrast:
- Simulations: personality traits (Extraversion, Agreeableness) and AI attributes had comparable influence; personality often drove socio-emotional and outcome measures.
- Human subjects: AI attributes—especially Transparency—were far more influential on subjective experience and many scenario outcomes than participant personality.
- Important nuance: effects varied by scenario type (negotiation vs deception) and by which AI attribute was ablated.
- Design implication emphasized: simulation-derived findings should be validated with humans because simulations can misestimate effect sizes and omit human sensitivity to exposed AI behavior (e.g., visible “thoughts”).
Data & Methods
- Simulation study
- Platform: Sotopia-S4 multi-agent social simulator, GPT-4o (temperature 0.7).
- Experimental design: 5 scenario setups × 5 AI-intervention settings × 4 personality profiles × 10 replications = 2,000 transcripts.
- AI interventions: baseline (all high) + five single-attribute ablations (one trait low per non-baseline condition); Transparency manipulated at system level via intermittent thinking-token visibility.
- Measures: automated Sotopia-Eval (Warmth, ToM, Adaptability, Transparency), scenario-specific outcomes (deal made, points), and communication metrics (transactivity, verbal equity).
- Analysis: causal discovery / causal analysis comparing drivers of measures across manipulations.
- User study
- Sample: 290 U.S. participants recruited on Prolific (>=100 tasks, 99% approval).
- Design: mirrored a subset of simulation conditions. Personality (Extraversion, Agreeableness) measured via Big Five scales and treated as covariates rather than experimental treatments.
- Procedure: participants randomly assigned one scenario and AI intervention; up to 20-turn dialogue with configured AI; post-interaction survey measuring perceived Transparency, Warmth, ToM, Adaptability, Expertise and scenario-specific subjective outcomes.
- Analysis: compared effects of AI interventions vs measured personality on both objective/annotated outcomes and subjective ratings; contrasted with simulation-derived causal patterns.
- Additional notes:
- Scenarios stopped at agreement or after 20 turns.
- Transparency implemented as explicit “internal reasoning” tokens for high transparency condition.
- Evaluations combined transcript-scoring by an evaluator LLM and participant self-reports.
- Appendices provide prompts, point allocations, power analysis, and debriefing details.
Implications for AI Economics
- Market and bargaining models should explicitly include AI-design attributes as strategic factors.
- AI transparency is a potent signal that changes user behavior and bargaining outcomes in real humans. Economic models of bargaining, matching, or platform-mediated negotiation must account for signaling effects from visible AI reasoning.
- Simulations are useful but insufficient for policy or mechanism design without human validation.
- LLM-driven simulations can scale exploration of personality and agent design, but they tend to misestimate which levers matter for actual users. Relying solely on simulated effect sizes risks mis-specified forecasts of market behavior, adoption, or welfare.
- Information asymmetry and manipulation risks
- In imperfectly cooperative settings (e.g., hiring platforms, customer service, sales), AI agents that can conceal or reveal internal reasoning will materially affect outcomes and distributional welfare. Transparency can improve trust and outcomes but also exposes strategic pathways; regulators should consider standards for when AI must disclose reasoning or incentives.
- Platform design and incentives
- Platforms that host agentic AIs (marketplaces, hiring systems, financial advice platforms) should incorporate AI-attribute controls (transparency, adaptability, ToM) into platform-level rules and user interfaces, and evaluate effects with A/B tests on real users rather than relying on simulations alone.
- Measurement and empirical strategy recommendations for economists
- Use mixed methods: combine scalable LLM simulations for hypothesis generation with field or lab experiments for validation. Treat simulated effect sizes as upper bounds or exploratory signals.
- Incorporate heterogeneous agent preferences: personality and demographic variation matter in simulation but actual user sensitivity to AI attributes implies interaction terms (user type × AI design) are economically meaningful.
- Policy and regulation
- Given transparency’s outsized real-user effects, regulators should prioritize transparency norms and auditing requirements for agentic AIs engaged in negotiations or where incentives conflict with user interests.
- Consider disclosure rules for chain-of-thought or “internal” reasoning when it materially influences user choices.
- Broader welfare questions
- AI design choices can shift bargaining power, alter distribution of surplus, and create new externalities (e.g., manipulation of trust). Cost–benefit analyses of agent deployment must include behavioral responses to AI attributes, not just accuracy metrics.
Suggested next steps for AI economics research: - Replicate across broader populations, higher stakes, and domain-specific markets (e.g., housing, finance, labor platforms). - Study longer-term interactions and learning dynamics (reputation, repeated bargaining). - Quantify welfare impacts (consumer surplus, producer surplus, distributional effects) under alternative transparency and incentive regimes. - Design and test regulation-informed interfaces (e.g., mandated provenance or reasoning summaries) via randomized controlled trials.
Assessment
Paper Typequasi_experimental
Evidence Strengthmedium — The study uses a real human-subjects experiment (n=290) alongside a larger simulated dataset (2,000 simulations) and applies causal-discovery methods, which provides substantive empirical leverage; however, the human sample is modest in size for heterogeneous effects, causal-discovery methods rely on assumptions (e.g., no unmeasured confounding, correct model specification), and external validity is uncertain given artificial scenarios and potentially non-representative participants.
Methods Rigormedium — Strengths include parallel simulation and human experiments, multi-modal outcome measures (behavioral, communication, questionnaires), and explicit attention to personality × AI design interactions; limitations include modest human sample size, likely short-term laboratory/online interactions, reliance on causal-discovery algorithms with strong assumptions, and incomplete information about randomization protocols and robustness checks in the summary.
SampleTwo datasets: (1) 2,000 simulated interactions where agent and human proxies were simulated under varied AI design parameters; (2) a human-subjects experiment with 290 participants who engaged in two scenario categories—hiring negotiations with AI hiring agents and transactional interactions where AI could conceal information—while reporting Extraversion and Agreeableness; outcomes include scenario payoffs, communication content/metrics, and post-task questionnaires.
Themeshuman_ai_collab labor_markets
IdentificationExperimental variation of AI design characteristics (Adaptability, Expertise, Transparency) across two controlled scenario types (hiring negotiations; transactions with potential concealment) in a parallel simulated dataset and a human-subjects experiment, combined with conditioning on measured personality traits (Extraversion, Agreeableness) and use of causal-discovery algorithms to infer directional relations between AI attributes, personality, communication features, and outcome measures.
GeneralizabilityArtificially constrained scenarios (hiring and transactional vignettes) may not map to complex real-world organizational contexts, Human sample (n=290) may be non-representative (e.g., online convenience sample, limited demographics), Simulated agents may not capture behavioral richness of deployed AI systems, limiting external validity of simulation-human comparisons, Short-term interactions may not reflect longitudinal adaptation or learning between humans and AI, Findings may depend on specific operationalizations of Adaptability, Expertise, and Transparency and on cultural/contextual factors
Claims (10)
| Claim | Direction | Confidence | Outcome | Details |
|---|---|---|---|---|
| The study uses a purely simulated dataset comprising 2,000 simulations. Other | null_result | high | dataset_size |
n=2000
0.8
|
| The study includes a parallel human subjects experiment involving 290 human participants. Other | null_result | high | dataset_size |
n=290
0.8
|
| The study compares two interaction scenario categories: (1) hiring negotiations between human job candidates and AI hiring agents; and (2) human-AI transactions in which AI agents may conceal information to maximize internal goals. Task Allocation | null_result | high | scenario_type |
0.8
|
| The study examines user Extraversion and Agreeableness alongside AI design characteristics including Adaptability, Expertise, and chain-of-thought Transparency. Decision Quality | null_result | high | personality_and_design_factors |
0.8
|
| The analysis uses causal discovery methods and integrates scenario-based outcomes, communication analysis, and questionnaire measures. Research Productivity | null_result | high | analytic_method |
0.8
|
| Results reveal divergences between purely simulated and human study datasets. Decision Quality | mixed | high | comparative_outcomes_between_datasets |
0.48
|
| Results also reveal divergences between the two interaction scenario types. Task Allocation | mixed | high | scenario_specific_outcomes |
0.48
|
| In simulation experiments, personality traits and AI attributes were comparatively influential on outcomes. Decision Quality | positive | high | influence_on_interaction_outcomes |
n=2000
0.48
|
| With actual human subjects, AI attributes—particularly transparency—were much more impactful than personality traits. Decision Quality | positive | high | relative_influence_on_outcomes (AI_attributes_vs_personality) |
n=290
0.48
|
| These divergences (between simulation and human data and across scenarios) provide crucial insights for the future design of human-centered AI agents. Governance And Regulation | positive | high | design_implications |
0.08
|