In a controlled lab creativity test, GPT-4 partners yielded as much originality as human partners, while individual motivation and cognitive outsourcing determined who benefited from collaboration. The authors release a two-player AUT platform and dataset to standardize controlled studies of human–AI co-creation.
Two-player Alternate Uses Test: A Controlled Testbed for Interactive Human-AI and Human-Human Co-Creation
arxiv
quasi_experimental
medium evidence
7/10 relevance
Full text
usable extracted full text
Source
PDF
In a controlled two-player AUT pilot (N=62), participants partnered with GPT-4 produced originality scores statistically equivalent to those with human partners; benefits of interaction varied with approach motivation and cognitive outsourcing, and prior exposure to highly creative ideas (seeding) improved later originality.
Controlled research on AI ideation typically compares independent agents, while field studies of human-AI collaboration sacrifice experimental control. We introduce a controlled, two-player extension of the Alternate Uses Test (AUT) that enables comparison of human-human and human-AI co-creation under matched interactive conditions, alongside calibrated non-interactive baselines. The platform supports decomposition of performance into three typically confounded factors: participant traits, partner perceptions, and content dynamics. An in-person pilot (N = 62) demonstrates its utility. Under matched time limits, originality with a GPT-4 partner is statistically equivalent to that with a human partner. Approach motivation (BAS Drive) moderates whether interactive partnership benefits originality, and self-reported cognitive outsourcing predicts lower originality specifically in human-human dyads. Prior exposure to highly creative ideas improves later performance, suggesting a "seeding" intervention. We release the platform, code, and dataset as a shared testbed for controlled studies of human-AI co-creation.
Summary
Main Finding
A two-player, time-limited Alternate Uses Test (AUT) chat platform shows that, under matched interactive conditions, originality produced with a GPT-4 conversational partner is statistically equivalent to that produced with a human partner. The platform decomposes co-creative outcomes into participant traits, partner perceptions, and content dynamics; pilot results (N = 62) highlight heterogeneity in who benefits from AI collaboration (motivation and socioemotional sensitivity) and a manipulable “seeding” effect from exposure to highly creative exemplars.
Key Points
- New shared testbed: a browser-based two-player AUT chatroom supporting human–human (HUM), human–GPT-4 (GPT), and calibrated non-interactive (CON) conditions with matched instructions and enforced turn-taking for the GPT partner.
- Primary equivalence result: under matched 4-minute interactive sessions, originality scores for GPT vs human partners are statistically indistinguishable (β_GPT−HUM = −0.11, p = .42; TOST and Bayesian tests support the null).
- Heterogeneous treatment effects:
- BAS Drive (approach motivation) moderates benefits: high-Drive participants gain more originality from interactive partners (especially GPT) whereas low-Drive participants are harmed by human partners.
- Self-reported cognitive outsourcing predicts lower originality in human–human dyads but not in human–GPT dyads.
- Reading the Mind in the Eyes Test (RMET) relates to greater variance/spread in originality; socioemotional sensitivity matters even in text-only interactions.
- Content (seeding) effect: prior exposure to highly creative human exemplars for a different object raises later originality with both human and AI partners—an actionable intervention.
- Data & resources: pilot dataset of ~1,928 ideas, crowdsourced originality ratings (median 6 raters/idea), platform and code released.
Data & Methods
- Design: Within-subjects experiment; each participant (n = 62) completed three randomized 4-minute sessions (HUM, GPT, CON) on different objects (book, fork, tin can). Half of participants did a 2-minute solo AUT baseline before the sessions; the other half did it after.
- GPT configuration: GPT-4 via OpenAI Chat API, temperature = 0.7, max_tokens = 3000, server-enforced strict turn-taking; model instructed to obey one-idea-per-turn, single-sentence responses; token-overlap filter to avoid repetition.
- Non-interactive baselines (CON): pre-rated seed sets (highly creative human ideas, uncreative human ideas, or GPT-4-only ideas) used to calibrate content exposure effects.
- Outcomes: Originality rated on 5-pt Likert by 226 crowdworkers (median 6 ratings/idea); total ~1,923 distinct ideas.
- Analysis:
- Linear mixed-effects models with fixed effects for partner type and CON sub-arm, random intercepts for subject, item-within-subject, and rater.
- Random forest for nonlinear variable importance.
- Equivalence tested via TOST and Bayesian re-estimation (evidence favoring null of difference).
- Psychological measures: RMET for socioemotional sensitivity; BIS/BAS for motivational traits; five perception items (outsourcing, responsiveness, competence, etc.) collected post-session.
Implications for AI Economics
- Complementarity vs substitution is user-dependent: AI (GPT-4) can match a human collaborator on an interactive creative task, but gains are heterogeneous—high-approach-motivation workers benefit more, low-motivation workers may be hurt when working with humans. Labor-market impacts of co-creative AI will thus vary by worker traits (motivation, socioemotional sensitivity), not only by occupation.
- Task design and interaction structure matter for productivity estimates: enforced turn-taking and constrained model behavior influenced outcomes. Economic models of AI adoption should incorporate interaction modalities and institutional design (e.g., rules, interfaces) as determinants of realized productivity, not just raw model capability.
- Cognitive outsourcing risk: self-reported outsourcing reduced creativity in human–human dyads. Where workers defer cognitive effort (to people or tools), output quality can fall—implications for firm decisions about automation, monitoring, and incentive design. Policies or contracts that rely solely on passive AI suggestions may inadvertently reduce human-generated creative value.
- Seeding as a low-cost productivity lever: exposing workers to high-quality exemplars before collaborative ideation improved later creativity regardless of partner type. Firms can use curated “seed” content to raise aggregate creative output; this is a directly implementable intervention with economic return potential.
- Targeting and training: heterogeneity suggests targeted deployment (match AI tools to high-Drive workers) or training interventions (increase motivation or socioemotional sensitivity) to maximize AI augmentation benefits. Returns to training versus technology substitution should be evaluated in cost-benefit analyses.
- Measurement and causal inference: the testbed demonstrates the value of controlled interactive experiments for estimating causal impacts of AI on creative output. Economists should adopt similar controlled platforms to study complementarities, labor supply responses, wage effects, and equilibrium adjustments rather than relying solely on observational field data.
- Product and market design: designers of AI co-creative tools should consider interaction constraints (turn-taking, verbosity limits) to avoid unintended outsourcing and to enhance collaborative engagement. Market segmentation (tools tailored to motivational profiles) could emerge, with distinct pricing and feature bundles for different worker types.
- Research agenda: scale up samples across occupations and tasks, test alternative interaction protocols and model behaviors, and link individual-level creativity gains to labor outcomes (productivity, hiring, wages) to quantify aggregate economic effects and distributional consequences.
Limitations to bear in mind: pilot sample is modest and student-based; partner identity was not blinded; enforced GPT turn-taking differs from natural human dynamics; results are exploratory and require larger confirmatory studies.
Assessment
Paper Typequasi_experimental
Evidence Strengthmedium — The design offers strong internal control by matching interactive conditions and including non-interactive baselines, and it decomposes confounded factors, but the results come from a small in-person pilot (N=62) on a single creativity task, limiting statistical power and external validity.
Methods Rigormedium — Carefully constructed controlled platform, pre-registered comparisons implied, and decomposition of mechanisms increase rigor; however, small convenience sample, likely reliance on self-reports for some moderators, single-task measurement (AUT), and limited information on randomization/assignment and multiple-testing adjustments reduce overall methodological robustness.
SampleIn-person pilot with 62 adult participants (convenience sample) who completed a two-player extension of the Alternate Uses Test under matched time constraints, interacting either with a human partner or GPT-4, plus non-interactive baselines; measured originality scores, approach motivation (BAS Drive), self-reported cognitive outsourcing, and prior exposure (seeding) manipulation; code and dataset are released.
Themeshuman_ai_collab productivity
IdentificationControlled laboratory experiment comparing two-player interactive conditions (human-human vs human-AI with GPT-4) and calibrated non-interactive baselines under matched time limits; presumably random or counterbalanced assignment to partner conditions and a seeding exposure manipulation; decomposition of outcomes into participant traits, partner perceptions, and content dynamics to isolate the effect of partner type on originality.
GeneralizabilitySmall convenience sample and pilot-scale (N=62) — limited statistical power, Single creativity task (Alternate Uses Test) — may not generalize to other creative or productive tasks, Lab, in-person setting — different from real-world remote or workplace collaboration, Single AI model/version (GPT-4) — results may not hold for other models or later versions, Short, time-limited interactions — longer-term collaboration effects unknown, Demographic and cultural composition not reported — population external validity limited, Possible novelty/demand effects from interacting with AI in an experiment
Claims (8)
| Claim | Direction | Outcome | Confidence & Evidence | Details |
|---|---|---|---|---|
| We introduce a controlled, two-player extension of the Alternate Uses Test (AUT) that enables comparison of human-human and human-AI co-creation under matched interactive conditions, alongside calibrated non-interactive baselines. Other | positive | ability to compare human-human and human-AI co-creation under matched interactive conditions |
Reading fidelity
high
Study strength
medium
|
not reported
|
| The platform supports decomposition of performance into three typically confounded factors: participant traits, partner perceptions, and content dynamics. Other | positive | decomposability of performance into participant traits, partner perceptions, content dynamics |
Reading fidelity
high
Study strength
medium
|
not reported
|
| An in-person pilot (N = 62) demonstrates the utility of the platform. Other | positive | demonstration of platform utility (feasibility / proof-of-concept) |
Reading fidelity
high
Study strength
medium
|
n=62
|
| Under matched time limits, originality with a GPT-4 partner is statistically equivalent to that with a human partner. Creativity | null_result | originality |
Reading fidelity
high
Study strength
medium
|
n=62
|
| Approach motivation (BAS Drive) moderates whether interactive partnership benefits originality. Creativity | mixed | originality |
Reading fidelity
high
Study strength
medium
|
n=62
|
| Self-reported cognitive outsourcing predicts lower originality specifically in human-human dyads. Creativity | negative | originality |
Reading fidelity
high
Study strength
medium
|
n=62
|
| Prior exposure to highly creative ideas improves later performance, suggesting a 'seeding' intervention. Creativity | positive | later creative performance/originality |
Reading fidelity
high
Study strength
medium
|
n=62
|
| We release the platform, code, and dataset as a shared testbed for controlled studies of human-AI co-creation. Adoption Rate | positive | availability of platform, code, dataset |
Reading fidelity
high
Study strength
high
|
not reported
|