Large language models produce better individual product ideas and more top performers, but they converge on a narrower set of concepts than humans; careful model choice, pooling and prompt design largely restore human-level diversity.

Main Finding

LLMs produce higher-average and higher-tail-quality product ideas than humans (AI ideas are 7× more likely to fall in the top 10% by purchase intent), but they generate substantially less novelty and set-level diversity. This diversity loss can be largely mitigated (though not fully eliminated) by model choice, pooling across vendors, prompt engineering (e.g., Chain-of-Thought, injected personas/constraints), creative-agent workflows, and—owing to AI’s near-zero marginal cost—by scaling up the number of ideas generated.

Key Points

• Quality
  • Study 1a: AI-generated ideas (for physical products targeted at college students priced ≤ $50) have higher average quality measured by customer purchase intent; AI ideas are ~7× more likely to rank in the top 10%.
  • Study 1b: The quality advantage is not explained by superior AI “pitching” — rephrasing human ideas in AI writing style did not change purchase intent.
  • Study 1c: Human-rated novelty of AI ideas declined only slightly, indicating AI outputs are not simply verbatim regurgitations.
• Diversity and Novelty
  • Study 2a: AI-generated idea sets are substantially less diverse (ideas are more semantically similar to each other) than human-generated sets.
  • Study 2b (meta-analysis): Across multiple prior LLM-creativity datasets, AI ideas show consistently lower diversity (effect sizes ranging small→very large) under multiple diversity metrics.
• Mitigation strategies
  • Study 3a: Newer LLM versions and pooling across different vendors/models increase diversity; more recent models are more diverse but still below human-level diversity.
  • Study 3b: Procedural/prompt interventions—Chain-of-Thought prompting, injecting heterogeneous personas or constraints, and using specialized creative agents—meaningfully increase idea diversity.
  • Study 3c: Scaling the number of AI-generated ideas (e.g., from 100 → 1,000) steadily increases coverage of the idea space and can approach human-level coverage; some human-explored regions remain hard for AI to reach.
• Practical conclusion: LLMs are powerful for finding high-quality outliers and for high-throughput ideation, but managers should combine technical and procedural interventions to avoid homogenized portfolios.

Data & Methods

• Research design: Eight linked studies combining original experiments, controlled comparisons, meta-analysis, and mitigation experiments.
• Primary setting: New-product ideation for college-student-targeted physical products priced ≤ $50. Human ideas came from an MBA innovation course (50 students; ideas collected 2021, pre-LLM widespread use).
• Core outcome measures:
  • Idea quality: customer purchase-intent ratings (market-validated metric).
  • Novelty: human-judge novelty ratings.
  • Diversity: semantic-distance based metrics at the set level (multiple metrics applied; also applied in meta-analysis to external LLM-creativity datasets).
  • Coverage: extent of idea-space covered as number of AI ideas scaled.
• Experiments and comparisons:
  • Study 1a: Direct comparison of human vs. LLM outputs (zero-shot and few-shot prompting variants).
  • Study 1b: Control for communication by rephrasing human ideas with LLMs and re-evaluating purchase intent.
  • Study 1c: Human novelty ratings to assess regurgitation concerns.
  • Study 2a: Quantified semantic diversity difference between human and AI idea sets.
  • Study 2b: Meta-analysis applying diversity metrics to three prior studies (Stevenson et al. 2022; Hubert et al. 2024; Boussioux et al. 2024).
  • Study 3a: Technical levers — model version/vendor comparisons, sampling parameters, pooling vendor outputs.
  • Study 3b: Procedural levers — Chain-of-Thought prompting, personas/constraints, creative-agent architectures combining human and AI exploration.
  • Study 3c: Scaling experiments — generating 100→1,000 AI ideas and measuring coverage gains.
• Models evaluated: Multiple LLM vendors and versions (examples in paper include GPT-series and other vendors), with experiments showing newer models generally yield greater diversity.
• Robustness checks: Controlled for persuasion/pitch effects, applied multiple diversity metrics, and cross-validated results via meta-analysis.

Implications for AI Economics

• Productivity vs. Exploration trade-off
  • LLMs increase ideation productivity and raise the expected value of the best ideas (favorable for high-variance search problems), shifting managerial focus toward high-throughput search and selection.
  • Simultaneously, LLMs’ tendency to converge on high-probability associations narrows exploration, potentially reducing portfolio robustness and missing niche/novel opportunities.
• Returns to scale and marginal cost
  • Near-zero marginal cost of LLM ideation turns quantity into a practical mitigation: scale can restore coverage of the idea space, implying new optimal investment rules for search effort (generate many inexpensive AI ideas and then filter/select).
• Organizational strategy
  • Firms should combine AI ideation with methods that diversify the search (use multiple models/vendors, invest in prompt engineering, run creative-agent workflows, and retain human ideators for hard-to-reach solution regions).
  • “Model diversification” (pooling outputs across vendors/versions) is analogous to portfolio diversification and can be an inexpensive way to increase idea-space coverage.
• Market structure and competition
  • If many firms rely on similar LLMs and prompts, product-market homogenization risks increase (less product variety, increased direct competition). Firms that invest in mitigation strategies (bespoke prompts, creative agents, proprietary data) can gain differentiation advantages.
• Welfare and policy considerations
  • Consumer welfare: higher-quality mainstream products may increase welfare, but reduced diversity could harm consumers who prefer niche offerings.
  • Intellectual property and originality concerns: small but present reductions in assessed novelty and the possibility of training-data leakage warrant caution (due diligence, provenance checks).
  • Labor and task allocation: managers can reallocate human effort from quantity generation toward high-value curation, exploration of under-explored regions, and integration tasks.
• Research implications for AI economics
  • Need for models of innovation search that incorporate AI’s productivity and homogenization tendencies.
  • Empirical work should quantify when scaling AI ideation is substitute vs. complement to human ideation across domains.
  • Policy and antitrust analyses should consider how commoditization of ideation via common LLMs affects product differentiation and industry dynamics.

Limitations noted by authors - Domain: experiments focused on low-cost college-student products; generalizability to other product types, industries, or more complex innovation tasks requires further study. - Training-data and model-update specifics: model behavior depends on vendor training and updates; results may change as models evolve. - Some human-explored idea regions remained difficult for LLMs to reach even after mitigation—human creativity still contributes unique value.

Actionable managerial takeaways (brief) - Use LLMs to rapidly generate large idea pools to surface high-quality outliers. - Counteract AI homogenization by: pooling vendors/models, using Chain-of-Thought and persona/constraint prompts, deploying creative-agent workflows, and allocating human effort to explore remaining blind spots.