A cross-model audit of 70+ LLMs on ~26,000 real queries finds outputs have become strikingly similar, creating an 'Artificial Hivemind' that erodes stylistic and cognitive diversity; firms must invest in governance, process redesign and multi-model strategies to avoid correlated errors and loss of creative value.

Main Finding

Contemporary LLMs display strong intra-model repetition and inter-model convergence — an "Artificial Hivemind" — producing highly similar outputs across different architectures, training regimes, and scales. This homogenization erodes creative diversity in AI-assisted work, risks amplifying groupthink in organizational decision-making, and reveals miscalibration in reward models and evaluation systems that prefer consensus-style outputs even when stylistically diverse alternatives are equally high-quality.

Key Points

• Evidence of two related phenomena:
  • Intra-model repetition: single models often produce repetitive, low-diversity responses across similar prompts.
  • Inter-model convergence: different models (70+ examined) frequently generate strikingly similar outputs for the same real-world queries.
• Dataset and scale: analysis based on ~26,000 real-world user queries and outputs from 70+ language models.
• Evaluation gap: current reward models and automated evaluation metrics are biased toward consensus/high-probability responses, misaligning machine incentives with organizational needs for stylistic and cognitive diversity.
• Organizational risks:
  • Reduced creativity and solution variety in AI-augmented workflows.
  • Increased susceptibility to groupthink and correlated errors across teams using different models.
  • Misfit between organizational decision problems that require pluralistic perspectives and AI systems optimized for narrow consensus.
• Organizational responses proposed:
  • Leadership communication strategies to value and solicit diverse human/AI perspectives.
  • Work redesign: roles and processes to preserve human creativity (e.g., contrarian roles, ensemble-based workflows, mandated diversity checks).
  • Governance frameworks: auditing for homogenization, calibration of evaluation/reward systems to reward diversity, procurement policies avoiding monoculture.
• Practical tools: diagnostic frameworks and metrics (e.g., inter-model similarity, response entropy) for detecting and tracking AI homogenization in workflows.

Data & Methods

• Data: ≈26,000 real-world user queries paired with outputs from over 70 distinct language models spanning different providers, architectures, and scales.
• Analyses likely included (as reported):
  • Quantitative similarity measures (semantic/textual similarity, clustering) to assess intra- and inter-model output overlap.
  • Diversity metrics (e.g., entropy, distinct-n style measures) to quantify repetition and variability.
  • Human preference assessments demonstrating miscalibration: when outputs are similar in judged quality but differ in style, reward models and automated evaluations favor the consensus-style outputs.
  • Comparative evaluation of models across prompt types and task contexts to identify where homogenization is most pronounced.
• Note: the paper synthesizes empirical measurement with organizational-design recommendations; exact statistical methods, thresholds, and model lists are detailed in the article.

Implications for AI Economics

• Productivity and complementarities:
  • Homogenized AI outputs reduce the value of AI as a source of varied cognitive complements to human labor, potentially lowering productivity gains from human–AI collaboration in tasks needing creativity and exploration.
  • Firms that can design processes to preserve human diversity (and to elicit diverse AI outputs) may capture greater productivity gains, increasing returns to organizational capability rather than to raw model access.
• Market structure and vendor dynamics:
  • Inter-model convergence undermines product differentiation across AI providers, increasing price competition on marginal features (latency, cost) and shifting competitive advantage to governance, integration, and workflow design services.
  • Reduced differentiation could accelerate commoditization of base LLM outputs but open a market for value-adds (diversity-promoting tools, ensemble services, customization for non-conformity).
• Labor demand and skills:
  • Greater premium on human skills that preserve or generate diversity: contrarian reasoning, editorial curation, prompt engineering focused on diversity, and governance roles (AI auditors, diversity officers).
  • Routine augmentation tasks that rely on consensus outputs may be more easily automated, while tasks requiring pluralistic solutions remain human-intensive.
• Investment and governance costs:
  • Organizations will incur additional costs to audit, procure, and govern LLM use (diversity audits, recalibrating reward models, multi-model infrastructures), shifting some economic benefits of AI toward governance and integration spending.
• Policy and standards:
  • Findings motivate regulatory attention to systemic risks from algorithmic homogenization (e.g., correlated errors in critical systems) and potential standards for measuring and disclosing model diversity characteristics.
• Evaluation economics:
  • Miscalibrated reward/evaluation systems can lead firms to prefer models that maximize apparent evaluation scores but minimize useful diversity, producing allocative inefficiencies in model choice and adoption decisions.

Actionable takeaway for economists and practitioners: measure inter-model similarity and response diversity as part of ROI and procurement analyses; factor in governance and role redesign costs when estimating net returns to LLM deployment; and treat organizational capability to sustain cognitive diversity as a critical, economically valuable complement to model access.