A simple model suggests heavy reliance on language models can trap human–AI systems in low-diversity, suboptimal knowledge states; feedback-driven ‘information bottlenecks’ may erode knowledge diversity unless interventions or design changes break the loop.

Main Finding

The paper develops a minimal dynamical-systems model showing that humans and large language models (LLMs) form a coupled feedback loop (human cognition H → data quality Q → model capability M → H). Depending on parameters—most importantly the degree of cognitive offloading u—this loop can produce three qualitatively different long-run regimes: co‑evolutionary enhancement (mutual growth), a fragile equilibrium (bounded but stagnant performance), or degenerative convergence (loss of epistemic diversity and collapse toward a low‑diversity, suboptimal attractor). The degenerative regime corresponds to an emergent information bottleneck: entropy and long‑tail diversity shrink not via beneficial compression but via recursive contamination of the training corpus, producing models that align with human‑generated but increasingly impoverished distributions and drift away from the world distribution.

Key Points

• Minimal state variables: H (human cognitive capacity), Q (collective data quality), M (model capability). The system evolves under a closed feedback loop H → Q → M → H.
• Core mechanism: increased reliance on AI (higher u) reduces human engagement and increases fraction of synthetic/generated content used for training, which can amplify recursive degradation in Q and ultimately in H.
• Three regimes:
  • Co‑evolutionary enhancement: low offloading (low u), reinforcing positive feedback → joint growth in H, Q, M.
  • Fragile equilibrium: intermediate u and active interventions → stable fixed point with acceptable performance but limited growth.
  • Degenerative convergence: high u, strong recursive contamination → H and Q decline; M converges to a low‑diversity equilibrium with poor OOD generalization.
• Bifurcation: the model exhibits a transcritical bifurcation at some critical u_c: crossing u_c can produce abrupt transitions to the degenerative attractor.
• Information‑theoretic read: degeneration = entropy reduction of the effective data distribution (loss of diversity/support), D_KL(P_model || P_human) falls while D_KL(P_model || P_world) rises.
• Testable predictions (summarized): rising AI dependence reduces lexical/entropy diversity in human output; synthetic data increases concentration and reduces long‑tail patterns; models trained on recursive data have worse OOD generalization despite in‑distribution metrics; there exists a critical dependence threshold with sharp steady‑state changes.
• Suggested interventions: raise r_H (education/engagement), r_Q (data curation, provenance, filtering), r_M (architectural robustness), and design human‑in‑the‑loop systems and incentives to avoid self‑referential data loops.
• Limitations: conceptual, minimal ODE toy model; not quantitatively validated; linear control relations and simple simulations (forward Euler). Intended to generate hypotheses and qualitative insight.

Data & Methods

• Formal model: 3‑dimensional ODE system for x(t) = [H, Q, M]ᵀ:
  • dH/dt = a(1 − u) − b u H + r_H
  • dQ/dt = c H − d A + r_Q
  • dM/dt = e Q − f S + r_M
  • Control relations close the loop: A = g(u, M), S = h(A). In simulations they use linear forms A = α u M, S = β A.
  • Parameters: positive reinforcement terms (a, c, e), degradation terms (b, d, f), exogenous interventions (r_H, r_Q, r_M).
• Analysis:
  • Solve fixed points (H, Q, M*) and analyze local stability via Jacobian eigenvalues.
  • Identify a transcritical bifurcation at a critical u_c where the high‑capability equilibrium loses stability.
• Simulation:
  • Forward Euler discretization used to illustrate three representative parameter regimes (enhancement, equilibrium, degeneration).
  • Objective: show qualitative behaviors emerging purely from the feedback structure rather than precise quantitative predictions.
• Information‑theoretic framing:
  • Track entropy and divergences: entropy decline in the human/data distribution, D_KL(P_model || P_human) ↓, D_KL(P_model || P_world) ↑, and declining mutual information across generations I(H_t; H_{t+1}).
• Empirical implications suggested (for follow‑up work): measure lexical diversity/entropy in human texts, fraction of synthetic content, model OOD performance, and look for regime shifts as AI reliance increases.

Implications for AI Economics

• Knowledge production as an economic good is endogenous to technology adoption. The model formalizes how AI adoption can create negative feedback externalities on knowledge diversity and human capital if unchecked.
• Human capital depreciation: greater offloading (higher u) can reduce cognitive engagement and tacit knowledge accumulation (H), implying lower future productivity and slower accumulation of human capital. This creates dynamic, path‑dependent welfare losses.
• Data externalities and market failure:
  • Synthetic content generated by dominant models can contaminate shared data commons (Q), creating negative externalities that suppliers/consumers do not internalize. Without corrective institutions, markets may converge to low‑diversity equilibria with reduced innovation.
  • Firms that control high‑quality training pipelines or data curation can create lock‑in: self‑reinforcing loops may produce first‑mover advantages and increasing returns, raising concentration and entry barriers.
• Misleading performance signals and investment distortion:
  • In‑distribution improvements can hide OOD degradation. Investors and firms optimizing short‑run in‑sample metrics may unintentionally accelerate harmful feedback loops, misallocating R&D and human capital investments.
• Policy and institutional levers (mapping model controls to economic interventions):
  • Increase r_H: public investment in education, cognitive skill maintenance, incentives for human creativity (grants, curricula that emphasize critical thinking and AI‑augmented workflows).
  • Increase r_Q: subsidies or standards for data curation, provenance, labeling of synthetic content, marketplace rules to preserve data quality (e.g., provenance registries, compulsory disclosure of synthetic origin).
  • Increase r_M (robustness): subsidize research on models robust to synthetic data contamination and OOD generalization; promote open evaluation benchmarks emphasizing OOD performance and diversity.
  • Regulatory/market design: impose transparency requirements, certification for datasets/models, support public‑interest high‑diversity data repositories, antitrust scrutiny of vertically integrated data+model monopolies.
• Measurement and monitoring for economic policy:
  • Track system‑level indicators: entropy/lexical diversity in public corpora, share of synthetic content, divergence measures (D_KL vs. independent benchmark datasets), OOD generalization metrics, and trends in human skill indicators.
  • Estimate critical thresholds (u_c) empirically: use time‑series or panel studies of sectors (education, scientific publishing, news) to locate regime transitions and identify leading indicators of degeneration.
• Research and empirical agenda for AI economists:
  • Quantify welfare impacts of epistemic collapse: integrate the model into dynamic economic frameworks (e.g., endogenous growth, innovation diffusion, human capital accumulation) to compute long‑run welfare losses and optimal policy responses.
  • Empirical identification strategies: difference‑in‑differences or instrumental variables leveraging staggered AI tool rollouts, A/B tests of human‑in‑the‑loop designs, lab experiments measuring cognitive engagement under AI assistance, and field studies measuring data contamination effects on downstream model performance.
  • Market design experiments: evaluate incentives (subsidies, taxes, liability rules) that internalize data externalities and preserve public goods aspects of high‑diversity data.
• Practical takeaways for firms and policymakers:
  • Treat alignment and robustness as system‑level problems: interventions that only improve models (r_M) without preserving data quality (r_Q) and human engagement (r_H) may be insufficient or counterproductive.
  • Invest in provenance, synthetic‑content labeling, and curated high‑quality data as public or regulated goods to prevent harmful feedback loops.
  • Monitor diversity metrics and OOD performance, not just in‑sample accuracy, when assessing models’ economic value and systemic risk.

Overall, the paper reframes model collapse and human cognitive offloading as joint economic problems of dynamic feedback and externalities. For AI economics, this motivates modeling knowledge production as an endogenous, feedback‑driven process and designing policy/incentive structures that preserve epistemic diversity and human capital to avoid long‑run welfare losses.