AI writing assistants are seeding academic papers with fake citations: an audit of 111 million references finds a surge in non-existent bibliographic entries after widespread LLM adoption, concentrated in fast-AI fields and among small or early-career teams. These hallucinated citations disproportionately credit already-prominent, male scholars and often survive moderation and peer review, risking biased and unreliable knowledge accumulation.

Main Finding

LLM-generated “hallucinations” have materially infiltrated the scientific literature: using citation existence as a verifiable signal, the authors document a sharp rise in non-existent (unverifiable) references after widespread LLM adoption. Conservatively, the four corpora studied would contain ~146,932 hallucinated citations in 2025 alone. Hallucinations are diffusely embedded across many papers (often as a few bad references per paper), are concentrated in fields with rapid AI uptake, correlate with linguistic signatures of LLM use, are disproportionately produced by smaller or early‑career teams, and tend to allocate credit toward already-prominent and male scholars. Existing moderation and peer‑review processes catch only a minority of these errors, and most hallucinations persist into the published record.

Key Points

• Scale and timing
  • Sudden increase in unmatched (unverifiable) citations begins in 2023 and accelerates mid‑2024; steep growth through Aug 2025.
  • Estimated excess (interpreted as hallucinations) as of Aug 2025: arXiv 0.39%, bioRxiv 0.21%, SSRN 1.91%, PMC 0.27%.
  • Monthly hallucinated-reference estimates in Aug 2025: arXiv ~3,353; bioRxiv ~478; SSRN ~767; PMC ~8,140. Extrapolated to 2025 (four corpora): ~146,932 hallucinated citations.
• Distributional pattern
  • Not concentrated in a few manuscripts; many papers contain a modest number of hallucinated citations (rise mainly in papers with 0–10% unmatched refs).
  • Higher rates in social sciences and computer science; positive correlation (r = 0.441) between estimated LLM use and hallucination rates at the subfield level.
• Who produces them
  • “Hallucination citers” are more likely to be less-experienced authors: large reductions in prior publication counts (e.g., ~62% fewer prior papers in arXiv/bioRxiv; up to ~73% in SSRN).
  • Hallucination citers exhibit rapid productivity increases by 2025 (relative increases of ~1.3–3.1× depending on corpus).
  • Hallucination rates fall sharply with team size.
• Who benefits
  • Hallucinated references often invent authors but, when mapped to real profiles, disproportionately credit high‑productivity and high‑impact authors (e.g., ~68.8% more publications; ~58.3% more citations) and skew toward male names (~6.4 percentage point increase).
  • Valid citations appearing in papers that also contain hallucinations also skew toward prominent scholars—suggesting LLMs reallocate credit toward incumbents even when not fabricating.
• Failure of safeguards
  • arXiv moderation rejects manuscripts with higher hallucination rates but an estimated 78.8% of non-existent citations still pass moderation and appear on the platform.
  • Tracing bioRxiv → published versions shows ~85.3% of preprint hallucinations persist into the published record.
  • Hallucinations appear across journal impact deciles; lower‑ and highest‑decile journals show higher and lower rates respectively, but most journals remain susceptible.

Data & Methods

• Datasets (Jan 2020–Aug 2025, unless noted)
  • arXiv: 1,465,145 preprints; ~44,107,529 citations (60.9% LaTeX source; remainder via GROBID from PDFs).
  • bioRxiv: 261,928 preprints; 21,183,111 XML citations.
  • SSRN: 421,698 preprints; 26,815,043 citations (Crossref metadata for SSRN DOIs).
  • PubMed Central (PMC): 10% random sample of recent papers — 374,807 manuscripts; 19,245,787 references.
• Hallucination detection pipeline
  • Built a local Elasticsearch index from Semantic Scholar and OpenAlex for title matching.
  • Initial match rate ~95.1%. Unmatched cases manually inspected: many were non-academic items or parsing errors.
  • Applied LLM-based cleaning (GPT-4o-mini) to remove non-academic or malformed references; refined title extraction to reduce unmatched rate to ~1.54%.
  • Final verification step: Google Scholar API lookup to capture items outside the indexes; remaining unverifiable references labeled “unmatched.”
  • Conservative focus: only non-existent titles (i.e., factual invalidity of the reference title) are counted; measurement errs on being a lower bound.
• Identification strategy
  • Use pre‑LLM unmatched citation rate as baseline for routine bibliographic/matching errors; interpret post‑LLM excess as population‑level signature of hallucination.
  • Regression framework and mixture model to separate baseline match failures (q) from LLM‑hallucinated probability (p). Controls for field, time, and other confounders; robustness and manual validations reported in SI.
  • LLM‑use intensity inferred via linguistic signatures in abstracts (standard methods) and correlated with hallucination incidence.
  • Author linking/disambiguation: name extraction + text‑based paper embeddings to link hallucinated cited names to real author profiles conservatively.

Implications for AI Economics

• Knowledge‑production externalities and cumulative risk
  • Even modest rates of hallucinated citations create systemic negative externalities because science is cumulative: fabricated references can mislead subsequent synthesis, meta‑analysis, automated summarizers, and downstream policy/industry decisions.
  • The persistence of hallucinations into published literature means market actors (funders, firms, policy bodies) and AI systems that scrape literature will internalize and amplify false signals.
• Redistribution of scientific credit and incentives
  • LLMs appear to skew citation patterns toward established, high‑status authors (and toward male names), potentially reinforcing incumbency advantages in reputation, hiring, funding, and bibliometric metrics (h‑index, citation counts).
  • This can alter incentives in academic labor markets and funding allocation: more citations to incumbents feed a feedback loop, concentrating attention and resources.
  • At the same time, LLMs lower production costs for less‑experienced researchers, increasing output but with higher contamination—producing a tradeoff between quantity and reliability.
• Market for verification and detection services
  • There is likely to be growing demand (and economic opportunity) for scalable verification tools, provenance trackers, and citation‑validation services (for journals, preprint servers, institutional repositories, and AI tool vendors).
  • Platforms and publishers face potential liability or reputational costs if they fail to contain hallucinations; this creates market incentives for third‑party validators and certification services.
• Platform and regulatory implications
  • Existing editorial and moderation systems are inadequate at scale; platform-level interventions (automated citation verification, provenance metadata for AI‑assisted writing, mandatory disclosure of tool use) could be economically justified.
  • Policymakers and funders may consider standards for AI‑assisted scholarly work, reporting requirements, or audits—creating compliance costs and new governance markets.
• Measurement and research evaluation distortions
  • Widespread LLM influence on citation behavior undermines the reliability of citation‑based metrics used in hiring, promotion, and funding decisions; measurement models and evaluation algorithms need to be adjusted to account for AI‑driven distortions.
• Heterogeneous effects across fields and actors
  • Fields with rapid LLM uptake (CS, some social sciences) and smaller teams are especially exposed — implying uneven economic impacts across disciplines and institutions.
  • Early‑career researchers are both disproportionately producing hallucinations and benefiting from increased productivity; policies should weigh short‑term productivity gains against longer‑term reputational and epistemic costs.
• Policy and market recommendations (high‑level)
  • Build and deploy automated, open, standardized citation‑verification pipelines integrated into authoring tools, preprint servers, and journal submission systems.
  • Require provenance metadata and disclosure for AI assistance in manuscript preparation (enforceable by journals/funders).
  • Invest in public, comprehensive bibliographic indexes and cross‑platform APIs to reduce false negatives/positives and lower verification costs.
  • Fund research into incentive‑compatible verification mechanisms and into methods that reduce biased outputs from LLMs (e.g., debiasing, calibrated retrieval).
  • Reassess evaluation metrics and hiring/funding criteria to guard against AI‑amplified incumbency biases.
  • Create marketplaces for third‑party verification and reputation services and consider liability frameworks for AI tool providers and platforms.

Limitations to keep in mind - Conservative focus on non‑existent titles means the estimates are likely lower bounds; other hallucination types (misattributed claims, incorrect page/volume, mischaracterized findings) are not captured. - LLM use is inferred indirectly (linguistic signatures); causality is supported by timing and correlations but direct causal attribution to specific tools is not established at the individual manuscript level. - Matching and disambiguation use conservative linking criteria; some real citations outside indexed sources could be misclassified despite multi‑step verification.

Summary takeaway LLM hallucinations have begun to penetrate scientific citation networks at scale, creating measurable epistemic and distributional distortions. This poses economic externalities across knowledge markets, incentives, evaluation systems, and platform governance — and creates immediate demand for verification infrastructure, updated norms, and policy interventions to preserve the reliability and equity of scientific production.