Auto-generated, not human-reviewed

Research Productivity

Updated Jun 13, 2026

Papers 373 (252 full-text)

Claims 988

Evidence strength: Mixed: near-term workflow gains are causally identified; broader impacts mostly observational

Bottom Line

LLM feedback raised revision rates by 12.55% in a randomized field experiment on 31,000+ arXiv manuscripts Wang (2026). Countries with stronger AI-assisted peer review are associated with 18-25% higher scientific output Han (2026). Quality is the main risk: audits and benchmarks show fabricated citations, low accuracy on predicting experimental outcomes, conservative idea generation, and reproducibility gaps. These call for verification and governance, not full autonomy Zhao (2026); Sehwag (2026); Bao (2026); Iarygina (2026).

What This Means in Practice

Start with tasks where evidence is strongest: drafting and editing, literature search, writing specs, and skills inventories. Use retrieval-grounded systems (they search and cite sources) and add automated citation-audit checks to catch fake references. Keep humans responsible for verification Jiang (2026); Zhao (2026); Dass (2026).
Keep humans in the loop for hypotheses and critical decisions. Benchmarks show low accuracy on experimental-outcome prediction and conservative idea generation Sehwag (2026); Bao (2026).
Measure impact credibly: run internal randomized controlled trials (RCTs) or natural experiments that exploit rollout or policy changes, and track productivity-quality trade-offs during deployment Kelly (2026); Paskov (2026).
Build integrity and provenance into workflows: require source-grounded outputs, automated reference checks, visible AI disclosure, and reproducibility artifacts. Add open-world evaluations (realistic tasks outside fixed benchmarks) before adoption Zhao (2026); Iarygina (2026); Kapoor (2026).
Use agents as assistants, not substitutes, for long-horizon or complex tasks. Agents are software that plan and execute multi-step actions across tools. Set human checkpoints and clear handoffs where agents underperform Wang (2026); Wang (2026).

What the Research Finds

1) Documented gains in routine research workflows

In a randomized field experiment on 31,000+ arXiv preprints, tailored LLM feedback increased manuscript revision rates by 12.55% vs. control Wang (2026).
Cross-country analyses link stronger AI-assisted peer review to 18-25% higher scientific output, faster review, and better reproducibility metrics within countries over time Han (2026).
In controlled demos, decentralized agent teams beat single agents on computational science tasks, and an LLM-based lab system reproduced results and validated a novel optical interaction Gao (2026); Yang (2026).
Feedback-loop tuning improved performance across 21 scientific tasks, and retrieval-grounded idea generation outscored unguided generation on future-impact proxies in time-split tests (train on past, test on future) Ye (2026); Jiang (2026).

2) Clear limits for scientific reasoning, novelty, and autonomy

On 405 experimental-outcome prediction tasks in physics, biology, and chemistry, LLMs reached 14-26% accuracy and were poorly calibrated (weak at knowing when they might be wrong). Human experts were better calibrated and better at deciding when to run experiments Sehwag (2026).
In a large scientist-in-the-loop test, LLM ideas were plausible but conservative, converging and rarely proposing risky or falsifying ideas. Automated judges mis-scored usefulness; a reward model trained on human ratings improved alignment Bao (2026).
Benchmarks indicate agents are far from research replacements. The best setups emulated 68.3% of researcher lifecycle tasks, with frequent misses on nuanced judgments Wang (2026).
Long-horizon desktop-agent tests show persistent failures in multi-step execution and asking clarifying questions. Top systems scored about 31.6% on standard protocols and often failed on professional tasks Wang (2026); Jian (2026).

3) Integrity and reproducibility risks that can erase productivity gains

Audit of 111 million references: sharp rise in non-existent citations since LLM uptake, concentrated in AI-active fields, smaller or early-career teams, and manuscripts with linguistic signs of AI assistance Zhao (2026).
Only 49% of tested HCI (human-computer interaction) papers with shared code or data were computationally reproducible due to missing data, non-runnable code, and undocumented preprocessing Iarygina (2026).
When 150 autonomous coding agents analyzed the same dataset and questions, results varied widely due to agent-specific methods. Exposure to exemplars reduced dispersion via imitation rather than better methods Gao (2026).
In urban-infrastructure scenarios, LLMs produced well-structured but often ungrounded recommendations, fabricating over half of cited sources as complexity rose Poudel (2026).

4) How to measure and govern research-productivity effects credibly

A 5-principle, 33-guideline framework adapts randomized-trial and transparency standards to AI evaluation. Fast-changing models strain validity, so use adaptive designs and ongoing monitoring Kelly (2026); Paskov (2026).
Statistical methods that include AI outputs as covariates (control variables) can keep estimates valid and cut human labeling. In RCTs, adding AI-based risk predictions as covariates can lower variance with do-no-harm guarantees Lu (2026); Arbour (2026).
Open-world evaluations complement benchmarks by testing near-deployment capabilities. In one pilot, an agent developed and shipped a simple iOS app with one avoidable human intervention Kapoor (2026).
Retrieval-grounded idea generation produced 2.5x higher downstream-impact scores than an unguided generator. LLM judges missed this advantage, suggesting human or reward-model evaluators for high-stakes gates Jiang (2026).

5) New since the cutoff: stronger near-term gains, sharper integrity and autonomy caveats

2026 randomized arXiv experiment: AI feedback increased revision rates Wang (2026).
2026 audits and scientist-in-the-loop tests flag quality risks and autonomy limits: rising hallucinated citations, conservative ideas, and evaluator misalignment Zhao (2026); Bao (2026).
New agentic-science demos show faster loops and some novel findings in controlled settings; they do not substitute for general research Gao (2026); Yang (2026); Wang (2026).

What We Still Don't Know

Long-run, causal effects on scientific quality, novelty, acceptance, and citation impact across fields and institutions. Most evidence is short-term or observational Han (2026); Wang (2026).
Net ROI for labs and universities once verification, reproducibility, and governance costs are counted, including how hallucinated citations affect workload and reputation Zhao (2026); Iarygina (2026).
How benchmark performance maps to field performance on long-horizon, complex tasks, and when agents degrade outcomes without human checkpoints Wang (2026); Wang (2026).
Task-level complementarity: which activities (hypothesis generation, methods selection, analysis, writing) benefit most, and where AI induces convergence or reduces idea diversity Bao (2026).
Scalable, reliable ways to prevent, detect, and correct fabricated references and ungrounded claims in publishing and peer-review pipelines without excessive burden on researchers and editors Zhao (2026).