AI-generated code is already common in new open-source contributions—mostly in boilerplate and tests—but certain classes of insecure templates are overrepresented and can spread across projects when human review is cursory.

Main Finding

AI-generated code (AIGCode) is already a substantial and structured component of modern open-source development. Adoption concentrates in boilerplate areas (glue code, tests, refactorings, docs) rather than core logic or security-critical configs, but AIGCode nonetheless creates distinct security risks: some CWE families are overrepresented, near-identical insecure templates recur across unrelated projects (indicative of model-driven propagation), and AI-driven high-throughput edits expand vulnerability exposure when human review is shallow. The authors build a high-precision ensemble detector and apply it at scale to trace AIGCode through projects and vulnerability life cycles.

Key Points

• Detection advance: the paper introduces the Cascade–Aggregation (ensemble) detection framework for distinguishing AIGCode from human-written code; the authors will open-source the model, dataset, and analysis artifacts.
• Benchmark dataset: a large, high-confidence evaluation benchmark with two balanced subsets:
  • Trusted human code: ~40k files from 172 repositories, commits dated 2008–2010 (pre-LLM era) to ensure label purity.
  • Trusted AIGCode: ~40k synthesized files from 11 LLMs across 165 tasks and 33 topics to capture cross-model diversity.
• Languages & domains: coverage across 10 mainstream programming languages (top: Java, C++, Python) and nine application domains (Web/apps largest share).
• Complexity measure: a lightweight Lexical Complexity Score (LCS) (counts control-flow statements and logical operators) used to compare complexity distributions; human-written data shows higher incidence of very high-complexity code.
• Large-scale application: detector applied to (i) development commits in the top 1,000 GitHub repositories (2022–2025) and (ii) 7,000+ CVE-linked code changes to study prevalence and security correlations.
• Ecological patterns found:
• Structured adoption: AI is used heavily for copy/boilerplate tasks (glue code, tests, refactors, docs), while core business logic and security-critical config remain predominantly human-authored.
• Security signals: certain CWE families are overrepresented in AI-tagged code; repeated, near-identical insecure templates appear across unrelated projects, consistent with propagation through shared models rather than shared maintainers.
• Human-AI edit chains: AI enables high-volume edits; humans often act as gatekeepers, but when reviews are cursory, AI-introduced defects persist longer, remain exposed on network-facing surfaces, and spread across files/repos.
• Limitations acknowledged: attribution/detection is inherently noisy and adversarially fragile; dataset choices (e.g., human subset time window) and synthesized AI samples may not capture all real-world generation modes.

Data & Methods

• Detection framework:
  • Cascade–Aggregation ensemble that combines multiple base detectors to improve accuracy and robustness against stylistic variation across models and languages.
  • Compared against seven representative content-detection approaches (transferred from natural-language AIGC detection) and evaluated using multiple metrics (accuracy, precision, recall, etc.).
• Benchmark construction:
  • Human subset: crawled source files (2008–2010) from high-starred repositories, filtered by file types, total ~40k files.
  • AI subset: structured synthesis of 165 tasks across 33 topics; generated by 11 mainstream closed- and open-source LLMs to capture generation diversity; total ~40k files.
  • Dataset covers 10 languages and nine domains; scanned with CodeQL + LLM-based checks to surface CWE distributions.
• Complexity metric: Lexical Complexity Score (LCS) = 1 + N_cf + (N_op)/2, where N_cf = control-flow occurrences, N_op = logical operators (language-adapted regex counting).
• Empirical application:
  • Top-1000 GitHub repositories: labeled commits, files, and functions on a human/AI axis for 2022–2025 to measure penetration and edit chains.
  • CVE-linked changes: analyzed 7,000+ vulnerability-associated code changes to correlate AIGCode presence with specific CWE families and propagation patterns.
• Analyses performed:
  • Granular measurements across commit→file→function units.
  • CWE frequency comparison between AI-tagged vs. human code.
  • Cross-repository template matching to identify near-identical insecure snippets suggesting model-propagated patterns.
  • Edit-chain analysis to quantify persistence, exposure (network-facing surfaces), and spread across files/repos under varying human review depth.

Implications for AI Economics

• Labor substitution vs. complementarity:
  • Partial automation: AIGCode concentrates in low-complexity, boilerplate tasks—implying productivity gains for routine work but limited replacement of skilled core-development labor. Expect complementarity: developers focus more on high-complexity design, security, and review.
  • Skill premium shifts: demand and wages may rise for roles emphasizing security review, code-ownership, and systems thinking (human gatekeepers), while routine coding tasks may see reduced marginal value.
• Firm-level costs and investment:
  • Security externalities: AI-induced vulnerabilities that propagate across unrelated projects create systemic supply-chain risk, increasing expected costs from breaches and remediation. Firms may need to invest more in code provenance, auditing, and security pipelines.
  • Compliance and tooling spend: increased demand for detection, verification, and automated auditing tools (and for insurers’ risk-assessment services). Vendors offering secure, provenance-aware AI code generation or integrated code-audit toolchains may capture significant value.
• Market structure and concentration:
  • Model-driven correlation risk: if many projects use the same LLMs, a single model’s insecure pattern can cause correlated losses across firms—raising the value of diversified modeling, model certification, or proprietary vetted generation services.
  • Platform advantages: providers that can certify/verify generation provenance or embed security alignment may command premium pricing (or become de facto defaults for enterprise usage).
• Insurance, liability, and regulation:
  • Insurers: need new actuarial models to price correlated systemic cyber risk from model-driven vulnerabilities; may require proof of provenance, mandatory audits, or minimum review standards.
  • Liability frameworks: findings strengthen the case for regulation or industry standards on provenance, watermarking, mandatory security reviews, and disclosure of model training provenance for high-risk code.
• Open-source ecosystem effects:
  • Faster code churn but higher systemic vulnerability risk may shift maintainers’ priorities—more time on triage and review, potential contributor fatigue, and higher coordination costs. Projects with strict review workflows may disproportionately attract enterprise adoption.
• Research and evaluation markets:
  • Growth opportunity for empirical security-as-a-service: continuous scanning for AI-pattern vulnerabilities, cross-repo template detection, and remediation patch markets.
• Policy-relevant trade-offs:
  • Productivity gains vs. systemic risk: policymakers and firms must balance faster development and lower per-unit coding costs with amplified systemic vulnerability exposure. Interventions (standards, provenance requirements, audit subsidies) can mitigate externalities.

Suggested next economic research directions: quantify net labor-hour savings vs. expected remediation/incident costs; measure wage dynamics for security-focused developer roles; estimate insurer premiums under model-correlated risk; and study adoption differences across firm size, governance regimes, and review intensity.