AI coding assistants didn’t raise architectural defects but they did bloat code: in 151 open-source Java projects, adopters saw no change in total architectural smells while code size grew ~12.8%, causing a 6.7% drop in smell density driven by larger codebases rather than cleaner architecture.

Main Finding

Agentic AI adoption (detectable via project config files or Co-Authored-By commit trailers) is causally associated with a statistically significant 6.7% decline in architectural smell density (ASD) in open-source Java repositories. That decline is not driven by fewer architectural smells but by faster code growth: total smell counts are essentially unchanged (+1.1%, p = 0.82) while lines of code (KLOC) increase +12.8% (p = 0.003). In short, the ASD drop is a denominator (code-volume) effect rather than an architectural-quality improvement.

Key Points

• Sample and scope: 151 Java repositories (74 treated, 77 matched controls), monthly panel over 13 months per repo → 1,811 Arcan snapshots used for analysis.
• Treatment detection: agentic usage inferred from tool-specific configuration files and Co-Authored-By commit trailers (Cursor, Copilot, Claude Code, Aider, Codex). This detects agentic usage only (not inline autocomplete).
• Causal design: staggered difference-in-differences with Borusyak imputation estimator (primary), supplemented by Sun & Abraham TWFE correction and event-study dynamics. Repository and time fixed effects; SEs clustered by repo.
• Main estimates:
  • ASD: −6.7% (ˆβ = −0.070, p = 0.004).
  • Total smell counts: +1.1% (p = 0.82, not significant).
  • KLOC: +12.8% (p = 0.003).
  • Pre-trend falsification: Wald joint test not significant (p ≈ 0.90), consistent with parallel trends.
• Per-smell results: hub-like dependency density falls (~−5.0%, significant after Holm correction); cyclic dependency shows a similar point estimate but significance weakens after multiple-test correction. God components and unstable dependencies show no density change; raw UD counts show a marginal suggestive increase (not robust after correction).
• Robustness: wild cluster bootstrap, Lee bounds for attrition, stale-observation sensitivity, tool-stratified checks, Bacon decomposition and other sensitivity analyses reported. Attrition: some snapshots/repositories dropped due to Arcan failures (final panel 151 repos, 80.8% snapshot success).
• Measurement caveat: density-normalized outcomes (smells/KLOC) can be misleading if treatment alters system size; explicit numerator/denominator decomposition is required.

Data & Methods

• Data: public GitHub Java repositories meeting inclusion criteria (≥2,000 KB, ≥2 commits/month, ≥6 months pre and ≥3 months post adoption). Initial candidates filtered & propensity-matched; final analytical panel: 1,811 repo-month observations across 151 repos.
• Architectural metrics: Arcan CLI for smell detection (four smell types: cyclic dependency, unstable dependency, hub-like dependency, god component). Primary outcome ASD = total smells / KLOC; other outcomes: per-type densities, total counts, KLOC, coupling/instability measures.
• Identification strategy:
  • Staggered DiD to handle differing adoption months.
  • Primary estimator: Borusyak imputation (fits on untreated observations, imputes counterfactuals for treated post-periods).
  • Secondary estimator: Sun & Abraham interaction-weighted TWFE correction.
  • Event study used to inspect dynamics and pre-trends.
• Treatment definition: earliest observable agentic marker (config file addition or Co-Authored-By commit trailer). Recognized limitation: agentic-only detection — undeclared autocomplete use in controls creates attenuation bias; authors estimate empirical contamination ~8.9% with a corrected effect magnitude that strengthens (−7.4%) rather than reversing conclusions.
• Replication: full replication package (repo panel and code) is publicly available.

Implications for AI Economics

• Measurement matters for productivity/quality economics: density-based quality metrics (e.g., defects per LOC, smells per KLOC) can be confounded if AI adoption changes output scale. Empirical studies of AI effects on developer productivity or code quality must decompose ratio outcomes into numerator and denominator to avoid misleading inferences.
• Short-run output increases vs. structural risk: observable agentic AI use appears to increase code volume substantially in the short run without an immediate increase in architectural smell counts. From an economic standpoint, this resembles higher measured output per unit time (more LOC produced). But increased codebase size may have delayed maintenance costs and potential erosion effects not captured in the short window studied.
• Quality-adjusted productivity ambiguous: simple measures that treat LOC as output can overstate welfare/productivity gains if additional LOC imposes future technical debt or increases maintenance effort even while per-LOC smell density falls. Cost–benefit analyses and ROI models for AI developer tools should include long-run maintenance risk and not rely solely on density reductions.
• Heterogeneous effects and monitoring: architectural-level harms reported at code-level (e.g., higher static-analysis warnings) do not automatically translate to architecture-level deterioration in the studied horizon. Policymakers, firms, and tool vendors should deploy longer-term monitoring (architectural and code-level), instrument heterogeneity (tool types, agentic vs. autocomplete), and triangulate multiple metrics before drawing conclusions on systemic risk.
• Policy and managerial guidance: given the denominator effect, managers should complement productivity metrics with absolute counts of defects/smells and with architectural audits—especially after AI-driven bursts of development—to detect latent erosion. Incentives tied solely to output volume risk obscuring quality erosion that manifests only later.
• Research directions for AI economics: integrate these causal findings into models of developer labor productivity that allow for dynamic quality externalities (maintenance costs, defect arrival over time). Future work should evaluate longer horizons, other languages/ecosystems, private-codebases, and link architectural metrics to downstream economic outcomes (bug-fixing costs, time-to-change, developer churn).

Suggested next steps (brief): - Replicate with longer post-adoption windows to observe whether smell counts eventually rise with LOC. - Extend to other ecosystems and commercial repositories to assess external validity. - Combine architectural metrics with economic outcomes (maintenance cost, issue-fix time) to quantify welfare impacts.

If you want, I can produce a one-page slide-ready version of this summary or extract the key numeric tables/figures for a presentation.