Integration headaches are an ecosystem problem, not just a bad bot: roughly 50% of pull-request friction persists at the repository level, and AI-authored changes concentrate that repository-level risk about twice as often as human contributions.
Autonomous coding agents now open and merge pull requests in shared repositories at scale, and the field evaluates them the way it has always evaluated components, one agent at a time, on isolated benchmark tasks. Yet agents that each pass their own tests still leave repositories that accumulate problems no single contribution accounts for. We ask whether this problem belongs to the individual agent or to the repository where it accumulates. We study integration friction, the cost of integrating a contribution into a codebase that other contributors are concurrently changing. Across more than 930,000 agent-authored pull requests, we measure how much of the variation in friction stays with the repository after the contribution, its author, its size, and its agent are accounted for. About half does, and it survives full controls. In the same repositories, agent-authored contributions concentrate this repository-level friction roughly twice as much as human ones (intraclass correlation 0.30 versus 0.16), a gap that holds after controlling for codebase size, age, task shape, process maturity, and merge path. The risk is a property of the ecosystem, not the agent. AI-native software is therefore better measured and governed at the ecosystem level than one agent at a time.
Summary
Main Finding
Repository-level properties—not individual agents—explain a large and persistent share of integration friction in AI-native software. About half of the variation in integration friction is attributable to the repository (an ecosystem-level signal) and this repository-level signal is concentrated roughly twice as strongly in agent-authored contributions as in human-authored ones (intraclass correlation ICC ≈ 0.30 vs 0.16). The result holds after controlling for PR size, author, agent identity, codebase size and age, task shape, process maturity, and merge path. Conclusion: risk is an ecosystem-level externality and should be measured and governed at that level rather than one agent at a time.
Key Points
- Problem reframing: Practitioners report faster arrival of agent-authored code than teams can absorb; the paper frames this as a measurement question of where friction originates—agent vs. repository (ecosystem).
- Operational definition of emergence: statistical non-reducibility — the share of variance in an outcome (integration friction) that remains at the repository level after accounting for PR- and agent-level covariates.
- Hypotheses tested:
- H1 (coupling): agents interact pervasively with humans and automation (not isolated).
- H2 (locus): friction is driven by base-branch churn (concurrent changes), not just number of agents.
- H3 (non-reducibility): a substantial share of friction variance is repository-level.
- H4 (specificity): repository-level non-reducibility is stronger for agent-authored contributions than human ones.
- Quantitative outcome: Repository ICC (variance between repos / total variance) is ~0.5 for overall friction measures (about half of variation); agent-authored PRs concentrate repository-level friction ≈ 0.30 ICC vs human-authored ≈ 0.16 ICC in the same repositories.
- Mechanism evidence: friction links to concurrent changes in the base branch (base-branch churn) rather than merely the presence of multiple agents editing simultaneously.
- The repository-level signal survives extensive controls, suggesting it is not explained away by measured confounders.
Data & Methods
- Datasets:
- AIDev (primary): agent-authored pull requests. Curated subset: 33,596 agent PRs across 2,807 repositories (>=100 stars). Full AIDev for cross-checks: 930,292 agent PRs across 116,211 repositories.
- AgenticFlict: deterministic replay of PR merges against evolving base branches, recording whether a PR would conflict (linked to 9,582 curated PRs, ~28.5% of curated set).
- Supplementary GitHub API fetches for human-authored PRs in the same repositories and for repository covariates (diskUsage, created_at).
- Constructs (examples): integration friction operationalized as merge latency (time to merge), number of review iterations, presence of merge conflicts, repeated review/comments, and base-branch churn while a PR is open.
- Method: multilevel (hierarchical) models with repository random intercepts. Model form (informal): η(y_ij) = fixed PR-level covariates + fixed repo covariates + u_j (repo random effect) + ε_ij. ICC = τ^2 /(τ^2 + σ^2) used as the metric of non-reducibility (τ^2: between-repo variance; σ^2: within-repo variance).
- Comparative test: matched human baseline — compute identical constructs for human PRs in the same repositories to test specificity (H4 / RQ3).
- Provenance: replication package, dataset revision stamps, re-derivation scripts.
Implications for AI Economics
- Externalities and risk allocation
- Agentic development produces ecosystem-level externalities: individual productivity from an agent can impose coordination and integration costs on the repository. Economic allocation of these costs is not captured by per-agent performance metrics.
- Market design and contracting (procurement, SLAs): buyers and platforms should price and contract for ecosystem risk (e.g., require or incentivize repository-level risk measures) rather than only agent accuracy metrics.
- Measurement for valuation, insurance, and M&A
- Valuation of software assets, insurance underwriting, and M&A due diligence should incorporate repository-level friction metrics (ICC-style measures, base-branch churn impact, conflict rates) because agent-level benchmarks understate systemic risk.
- Insurers and investors should treat “agent-enabled” development as a source of correlated operational risk across contributions to a repo; premium models and stress tests should account for elevated ecosystem concentration.
- Platform and regulator policy
- Platforms (GitHub, GitLab, marketplaces for coding agents) should surface repo-level signals (e.g., repository ICC, conflict replay results, churn while PRs open) to help governance, discovery, and compliance.
- Regulatory reporting or standards for “AI-assisted development” could mandate repository-level risk disclosures (e.g., proportion of changes authored by agents, conflict replays, integration-friction indicators).
- Incentives and governance
- Firms should invest in ecosystem-level governance (integration capacity, human oversight, explicit coordination protocols) rather than optimizing isolated agent performance. Incentives for contributors and agent vendors should align on repository-level outcomes.
- Design incentives for slower, more conservative merge policies (canarying, staged rollouts), stronger CI, or mandatory human sign-off for areas of high repository-level friction to internalize coordination costs.
- Productivity accounting and labor markets
- Productivity gains from agents are front-loaded and may produce downstream costs (rework, increased coordination overhead). Labor-market assessments of developer productivity and compensation models should account for these temporal externalities.
- Employers and analysts should avoid simple counts of agent-generated commits as positive productivity signals without controlling for ecosystem friction.
- Research and market signals
- New product opportunities: tools that measure, visualize, and mitigate repository-level friction (conflict simulators, ecosystem risk dashboards) have economic value. Standards for repo-level metrics could become a basis for third-party assurance services.
- Future economic models should treat agents as components that change the structure and externalities of production—e.g., endogenous model of integration capacity, investment in coordination, and platform-mediated liability.
Suggested practical steps (policy & product): - Require repository-level risk metrics (ICC-like, conflict replay rates, base-branch churn) in audits and procurement processes. - Insurers underwrite agent-enabled development using repo-level indicators; price correlated risk accordingly. - Platforms expose ecosystem signals and enable staging/canary environments to absorb integration friction. - Firms invest in integration capacity (review bandwidth, automated conflict detection, explicit documentation standards) rather than focusing only on agent throughput.
Areas for further economic research: - Causal identification of ecosystem-level risk (interventions, effective information measures). - Welfare analysis of agent adoption when integration costs are internalized vs. externalized. - Market design for liability, information disclosure, and incentives that align agent vendors, repositories, and downstream users.
Assessment
Claims (4)
| Claim | Direction | Outcome | Confidence & Evidence | Details |
|---|---|---|---|---|
| We analyze more than 930,000 agent-authored pull requests. Other | null_result | number of agent-authored pull requests analyzed |
Reading fidelity
high
Study strength
high
|
n=930000
|
| About half of the variation in integration friction stays with the repository after the contribution, its author, its size, and its agent are accounted for. Organizational Efficiency | positive | integration friction (cost of integrating a contribution into a concurrently changing codebase) |
Reading fidelity
high
Study strength
medium
|
n=930000
About half
|
| In the same repositories, agent-authored contributions concentrate repository-level friction roughly twice as much as human ones (intraclass correlation 0.30 versus 0.16); this gap holds after controlling for codebase size, age, task shape, process maturity, and merge path. Organizational Efficiency | negative | concentration of repository-level integration friction (measured by intraclass correlation) |
Reading fidelity
high
Study strength
medium
|
n=930000
intraclass correlation 0.30 versus 0.16 (agents vs humans); roughly twice as concentrated for agents
|
| The risk (integration friction) is a property of the ecosystem, not the agent; AI-native software is therefore better measured and governed at the ecosystem level than one agent at a time. Governance And Regulation | positive | appropriateness/effectiveness of measurement and governance approach (ecosystem-level vs agent-level) |
Reading fidelity
medium
Study strength
speculative
|
not reported
|