Autonomous coding agents can produce mergeable pull requests—but they rarely get in without human engagement: reviewer interaction is the strongest predictor of acceptance, while big edits and force-pushes hinder integration.

Main Finding

Most agent-authored pull requests (PRs) on popular public GitHub repositories are merged (≈71.5%), but successful integration is driven far more by review-time collaboration signals (especially receiving reviewer attention and avoiding coordination-disrupting actions) than by iteration volume (number of commits) or simply adding tests. Agents differ substantially in merge rates and decision speeds, implying that economic value of agents depends on how well they reduce human reviewer coordination costs, not just raw code correctness.

Key Points

• Scale and dataset
  • Analysis of 33,596 agent-authored PRs from the AIDev dataset (version 3, Nov 2025), filtered to “popular” repos (≥100 stars). Covers 2,807 repositories, 1,797 distinct developers and multiple coding agents.
• Outcomes (RQ1)
  • Aggregate: 71.5% merged, 21.6% closed without merge, 6.9% remained open (95% CI for merge share ≈ [0.710, 0.720]).
  • Strong heterogeneity by agent: OpenAI_Codex ≈ 82.6% merged; Copilot ≈ 43.0%; Devin ≈ 53.8%.
  • Decision latency varies by agent: OpenAI_Codex median <1 hour (mean ≈19.4 hrs), while agents like Copilot and Devin have medians of ~13 and ~9 hrs and much larger mean latencies (means exceeding tens of hours), indicating differences in triage and review effort.
• Collaboration signals (RQ2)
  • Method: logistic regression (merged vs closed-unmerged) with repository-clustered standard errors; controlled for agent identity and size-related features.
  • Strongest positive correlate: receiving at least one human review (reviewer engagement). PRs that get reviewer attention have substantially higher odds of merging.
  • Negative correlates: large change size (ΔLOC, files changed) and coordination-disrupting behaviors such as force pushes during review.
  • Iteration intensity (number of commits/revisions) and adding tests have limited independent association with merge probability once reviewer engagement and coordination stability are controlled for.
  • Conditional on receiving a review, longer time-to-first-review is positively associated with merging (likely reflecting repository prioritization patterns rather than a causal benefit of delay).
• Qualitative follow-up
  • Random/purposive sample of 60 PRs coded: the dominant successful pattern was an “actionable review loop” (agents respond to concrete feedback and converge on reviewer expectations) — of these, most merged. Failures were primarily due to design disagreements, incomplete solutions, process/policy blocks, coordination breakdowns, or failing CI.
• Interpretation
  • Integration success for agent-authored PRs is socio-technical: alignment with repo norms and low coordination cost matter more than sheer activity volume.

Data & Methods

• Data source
  • AIDev dataset v3 (Zenodo), accessed Nov 2025. Focus limited to curated popular repositories (≥100 stars).
• Sample
  • 33,596 agent-authored PRs from 2,807 repositories and 1,797 developers, covering five named coding agents.
• Operational definitions
  • Agent-authored PRs identified by non-null agent labels in pr_task_type (dataset does not distinguish between autonomous vs human-invoked agent submissions).
  • Outcomes: merged if merged_at present; closed-unmerged if closed_at present and merged_at absent; open otherwise.
  • Time-to-decision = created_at → merged_at or closed_at for resolved PRs (report mean and median).
• Features / collaboration signals
  • Iteration intensity: commit-count buckets or log(1 + commit_count).
  • Change magnitude: log(1 + ΔLOC), number of files changed.
  • Coordination stability: binary indicator for force-push during review.
  • Testing behavior: binary indicator for test-file modifications.
  • Reviewer engagement: indicator for whether PR received ≥1 review; log time-to-first-review.
• Modeling
  • Logistic regression (binary merge outcome) with repository-clustered standard errors; agent indicators to control for agent heterogeneity; results reported as odds ratios (associational).
• Replicability
  • Authors provide a replication package with scripts to reproduce RQ1 and RQ2 analyses.
• Threats to validity
  • Observational design: associations not causal.
  • GitHub trace limitations: artifacts may not fully capture review quality or off-platform coordination.
  • External validity: focused on large public repositories; results may not generalize to private or small-team contexts or future agent versions.

Implications for AI Economics

• Value of agents depends on integration cost, not just output quality
  • Economic benefit of coding agents is a function of reduced reviewer time and coordination cost. Agents that produce code aligned with project conventions and that engage in "actionable review loops" will generate more net value than agents that produce technically correct but coordination-heavy changes.
• Reviewer attention is a scarce economic resource
  • The single strongest correlate of merge probability is receiving reviewer attention. In economic terms, human reviewer bandwidth is a bottleneck. Agent providers and adopters should consider the marginal cost of reviewer time when evaluating ROI.
• Heterogeneous agent performance creates market differentiation
  • Wide variation in merge rates and decision latencies across agents implies market segmentation: buyers will prefer agents that minimize integration friction. This creates incentives for agent vendors to optimize for social-fit features (e.g., small, well-scoped changes; stable history; human-like responsiveness to review) and to display integration metrics (merge rate, median time-to-decision) as product signals.
• Coordination-disrupting behaviors impose negative externalities
  • Actions such as force-pushes raise coordination costs and reduce merge likelihood; such behaviors can impose negative externalities on downstream collaborators. Platform-level policies or agent certifications that discourage history-rewriting during active review could improve aggregate welfare.
• Productivity metrics should include socio-technical fit
  • Traditional productivity measures (lines of code, number of commits) are insufficient. Economic evaluations and compensation models should incorporate integration success and reviewer effort saved. Metrics like merge probability conditional on receiving a review, mean reviewer time per merged PR, and post-merge rework rates are more informative.
• Policy and governance implications
  • Repositories and platforms can steer agent behavior via policy (e.g., disallowing force-pushes during open PRs), review templates that elicit actionable feedback, and tooling that automates pre-review checks to reduce reviewer workload. Such interventions change the incentives and thus the economic value of different agent designs.
• Opportunities for ancillary markets and tools
  • Demand for tooling that pre-aligns agent output to repo norms (linters, automated PR descriptions that explain rationale, automated replies to reviewer comments) will grow; these tools can be priced as value-capture for reducing human review costs.
• Labor-market considerations
  • If high-quality agents materially lower reviewer time per merged PR, firms may reallocate human effort toward higher-level design tasks or expand throughput without proportional hiring. Conversely, agents that increase coordination friction could raise reviewer demand or slow delivery, negatively affecting adoption.
• Measurement and procurement guidance
  • Organizations procuring coding agents should evaluate agents on integration metrics (merge rate, reviewer time consumed, incidence of force-pushes, distribution of change sizes) and not only on isolated code-quality benchmarks.

Concluding note: This study reframes the economic value of autonomous coding agents around their ability to fit into human socio-technical workflows. For policy-makers, vendors, and adopters, the key lever is reducing human coordination costs and enabling actionable, convergent review interactions rather than maximizing raw code generation.