Open-source package careers are largely lottery-like: biggest hits arrive at roughly random positions while a small, stable positional advantage explains about one-fifth of impact variance. The emergence of generative AI tools like ChatGPT produced no measurable rupture in this luck–skill balance among package authors.
Chance, Persistent Advantage, and the Generative-AI Era in Open-Source Package Careers
Open-source package careers follow the random-impact rule—timing of a contributor’s biggest hit is close to random—while a modest persistent positional advantage explains roughly 20% of impact variance, and the arrival of generative AI produced no detectable change in this luck–skill structure.
Citation observations
Cumulative provider counts captured on specific dates; providers are never combined.
No provider observation is available for this paper.
Missing data, not a zero citation count.
Studies of careers in science, film, music, and books report a common pattern. When a person's most successful work arrives is close to a random draw over the works they produce. How large their successes tend to be, in contrast, follows a stable, person-specific factor. We test whether this pattern holds for open-source software careers and whether it changed when generative AI coding tools arrived. From the complete public record of GitHub push events (2015-2025), we reconstruct 102.2M career works by 6.15M contributors, and for the 908k contributors whose repositories publish packages, we measure each work's impact by how many downstream packages come to depend on it. First, we find that the timing of a career's biggest hit is close to a lottery over their works, as in science and the arts, with a small, replicable lean toward early career that grows as careers get longer. Second, some coders reliably produce higher-impact work than others, but this lasting personal factor accounts for only part of why impact persists (about a fifth in our primary specification); the rest behaves like momentum, success feeding on itself for a period of time. Third, within the same contributors, this structure did not change after ChatGPT's release. The stable factor's weight grew by about as much as it grew for an earlier cohort that simply aged, and subtracting the effect of aging from the effect of generative AI puts the shift at +0.03 (95% CI [-0.22, +0.23]), indistinguishable from zero. The success pattern documented in science and the arts therefore describes open-source careers too, and it shows no detectable break across the arrival of generative AI. These results have implications for how track records on open platforms should be read and on what to expect from generative AI for the careers built on them.
Summary
Main Finding
Open-source package careers follow the same luck–skill pattern documented in science and the arts: the timing of a coder’s single biggest hit is close to a lottery over their works, while the size of successes splits into a persistent, person-specific positional-advantage factor and a transient momentum effect. The persistent factor explains only about one-fifth of persistent differences in impact in the paper’s primary specification; the remainder behaves like momentum. Crucially, within the same contributors, the arrival of mainstream generative-AI coding assistants (post–ChatGPT/Copilot) produced no detectable change in this luck–skill decomposition (estimated era shift +0.03, 95% CI [−0.22, +0.23]).
Key Points
- Random-impact rule holds: a contributor’s highest-impact work is nearly equally likely to occur at any position in their career, with a small, reproducible lean toward earlier works that grows with career length.
- Decomposition of impact:
- Persistent person-level factor (Φ, interpreted conservatively as positional advantage) explains ≈20% of variance in impact in the primary specification.
- Remaining persistence is largely momentum (success feeding on itself for a period), plus work-level luck and noise; many works have zero measurable downstream impact (≈63% at 60 months).
- Robustness: results hold across three credit-allocation rules (R-share primary, R-lead, R-core), multiple impact horizons (24/36/60 months), and alternate impact measures (reach through dependencies and in-degree).
- Generative-AI era comparison: using a within-person, pre-registered, placebo-controlled design (discovery/held-out split), the structure of success (stable factor vs. momentum) did not shift in a statistically detectable way after the wide release of Copilot/ChatGPT once calendar aging of careers was accounted for.
Data & Methods
- Data sources:
- GitHub Archive push events (Jan 2015 – Sep 2025) yielding 102.2M "works" (a person’s contribution to a repository) by 6.15M contributors.
- Dependency graphs from Libraries.io and Ecosyste.ms (multiple snapshot releases); interpolation used to bridge 2020–Aug 2022 gap (sensitivity checks performed).
- Estimable population (repositories that publish packages): 2.44M works by 908,417 contributors across 1.82M repositories.
- Identity resolution:
- Commit-author email hashes decoded via a hash-dictionary built from account login/id formats; backup 80% rule and a bot filter applied.
- Validated on ORCID-linked gold set: precision ≈1.00 for 2015–19, ≥0.98 in later eras; higher resolution in later years (resolved share: 60.6% in 2015–19 → 74.7% in 2023–25).
- Definitions and measurement:
- Work impact = number of downstream packages that depend on the repository (direct or via one intermediary = reach2); alternate: in-degree (direct dependents).
- Impact evaluated at W ∈ {24, 36, 60} months after the work; log-transformed and year×ecosystem-centered to make scores comparable across languages/ecosystems and calendar time.
- Credit rules:
- R-share (primary): split proportional to commit shares.
- R-lead: largest committer gets credit.
- R-core: equal split among committers responsible for 80% of commits.
- Experimental design / pre-registration:
- Full pre-registered analysis grid (54 conditions: 3 credit rules × 3 horizons × 3 minimum career lengths × 2 impact measures).
- Contributors split 50/50 into discovery and held-out halves; exploration on discovery, confirmatory results from held-out half.
- Multiple-testing correction via Benjamini–Hochberg.
- Tests and models:
- RQ1 (lottery test): for each career compute P = position-of-max / career-length; compare observed P distribution to the uniform prediction (metrics: mean P, total variation distance, permutation p-values).
- RQ2 (stable-factor decomposition / Q-model analogue): compute r(k) = correlation between impacts of works k positions apart (on normalized log scale), pooled across contributors; fit r(k) = A + B ρ^k with constraints (A,B ≥ 0; 0 ≤ ρ ≤ 0.9). Interpretation: A = long-run plateau ≈ share of variance due to stable person factor (Φ); Bρ^k = momentum term. CIs via contributor-level bootstrap (1,000 resamples). Pre-registered checks ensure A is meaningful (e.g., A > 0.10 threshold, plus other robustness tests).
- RQ3 (generative-AI era): within-person, placebo-controlled comparison of the same contributors before vs. after the generative-AI era cutoff (post–Nov 2022), controlling for career aging and for incomplete impact accumulation of recent works. Also compared era effect against an aging cohort to isolate AI-specific shifts.
- Sample sizes and features:
- Main estimable sample: ≈14,091 careers in the discovery half for one primary condition (similar scale in held-out half).
- High prevalence of zero-impact works: 63% zero reach at W=60 in the full pipeline; 56% within primary sample.
- Limitations noted by authors:
- Measurement limited to package-publishing contributors (infrastructure-focused population).
- Dependence on external snapshot datasets and an interpolation for 2020–2022.
- Identity resolution leaves some commits unassigned (reducing coverage rather than creating false attributions).
- The stable factor is interpreted conservatively as positional advantage (will conflate skill, visibility, institutional effects, and platform biases).
Implications for AI Economics
- Productivity gains ≠ reordering of winners: observed generative-AI adoption did not measurably change who ends up with persistent high-impact outcomes in this infrastructure-focused open-source segment. AI may increase throughput without altering the luck vs. position decomposition of career success.
- Persistence of positional advantage: ≈20% of long-run variance in impact is tied to a stable, person-specific factor that likely combines skill, network/visibility, and platform position. Economists and practitioners using GitHub activity as a signal of ability should account for substantial luck and momentum components.
- Momentum and cumulative advantage remain important: much of the persistence in impact is transient momentum (success begets further success), reinforcing cumulative-advantage mechanisms in platformed creative labor markets. Policies or hiring practices that ignore momentum may over-attribute causality to individual ability.
- Heterogeneous effects of AI require micro-to-macro linkage: lab and field experiments show AI speeds task completion (especially for less-experienced workers), but career-level outcomes may be shaped by network structure, platform evaluation norms, and path-dependent dynamics that blunt or re-channel those task-level gains. Evaluations of AI’s distributional effects should therefore consider network externalities and cumulative processes, not just per-task productivity.
- For measurement and evaluation: public-track-record platforms (GitHub) contain substantial noise from chance; downstream-dependency measures and long horizons are necessary to evaluate lasting impact. Short-run activity spikes produced by AI-assisted work may not translate into durable increases in relative career standing.
- Directions for further research:
- Examine other segments of software work (e.g., application-level projects, corporate codebases) and non-package-maintainer populations to test external validity.
- Longer post-AI observation windows to detect slower-moving redistribution effects.
- Mechanism-focused work to trace how AI affects visibility, onboarding, review rates, and other channels that could alter momentum or positional advantage.
- Policy experiments aimed at altering cumulative-advantage channels (e.g., discoverability algorithms, reviewer/maintainer incentives) to evaluate whether platform design moderates persistence and inequality.
If you want, I can extract the main quantitative estimates into a one-page visual-ready brief (figures/tables-ready text) or prepare a concise slide outline of the results and implications for policymakers and firms.
Assessment
Paper Typecorrelational
Evidence Strengthmedium — Large, high-quality observational dataset, pre-registration, held-out replication, and many robustness checks provide credible descriptive evidence that the random-impact rule and Q-model decomposition apply to open-source package careers; however, the analysis is observational with potential confounders (e.g., simultaneous trends, platform changes, measurement gaps, selection into package-publishing) that limit causal inference about the effect of generative AI.
Methods Rigorhigh — The study uses a pre-registered analysis plan, a discovery/held-out split for confirmation, many robustness specifications, bootstrap confidence intervals, and explicit checks for measurement pitfalls (credit rules, horizon choices, interpolation of dependency snapshots, identity-resolution validation). The main limitations are inherent to observational data (unobserved confounding, reliance on dependency-based impact) rather than sloppy design.
SampleComplete GitHub push-event record (GitHub Archive) Jan 2015–Sep 2025 producing 102.2M works by 6.15M contributors; estimable subsample for package-impact analysis: 2.44M works by 908,417 contributors across ~1.82M repositories; downstream dependency data from Libraries.io (snapshots to Jan 2020) and Ecosyste.ms (from Aug 2022 onward) with an interpolated bridge for 2020–Aug 2022; identity resolution via commit-hash dictionary plus an 80% push-share backup and bot filters (precision ~1.00 on 2015–19 gold set, ≥0.98 in later eras), resolving 60.6%/71.8%/74.7% of commits in 2015–19, 2020–22, 2023–25 respectively.
Themeshuman_ai_collab labor_markets
IdentificationPre-registered, within-person pre/post comparison around the generative-AI rollout (ChatGPT/Copilot) using a 50/50 discovery/held-out split; controls for career aging and right-censoring of recent works; robustness via 54 specification grid (credit rules, horizons, career-length floors), bootstrap CIs, and multiple-testing correction (Benjamini–Hochberg). No randomized assignment or instrumental variable is used, so causal claims rely on before/after contrasts and robustness checks rather than a clean exogenous shock.
GeneralizabilityRestricted to contributors whose repositories publish packages (infrastructure-focused subset of open-source authors)., Impact measured via package dependency counts (reach within dependency graph), which may not capture other forms of impact (downloads, corporate use, reputation)., Platform-specific (GitHub) — findings may not generalize to private repositories, other hosting platforms, or non-package-based software workflows., Dependency snapshot coverage has a gap (Jan 2020–Aug 2022) that was interpolated, potentially affecting temporal comparisons., Identity-resolution leaves a substantive fraction of commits unassigned (coverage rather than correctness), which may bias toward more visible contributors and undercount others., Ecosystem heterogeneity (npm vs Cargo vs RubyGems) means absolute impact scales differ across communities despite normalization., Observational pre/post comparisons around generative-AI rollout are vulnerable to concurrent platform or ecosystem changes not fully controlled for.
Claims (7)
| Claim | Direction | Outcome | Confidence & Evidence | Details |
|---|---|---|---|---|
| The timing of a contributor's highest-impact work is close to a lottery over the works in their career, with a small and replicable tendency for the highest-impact work to occur early in the career; this early tendency grows as careers become longer. Innovation Output | mixed | Relative career position of the contributor's highest-impact package-related work |
Reading fidelity
high
Study strength
medium
|
n=14091
small, replicable lean toward early career
|
| A stable, person-specific factor explains only part of the persistence in contributors' work impact—about one-fifth in the primary specification—while the remainder behaves like temporary momentum that fades over time. Innovation Output | positive | Persistence and cross-work correlation of a contributor's package impact |
Reading fidelity
high
Study strength
medium
|
n=14091
about a fifth
|
| Within the same contributors, the luck-versus-persistent-factor structure of package-career impact did not change detectably after the arrival of generative-AI coding tools. Innovation Output | null_result | Change in the decomposition of career impact into luck, stable individual factor, and momentum |
Reading fidelity
high
Study strength
medium
|
shift of +0.03 (95% CI [-0.22, +0.23]) after subtracting the effect of aging
|
| The study reconstructs 102.2 million career works produced by 6.15 million GitHub contributors from push events between 2015 and 2025. Other | positive | Number of reconstructed open-source career works and contributors |
Reading fidelity
high
Study strength
medium
|
n=6150000
102.2M works by 6.15M contributors
|
| Among contributors whose repositories publish packages, the study measures impact for 2.44 million works produced by 908,417 contributors across 1.82 million repositories. Innovation Output | positive | Downstream package impact of open-source works |
Reading fidelity
high
Study strength
medium
|
n=908417
2.44M works by 908,417 contributors across 1.82M repositories
|
| Most package-related works have no measurable downstream impact: 63.0% reach zero downstream projects even at the 60-month horizon. Innovation Output | negative | Whether an open-source work obtains downstream package dependents |
Reading fidelity
high
Study strength
medium
|
n=2440000
63.0% reach zero downstream projects at W=60
|
| The contributor identity-resolution procedure has very high precision: precision is 1.000 for the 2015–2019 validation period and at least 0.98 in every later era and region after bot filtering. Other | positive | Accuracy of GitHub contributor identity assignments |
Reading fidelity
high
Study strength
high
|
n=28896
precision 1.000 for 2015–19; ≥0.98 in every later era and region
|