Larger reported GPU setups slightly increase a paper's chance of being highly cited, but compute is far more concentrated than influence: the top 20% of GPU-quantifiable NLP papers account for ~84–90% of reported GPU capability yet only ~27–32% of citations and ~20–33% of awards, and a tenfold rise in reported capability raises within-topic citation percentile by only ~3.5 points while adding almost no explanatory power.

Main Finding

Reported GPU capacity in leading NLP conference papers is positively associated with scholarly impact (citations and awards) but only weakly explains variation in impact. A tenfold increase in reported GPU capability predicts a modest 3.52 percentage-point gain in within-topic–year citation percentile while adding only 0.0042 to model R2. Compute resources are highly concentrated in a small share of papers, but that concentration substantially exceeds the concentration of citations and awards — i.e., more compute makes high impact more likely but is neither necessary nor sufficient.

Key Points

• Sample and coverage

  • Corpus: 13,921 ACL, EMNLP, and NAACL main-conference papers (2020–2025).
  • Model-reported sample: 6,900 papers (49.6%) report at least one standardizable GPU model.
  • Strict sample: 5,360 papers (38.5%) report both standardized GPU model and explicit GPU count (used for capacity regressions).

• Reporting trends and hardware evolution

  • Reporting completeness rose: model reporting increased from ~30% (2020) to 57% (2025); model+count from ~15% to 49%.
  • Median reported paper-level peak GPU capacity rose from ≈91 TFLOP/s (2020) to ≈1,248 TFLOP/s (2025).
  • Growth driven mainly by adoption of newer generations (V100 → A100 → H100 appearing 2024–25) and medium-scale multi-GPU configs (3–8 GPUs); very large clusters remained uncommon.

• Concentration vs. impact

  • Top 20% of GPU-quantifiable papers accounted for 83.9%–89.9% of reported GPU capability (2020–2023).
  • The same top-20% accounted for only 27%–32% of citations and 20%–33% of paper awards.
  • High-capability papers are more likely to be highly cited (14.5% of high-capability papers are in the top 10% by citations vs 9.1% for others; ~1.59× higher), but most high-capacity papers are not highly cited and most highly cited papers are outside the high-capacity group.
  • Robustness across thresholds: high-capability papers exhibit 1.49–2.14× higher high-impact rates depending on cutoff choices.

• Regression / association results

  • Primary outcome: within-NLP topic–year citation percentile (citation models restricted to 2020–2023, N = 2,194).
  • A tenfold increase in aggregate reported GPU capability → +3.52 percentage points in citation percentile; R2 increase is minimal (+0.0042).
  • GPU count showed more robust and consistent positive associations with citation and award outcomes than newer hardware generation.
  • Aggregate capability and hardware generation showed weaker, less consistent associations with awards.

Data & Methods

• Data sources and labeling

  • Papers scraped from ACL Anthology (ACL/EMNLP/NAACL 2020–2025); bibliographic metadata from OpenAlex; topic labels by GPT-4o-mini.
  • Full-text parsing via MinerU.

• GPU extraction and normalization

  • Manual annotation: 400-paper human-validated evaluation set; inter-annotator reliability high (Cohen’s κ = 0.94); exact-match rates ~91% (model) and 87.5% (count).
  • LLM extraction: DeepSeek-v3.2 achieved GPU-name F1 = 0.933 and exact model+count F1 = 0.879 on evaluation set; pipeline applied to full corpus.
  • Normalization to canonical GPU specs using a catalog (Epoch AI Machine Learning Hardware dataset, vendor docs). Each GPU mapped to memory, family, generation, and theoretical peak Tensor FP16/BF16 throughput.

• Capacity metric

  • Paper-level reported GPU capacity = max over reported configs of (GPU_count × per-GPU theoretical peak TFLOP/s for FP16/BF16).
  • For papers missing an explicit count but reporting a model, count conservatively set to 1 in the model-reported sample.
  • This captures reported hardware capability (theoretical peak), not actual compute used (no GPU hours, FLOPs, cost, energy, or utilization).

• Statistical models and controls

  • Main regressions include publication-year-by-venue fixed effects, primary-topic fixed effects (29-category taxonomy), team-size and organization-count controls.
  • Citation analyses restricted to 2020–2023 to reduce truncation; awards analyzed across available years (N ≈ 5,357 for awards).
  • Complementary outcomes: OpenAlex field-normalized percentiles, raw citations, binary high-citation status, and paper awards.
  • Robustness checks: alternative cutoffs, decomposition into GPU count vs hardware generation, author/team/institutional controls.

• Reproducibility

  • Code and data made available: https://github.com/ChenShuai00/Computational-Resources

• Key limitations of measurement

  • Relies on reported and standardizable GPU mentions in papers; missingness is nontrivial and selection may bias results.
  • Capacity uses theoretical peak TFLOP/s per card and the largest reported config — does not measure actual training compute (hours/FLOPs), utilization, cost, or other compute-related inputs.
  • LLM extraction, normalization, and conservative assumptions (e.g., default count=1) can under- or mis-estimate real usage.

Implications for AI Economics

• Inequality in compute access is pronounced but does not map one-to-one to scholarly influence

  • Compute resources are heavily concentrated in a small subset of papers and often in industry-involved or industry–academic collaborations.
  • However, the disproportionate concentration of compute yields only modest concentration in citations/awards — compute amplifies but does not determine scholarly influence.

• Marginal returns to reported hardware capability appear low

  • A tenfold increase in reported GPU capability produces only small average gains in citation percentile and explains little additional variance (tiny R2 change). For funders and institutions, investing solely in larger GPU allocations may have limited aggregate impact on research visibility per dollar unless paired with other factors (ideas, data, teams, dissemination).

• Policy and distributional considerations

  • Improving equitable access to compute remains important (to enable participation in compute-intensive work), but lowering barriers to non-compute avenues (methodological innovation, data sharing, tooling, evaluation) may be equally or more cost-effective for increasing research impact diversity.
  • Transparency standards: incomplete reporting of compute hampers measurement and policy. Mandating clearer reporting (models, counts, hours, FLOPs, cost) would improve research-evaluation and allocation decisions.
  • Funding design: grants that combine compute credits with support for model/data release, reproducibility, and dissemination may yield higher social return than compute-only provisioning.

• Design of incentive structures and evaluations

  • Evaluation metrics and hiring/promotion criteria that overweight hardware-intensive outputs risk reinforcing compute concentration without commensurate increases in influence.
  • Because GPU count (scale-out) showed more consistent associations than hardware generation (scale-up to newest chips), institutions choosing between buying many older GPUs vs fewer newest GPUs should weigh task-specific returns and utilization patterns.

• Directions for AI-economics research

  • Measure realized compute: incorporate GPU hours, training FLOPs, utilization, and costs to better estimate the production function of research output and impact.
  • Causal identification: exploit exogenous variation in compute access (e.g., sudden grants, cloud credits, or hardware outages) to estimate causal returns to compute.
  • Cross-domain comparisons: compare NLP to other AI subfields and to non-AI disciplines to assess generality of compute–impact relationships.
  • Cost-effectiveness analysis: quantify impact per dollar of compute to guide funders and institutions.

Summary takeaway: reported GPU resources matter but explain little alone. Policies that address compute inequality should be coupled with interventions targeting other drivers of research influence (data, teams, openness, reproducibility) and should push for better compute reporting to enable more accurate economic analyses.