ChatGPT did not meaningfully reduce Reddit help-seeking: informational subreddits saw no post-volume decline and AI-generated posts rose only a few percentage points—insufficient to account for earlier reports of large drops, which appear driven in part by pre-existing platform drift.

Main Finding

Informational help-seeking on Reddit did not decline after ChatGPT’s launch. Using contemporaneous hobby subreddits as controls and multiple robustness checks (including 66 placebo “launch” dates before ChatGPT), the authors estimate a 5.1% increase in informational posts and rule out any true decline larger than 3.4%. Detectors find only a small relative rise in LLM-like post text (≈2–3 percentage points) in informational communities versus controls, and no detectable rise in LLM-like content among comments (answers).

Key Points

• Sample: 151 Reddit communities meeting volume and age filters, classified into:
  • I (informational advice): 26 subreddits
  • H (human-anchored support): 29 subreddits
  • Q (low-stakes curiosity): 6 subreddits
  • C (controls — hobby communities): 90 subreddits
• Primary comparison window: Dec 2021–May 2022 (pre) vs Dec 2022–May 2023 (post), matching calendar months to remove seasonality.
• Main estimation: two-way fixed-effects difference-in-differences on log(1 + monthly posts), with community and month fixed effects, cluster-robust SEs and wild-cluster bootstrap p-values.
• Main result: informational communities showed a +5.1% change relative to hobby controls after ChatGPT; confidence bounds exclude declines >3.4% (i.e., a meaningful decline of the magnitudes reported in many prior studies is ruled out).
• Detector analysis:
  • Scored 274,411 posts and 223,775 comments with Fast-DetectGPT; added Binoculars for comments.
  • Calibrated detector responses using four LLMs (GPT-4o, Claude Sonnet 4, DeepSeek-V3, and GLM-5.2) to convert score shifts into implied shares of machine-generated text.
  • Result: AI-like posts rose only ~2–3 percentage points more in informational communities than in hobby controls (insufficient to conceal the previously reported declines); comments showed no relative increase in AI-like content.
• Placebo tests: rerunning the entire analysis at 66 pre-ChatGPT placebo dates produced nonzero “effects” as large as many published estimates, demonstrating that pre-existing drift between community types can mimic an AI impact if contemporaneous controls are not used.
• A notable exception: low-stakes curiosity communities (Q) show an 18.3% decline relative to controls after the launch (p = 0.002), but identical pre-existing trends explain this decline (they were shrinking ~18%/year before ChatGPT), so the decline is not attributable cleanly to ChatGPT.

Data & Methods

• Data sources:
  • Monthly post counts per subreddit from Arctic Shift (Pushshift successor) for the count-based DID analysis.
  • Text samples for detector analysis drawn from Arctic Shift (up to 100 posts and 100 comments per community-month), two windows: Jan 2021–Nov 2022 (pre-ChatGPT baseline) and Jan 2024–Nov 2025 (post period).
• Regression specification:
  • log(1 + y_st) = α_s + γ_t + Σ_g δ_g · 1[s ∈ g]·Post_t + ε_st
  • δ_g are group-by-post indicators (difference-in-differences relative to control group C).
  • Community fixed effects (α_s) and calendar-month fixed effects (γ_t) control for time-invariant community differences and platform-wide shocks.
  • SEs clustered by community; wild-cluster bootstrap used for p-values.
• Power & sensitivity:
  • Noise dominated by slow community-level drift; with 26 treated informational communities the detectable decline at 80% power was ~14% (5% α) for the primary window. Despite this, empirical confidence intervals still exclude declines >3.4%.
• Detector approach:
  • Used Fast-DetectGPT (curvature-based scoring using GPT-Neo-1.3B) as the primary detector; Binoculars added for comments.
  • Outcome for detector DID: change in mean detector score per community-period, which cancels time-invariant detector biases (e.g., against non-native English) because the comparison is relative to controls and baseline period.
  • Calibration: sampled real pre-ChatGPT posts and had four LLMs generate matching documents; compared detector responses to translate score shifts into plausible percentage-point increases in LLM-generated documents.
• Robustness & diagnostics:
  • 66 placebo dates (same calendar-month design) to quantify pre-existing drift and estimate false positives the design would produce in the absence of ChatGPT.
  • Longer window (2021–22 vs 2024–25) also reported but overlaps other platform events (e.g., Google–Reddit licensing, Google AI Overviews), so treated cautiously.

Implications for AI Economics

• Substitution magnitude: On Reddit, generative AI (ChatGPT-era models) does not appear to have substantially substituted away public informational help-seeking at the community level. Any displacement in posted questions would have to be smaller than a few percent on average, at least in the six-month post-launch window studied.
• Complementarity and persistence of human-mediated help: The stability of post volumes and the small relative rise in LLM-like posts suggest that people continue to ask humans in public forums, and — as detectable — humans still provide most answers (comments). This points to persistent complementarity or frictions to substitution (trust, norms, desire for human validation, or privacy preferences).
• Methodological lesson for AI-economics research: contemporaneous controls that are plausibly unaffected (or less affected) by the technology are crucial. Pre-existing trends between groups can generate spurious large estimated effects if not accounted for; placebo-date diagnostics are a practical tool to reveal such drift.
• Measurement of AI content: detector-based, score-differencing approaches calibrated on platform-relevant text and multiple LLMs provide a defensible, bias-cancelling way to measure relative change in LLM-like content without binary labeling. Still, detector limitations (bias vs non-native English, document-length sensitivity) mean conclusions about exact shares should be cautious.
• Policy and market forecasting:
  • Forecasts of labor or platform disruption from LLMs should allow for limited immediate displacement in low-cost, public Q&A contexts; large-scale substitution may require overcoming additional behavioral, privacy, or trust barriers (or will show up more in private product usage rather than public posts).
  • Firms and platforms whose business models assume rapid erosion of user-generated informational content should reassess timing and magnitude assumptions; replacement dynamics may be heterogeneous across domains and platforms (e.g., differences between Reddit and Stack Overflow reported in other studies).
• Directions for further research:
  • Account-level analyses (with careful privacy protections) to track whether individual human posters are replaced by LLM-written posts.
  • Cross-platform studies (e.g., Stack Overflow, private chat apps, search engines) to map where substitution is stronger.
  • Quality, usefulness, and welfare impacts of LLM-generated answers in community settings (not just volume).
  • Longer-term studies that account for later adoption phases and concurrent platform changes (search- and licensing-driven shifts).

Limitations to keep in mind: detector imperfectness (and known biases), inability to follow individual accounts for privacy reasons (limits tests of who left and who replaced them), small treated sample for Q (n=6), and later-window confounding events (Google–Reddit deals). Overall, the study provides evidence that public, community-based informational help-seeking on Reddit remained largely resilient to ChatGPT in the early adoption period.