Symbolic awards on Reddit rarely boost activity or influence—and random lottery-style awards from bots can lower engagement—yet they do nudge more direct replies between users; simple algorithmic nudges appear ineffective at manipulating behavior, highlighting the need for transparent labeling and scrutiny of more sophisticated automated agents.

Main Finding

A large field experiment on Reddit shows that simple symbolic interventions by apparent bots and humans (non-anonymous $1 “Diamond” awards plus a short 100-character rationale) rarely increase recipients’ subsequent activity, content positivity, or downstream impact. Awards do, however, elicit more direct private replies when sent by a human. Awards from a clearly labeled bot that framed the award as a random “lottery” can backfire—reducing subsequent posting volume and positive tone. One notable positive effect: awards from a human account framed as moral praise increased votes and comment length on recipients’ subsequent contributions.

Key Points

• Design: 2,442 treated contributions randomly assigned to 9 conditions (8 treatments: bot vs human × four rationales — logical (LOG), random/lottery (RAN), emotional (EMO), moral (MOR) — plus a no-intervention control).
• Treatments: a visible Diamond award (≈$1) plus a 100-character message: identity hint (“Hi, Reddit stranger” vs “I’m a bot, beep boop”) and one of four rationales.
• Main behavioral outcomes measured as within-user changes from the week before to up to 12 hours after treatment: activity volume (number of contributions, total words), content (LIWC positive tone, analytical thinking, emotion; moral language via moral foundations dictionary), and indirect impact (votes, comments, replies to treated users’ new contributions). Private thank-you messages to the awarding account were also recorded.
• Principal results:
  • No treatment produced consistent increases in posting volume or overall downstream impact versus control.
  • Recipients were significantly more likely to send private thanks to human accounts than to bot accounts.
  • Bot + random/lottery rationale (BRAN) produced significant decreases (vs control and several other treatments) in number of subsequent contributions, total words, and positive tone (i.e., a backfire).
  • Human + moral rationale (HMOR) increased votes received and the length of replies to the recipient’s subsequent contributions (HMOR > control, HLOG, HRAN).
  • Linguistic/content measures mostly null across treatments; significant language analyses were run on a reduced subsample (users with ≥50 words before/after; N ≈ 545).
• Statistical approach: pre-registered experiment, comparison of within-person before/after differences; primary tests used Mann–Whitney U (non-parametric). Reported p-values are uncorrected (authors note this and discuss strength of findings).
• Data collection: Reddit API, April 2025–Jan 2026. Ethical approval obtained; accounts were transparently labeled (bot username included “bot”).

Data & Methods

• Sample selection:
  • 12 non-sensitive, general-interest subreddits selected from top listings.
  • Contributions screened and filtered: recent (≤12 hours), limited prior engagement (1–5 comments and 2–6 upvotes for posts; 2–21 upvotes for comments), no downvotes, length 300–3000 characters.
  • Additional filtering for sensitive content: dictionary filter + GPT-4 screening.
  • Targeted relatively inexperienced users (account ≥30 days old, 10–499 comments, 0–99 posts).
  • Final N = 2,442 treated contributions (≈263–275 per condition).
• Intervention:
  • Two research accounts presented as a typical human and as a clear bot (username, avatar, profile text).
  • Non-anonymous Diamond award plus message: “[Identity hint]. [Rationale]. Keep it up!”
  • Rationale variants: logic (“I like your clear reasoning and logic”), random (“your post won today’s lottery”), emotion (“I like your emotional tone and empathy”), moral (“I like your moral intuition and integrity”).
• Outcomes/Measures:
  • Activity: number of new contributions and total text length after treatment.
  • Content: LIWC-22 Positive Tone, Analytical Thinking, Emotion metrics; moral word frequency via moral foundations dictionary.
  • Impact (indirect): estimated votes (post formula using upvote ratio; comments using |u|), number and total length of comments/replies to the treated user’s subsequent contributions.
  • Private messages to awarding account counted separately.
• Analysis details:
  • Use of within-individual differences ΔM = M_after − M_before (and log-ratio visualizations).
  • Main tests evaluated effects in a 12-hour post-treatment window (also visualized up to 5 days in SI).
  • Mann–Whitney U tests for between-condition comparisons. Pre-registration available; multiple-test correction planned but uncorrected p-values reported due to null/backfire patterns.
  • Linguistic tests restricted to users with ≥50 words pre- and post-treatment (N ≈ 545).

Implications for AI Economics

• Weak direct influence of simple automated actions at the individual level:
  • Symbolic, low-complexity automated actions (a labeled bot awarding and sending a short rationale) do not reliably raise activity or produce positive cascades. Models of platform-level influence and economic externalities should not assume large per-interaction effects from simple bot actions.
• Heterogeneity matters:
  • Effects depend on both the apparent agent (human vs bot) and framing (moral/logical/random/emotional). Economic models of information diffusion or behavior adoption should include agent-type and message-framing heterogeneity.
• Scaling and network effects remain a policy risk:
  • Small per-interaction effects can still matter if bots are numerous and persistently targeted at network leverage points. Even weak influence can cascade if concentrated on central nodes — incorporate scale and strategic placement in policy and economic impact assessments.
• Transparency/labeling reduces simple bot influence:
  • A labeled bot elicited fewer private replies and in one framing even a backfire; this supports policies requiring transparent identification of automated accounts to reduce manipulative influence and informational asymmetry.
• Message framing and perceived intent affect efficacy and valuation:
  • Moral praise from a human increased downstream engagement metrics (votes, reply length), whereas an explicit “lottery” explanation from a bot reduced activity. Economists designing incentive systems on platforms should account for perceived provenance and motive—randomized rewards from opaque or machine identities can devalue or depress participation.
• Methodological takeaways for AI economics research:
  • Field experiments with pre-registered treatments and within-user designs are valuable for causal inference in platform settings.
  • Use of linguistic and behavioral micro-measures (e.g., LIWC, moral dictionaries) is useful but sensitive to sample size—be cautious interpreting nulls from smaller subsamples.
• Policy and governance recommendations:
  • Maintain and enforce transparent labeling of automated agents; require disclosure when algorithms/agents interact with users.
  • Monitor and regulate large-scale automated reward/engagement campaigns that could exploit network externalities even when individual effects are weak.
  • Prioritize research into more sophisticated conversational agents — this experiment suggests resilience to simple manipulations, but more human-like AI may produce larger effects, so forward-looking regulation and experiments are needed.
• Directions for future economic work:
  • Quantify how weak per-interaction effects aggregate under different network structures and bot deployment strategies.
  • Study effects of unlabeled or human-imitating bots (ethical constraints permitting) to estimate upper bounds on manipulation potential.
  • Incorporate agent sophistication, message richness, and temporal persistence into models of contagion, platform competition, and welfare impacts.

Limitations to bear in mind: findings apply to one platform (Reddit), a specific intervention (visible $1 award + short message), a primarily short (12-hour) post-treatment window, and clearly labeled bot accounts. Linguistic analyses were underpowered and p-values reported uncorrected. Replication across platforms, longer horizons, and with different bot sophistication levels is needed to generalize.