0 cumulative citations
View corpus contextAI-generated review summaries steer shoppers’ opinions and buying intent, especially when negative; whichever source appears first anchors judgment, but user reviews are judged more helpful and are more likely to displace an existing anchor.
First Impressions: How Placement Shapes the Influence of AI Summaries
A preregistered randomized experiment (N=278) shows AI-generated review summaries shift shoppers' attitudes and purchase intent in the direction of the summary's valence—negative summaries have larger effects—and placement matters: the first source anchors judgments while user reviews are better at overturning existing anchors.
Citation observations
Cumulative provider counts captured on specific dates; providers are never combined.
AI-generated summaries increasingly mediate how people interpret information across platforms, including product reviews on e-commerce sites. Using Amazon's AI summaries as a case study, we conducted a preregistered, randomized experiment (N = 278) comparing how AI summaries and user reviews shaped product perceptions, and how their influence varied with valence and presentation order. We found that both AI summaries and user reviews influenced participants' opinions, with negative summaries having a larger effect than positive ones. Presentation order was the most important factor: the first source anchored judgment and only user reviews could displace an existing anchor. Although participants reported preferring user reviews, they often underestimated the influence of AI summaries on their judgments. Our findings show how the placement of AI summaries shapes user perception and highlight opportunities to design interfaces that support more deliberate judgments about when to rely on summaries and when to examine the underlying content directly.
Summary
Main Finding
AI-generated review summaries materially shift consumer perceptions and purchase intent, but their influence depends strongly on valence and placement. Negative AI summaries exert larger effects than positive ones. Presentation order is critical: the information source encountered first anchors judgments, and user-written reviews are better able than AI summaries to displace an existing anchor. Participants say they prefer user reviews, yet often underreport how much AI summaries affected their judgments.
Key Points
- Both AI summaries and user reviews move attitudes and purchase intent in the direction of their valence.
- Negative AI summaries produce larger changes in attitudes/purchase intent than positive AI summaries (negativity bias).
- Overall, AI summaries are not more influential than user reviews. When AI summaries and user reviews conflict, participants align more with user reviews.
- Order effects (anchoring): whichever source a participant saw first had disproportionate influence on subsequent judgments. AI summaries were more influential when shown before reviews; user reviews were influential whether shown first or second and could displace earlier anchors more readily.
- Self-reports diverge from behavior: participants rated user reviews as more helpful, yet AI summaries sometimes drove larger shifts—indicating unrecognized reliance on AI summaries.
- Study context: e-commerce search products (camera, dog leash, changing table, garbage bags), decisions framed as buying for a friend to minimize personal preference biases.
Data & Methods
- Design: Preregistered 2x2x2 mixed factorial experiment (N = 278 analyzed). Manipulated (1) order of presentation (AI-first vs review-first), (2) AI-summary valence (positive vs negative), (3) user-review valence (positive vs negative). Each participant evaluated four products in three stages.
- Stimuli: Real Amazon product descriptions, user reviews, and Amazon AI overviews gathered Apr–May 2025. AI summaries were perturbed from existing overviews to create matched positive vs negative versions (same structure, flipped valence wording) so comparisons isolated valence effects.
- Procedure: For each product participants saw (Stage 1) product description, (Stage 2) either AI summary or user reviews, (Stage 3) the other source. After each stage they reported (on 5-point Likert scales) product attitude and purchase intent. After Stage 3 participants ranked helpfulness of description, AI summary, and reviews and answered exit questions.
- Recruitment: US-based, English-fluent adults on Prolific. Power analysis targeted ~277; 290 surveys run, 12 excluded, final N=278. Attention checks included. Compensation: $2.50 for ~10 min.
- Analysis: Compared within-subject shifts across stages; regression models assessed effects of valence, order, and conflicts between summaries and reviews. Robustness checks and pilot studies (5 rounds) informed design.
Implications for AI Economics
- Demand and conversion sensitivity: AI summaries (particularly negative ones) can shift consumer demand—platforms’ use and placement of summaries may meaningfully change conversion rates and product-level sales. Economists modeling demand should treat AI-generated summaries and their placement as potential exogenous shocks to perceived product quality.
- Market power and platform design: Platforms control where summaries appear. Placement choices (AI-first vs reviews-first) can systematically advantage or disadvantage sellers. Platforms can therefore influence market outcomes beyond traditional ranking/price mechanisms.
- Information asymmetry and welfare concerns: Collapsing heterogeneous user reviews into a single AI narrative reduces informational nuance. Negativity amplification and anchoring risks may yield suboptimal consumer choices and distributional effects (e.g., harming new entrants or niche products).
- Potential for strategic manipulation and externalities: Because summaries can anchor users and users under-recognize AI influence, there is scope for biased generation (intentional or inadvertent) to steer demand—raising concerns about fairness, transparency, and competition. Sponsored content or optimization objectives could interact with summary generation to distort markets.
- Policy and platform remedies: To preserve consumer welfare and market integrity, platforms should consider:
- Interface design that mitigates undue anchoring (e.g., prompting users to view a representative sample of reviews, delaying AI summary prominence, or visually linking summaries to supporting reviews).
- Transparency labels explaining how AI summaries are generated and what they summarize.
- Providing simple affordances to jump to underlying reviews or to compare summary claims with sampled reviews.
- Auditing summaries for valence bias and heterogeneous effects across product types and seller groups.
- Research agenda: Quantify downstream sales impacts (field experiments with click/purchase outcomes), heterogeneity across consumer segments, long-term effects on reputation and reviews, and interaction with advertising/sponsored listings. Incorporate AI-summary placement into structural models of platform competition and consumer search costs.
Assessment
Paper Typerct
Evidence Strengthmedium — Strong internal validity from preregistered randomized assignment and within-subject pre/post measures yields credible causal estimates of how staged AI summaries and reviews shift self-reported attitudes and purchase intent; however outcomes are self-reported intentions (not real purchase behavior), stimuli were limited to four search products and manually perturbed summaries, and the sample is an online convenience (Prolific) restricted to US English speakers, limiting external validity.
Methods Rigorhigh — Preregistered RCT with power analysis, attention checks, real-platform stimuli, balanced 2x2x2 design, and stagewise repeated measurement provides rigorous identification and clean tests of anchoring and valence effects; remaining concerns include manual perturbation of summaries (possible subtle manipulation artifacts), reliance on hypothetical buying-for-a-friend framing and Likert self-reports, and a moderate N drawn from Prolific.
SampleN = 278 U.S.-based, English-fluent adults recruited on Prolific (after exclusions), compensated $2.50; participants evaluated four Amazon 'search' products (camera, dog leash, changing table, garbage bags) presented with real product descriptions, user reviews, and AI overviews (AI overviews were manually perturbed to create positive/negative valence); data collected April–May 2025; preregistered study with two attention checks.
Themeshuman_ai_collab adoption
IdentificationRandom assignment of participants to (1) information order (AI-summary first vs user-review first), (2) AI-summary valence (positive vs negative), and (3) user-review valence, combined with within-subject repeated measures across three stages (product description → first source → second source), so differences in attitudes/purchase intent can be causally attributed to the randomized presentation and valence manipulations.
GeneralizabilityProlific convenience sample (U.S.-only, likely not nationally representative), Outcomes are self-reported attitudes and purchase intent, not observed purchases or long-term behavior, Only four search-type products examined—may not generalize to experience goods or higher-stakes purchases, AI summaries were manually valence-perturbed rather than generated by multiple production LLMs, which may not capture real-world variance in summary quality or errors, Experimental 'buy for a friend' framing may differ from real personal purchase decisions or browsing contexts, English-language materials only
Claims (10)
| Claim | Direction | Outcome | Confidence & Evidence | Details |
|---|---|---|---|---|
| Both AI-generated summaries and user-generated reviews influenced participants' product opinions and purchase-related judgments. Consumer Welfare | positive | Changes in product attitudes and purchase intent after exposure to AI summaries and user reviews. |
Reading fidelity
high
Study strength
high
|
n=278
|
| Negative AI summaries produced larger shifts in participants’ attitudes and purchase intent than positive AI summaries. Consumer Welfare | negative | Magnitude of changes in product attitudes and purchase intent following exposure to AI summaries of different valence. |
Reading fidelity
high
Study strength
high
|
n=278
|
| AI summaries were not more influential than user reviews overall. Consumer Welfare | null_result | Relative influence of AI summaries versus user reviews on product attitudes and purchase intent. |
Reading fidelity
high
Study strength
high
|
n=278
|
| When AI summaries and user reviews conflicted, participants aligned more strongly with the user reviews. Consumer Welfare | positive | Alignment of final product attitudes and purchase intent with AI summaries versus user reviews under source conflict. |
Reading fidelity
high
Study strength
high
|
n=278
|
| Presentation order affected the influence of the information sources: AI summaries were more influential when presented before user reviews. Consumer Welfare | positive | Influence of AI summaries on product attitudes and purchase intent as a function of whether the summary was presented first. |
Reading fidelity
high
Study strength
high
|
n=278
|
| User reviews were similarly influential whether they were presented before or after the AI summary. Consumer Welfare | null_result | Influence of user reviews on product attitudes and purchase intent across presentation-order conditions. |
Reading fidelity
high
Study strength
high
|
n=278
|
| The first information source encountered anchored participants’ judgments, and user reviews were the only source that could displace an existing anchor. Decision Quality | mixed | Persistence and displacement of initial product judgments after exposure to a second information source. |
Reading fidelity
high
Study strength
high
|
n=278
|
| Participants consistently rated user reviews as more helpful than AI summaries. Consumer Welfare | positive | Participants’ perceived helpfulness of user reviews versus AI summaries for making product decisions. |
Reading fidelity
high
Study strength
medium
|
n=278
|
| Participants sometimes underestimated the influence that AI summaries had on their judgments. Decision Quality | negative | Discrepancy between the actual influence of AI summaries on judgments and participants’ perceived influence of those summaries. |
Reading fidelity
high
Study strength
medium
|
n=278
|
| The study used a preregistered randomized experiment with 278 participants recruited from Prolific. Consumer Welfare | other | Experimental comparison of AI-summary and user-review effects on product judgments. |
Reading fidelity
high
Study strength
high
|
n=278
N = 278
|