Tailored, group-conditioned ad image generation raises click-through rates across heterogeneous user segments; a new 600K-group dataset from 40M users shows group-specific creatives outperform one-size-fits-all optimization.

Main Finding

The paper introduces OSMF (One Size, Many Fits), a unified framework that improves advertising image generation by aligning generated creatives with diverse group-wise click preferences. Key components—Product-Aware Adaptive Grouping (PAAG), a Group-aware Multimodal LLM (G-MLLM) with preference-conditioned image generation (PCIG), and Group-DPO fine-tuning—produce group-tailored image prompts and generated images that increase per-group CTR and outperform prior methods on both offline metrics and online deployment. The authors also release GAIP, the first large-scale public dataset of group-wise advertising image preferences (~600K groups from ~40M users).

Key Points

• Problem: Existing CTR-driven image generation optimizes aggregated CTR (one-size-fits-all), ignoring heterogeneous preferences across user groups and producing suboptimal creatives for many segments.
• PAAG: Dynamically forms product-aware user groups by combining user attributes, product title and image features; clusters product-specific user representations (K chosen via silhouette score) and represents each group with a centroid plus percentile-sampled peripheral points to capture intra-group diversity.
• G-MLLM & PCIG: A single multimodal LLM (based on LLaVA) is augmented with a group token prepended to inputs to condition generation on group embeddings. Pretraining uses group-centric (group analysis, behavioral prediction) and product-centric (product comprehension, prompt generation) tasks to learn group-aware image-prompting.
• Group-wise preference alignment: A Group-aware Reward Model (GRM), initialized from G-MLLM, is trained on pairwise image CTR rankings to produce group-conditioned preferences. G-MLLM is then fine-tuned with Group-DPO (an adaptation of Direct Preference Optimization) using GRM signals; LoRA is used during fine-tuning for efficiency.
• Generation pipeline: Text prompts from G-MLLM are rendered into advertising images via Stable Diffusion + ControlNet-Inpaint; GRM evaluates group-specific preferences to form training pairs.
• GAIP dataset: Constructed from industrial logs covering ~40M users, ~2.1M products, ~9.6M advertising images; after filtering low-exposure products the dataset contains ~610K user groups with per-group CTRs for images and product titles.
• Empirical gains: Outperforms baselines on preference modeling metrics (example: NDCG@5 improved to 0.3066 and AUROC to 0.6372 vs baselines ~0.312–0.319 and ~0.618–0.621). Authors report state-of-the-art offline and online performance (details in paper).
• Practical considerations: Limits groups per product (≤5) for computational tractability; training uses ResNet and CLIP encoders, LLaVA backbone; pretraining and fine-tuning performed on 8× NVIDIA H100s with LoRA to reduce cost.

Data & Methods

• Data:
  • Source: 3 weeks of industrial advertising interaction logs from a major e-commerce platform.
  • Size: 40,027,535 users; 2,085,969 products; 9,565,154 advertising images; final GAIP contains ~610,172 groups.
  • GAIP items: quadruples (advertising image Ii_s, title Ts, group embedding Gs,k, CTR(Ii_s, k)).
• PAAG (Product-Aware Adaptive Grouping):
  • Inputs: user attributes (age, gender, location, purchasing power, etc.), product title embedding (ET), and image embedding (EI).
  • Architecture: user attribute embeddings → MLP → cross-attention with product title and image to obtain product-conditioned user embedding eu|c.
  • Clustering: K-means on product-conditioned embeddings for users who viewed product s; choose K by silhouette score; group embedding Gs,k = {centroid} ∪ peripheral samples at selected percentiles (e.g., 15th, 55th, 95th).
• G-MLLM:
  • Input augmentation: prepend encoded group token eGs,k to multimodal input sequence [group; visual tokens; text tokens].
  • Pretraining tasks: group analysis (describe prototypical user in group), behavioral prediction (predict next clickers), product comprehension (predict title from image), prompt generation (produce image prompts conditioned on product/group).
  • Backbone: LLaVA; image and text encoders for PAAG use ResNet and CLIP.
• Group-aware Reward Model (GRM):
  • Initialized from G-MLLM; trained as binary classifier on image pairs (same product) with group-conditioned CTR comparisons to predict which image yields higher CTR for group Gs,k.
• Group-DPO fine-tuning:
  • Uses preference tuples (product s, group Gs,k, winner prompt yw, loser yl) and optimizes a DPO-style likelihood-ratio objective: LAlign = −log σ(β log [πθ(yw)/πref(yw)] − β log [πθ(yl)/πref(yl)])
  • πref is the pre-trained reference model; β is temperature. LoRA used to reduce compute during fine-tuning.
• Generation:
  • Prompts from G-MLLM → Stable Diffusion + ControlNet-Inpaint to render advertising images for CTR evaluation and pair formation.
• Implementation details:
  • Embedding sizes mapped to 128-d; max 5 groups per product; percentile sampling strategy (e.g., 15th, 55th, 95th percentiles with repeated samples for coverage).
  • Training: G-MLLM pretraining ~10 epochs (LR 2e-6, cosine schedule) ~5 days; Group-DPO fine-tuning 3 epochs (LR 2e-5) ~50 hours on 8× H100.

Implications for AI Economics

• Improved ad targeting and ROI: Group-aware creative generation can increase CTR for specific user segments, improving conversion rates and ad spend efficiency. Advertisers can generate multiple creatives tailored to distinct consumer segments at scale.
• More granular market segmentation: PAAG enables dynamic, product-aware grouping rather than static demographic buckets, allowing advertisers to discover and exploit nuanced demand heterogeneity across products, potentially improving price discrimination and personalized offers.
• Resource allocation and pricing strategies: With reliable group-level uplift estimates, platforms and advertisers can reallocate budgets to creatives and segments with higher marginal returns, adjust bidding strategies, and better forecast return-on-ad-spend (ROAS).
• Welfare and distributional effects: While targeted creatives can boost efficiency, they also raise concerns about differential exposure and potential discriminatory outcomes (e.g., systematically favoring certain groups). These effects matter for regulation, consumer protection, and platform reputation.
• Privacy and compliance tradeoffs: High-resolution group formation relies on user attributes and behavioral logs. Deployments must balance targeting gains with privacy constraints (data minimization, differential privacy, consent), and comply with advertising and anti-discrimination regulations.
• Operational costs vs. gains: The approach requires additional modeling and compute (group formation per product, GRM training, multiple creatives generation). LoRA and group-limits mitigate costs, but platforms need to estimate whether incremental CTR gains justify infrastructure and dataset-management investments.
• Research infrastructure: GAIP provides a valuable benchmark for economic studies of personalized creative effects, enabling researchers to quantify heterogeneous treatment effects, design experiments for creative testing, and evaluate policy impacts of microtargeting.

If you want, I can extract quantitative results (CTR uplift, A/B test outcomes, or additional metric tables) from the full paper or create a short slide-style summary for stakeholders (product, policy, or finance teams).