A utility-aware contrastive loss steers generative images to sell better: applying the loss to image-generation models raises measured demand on Amazon and Airbnb versus state-of-the-art baselines while keeping images faithful to prompts and visually high-quality.

Main Finding

Incorporating estimated marketplace demand directly into multimodal contrastive learning (via a Utility-Aware InfoNCE loss) produces image representations that steer text-to-image generation toward images that both preserve semantic fidelity and increase downstream demand. Models fine-tuned with this Utility-Aware CLIP (and used in a Utility-Aware Generator) outperform state-of-the-art baselines on Amazon and Airbnb tasks (generation and editing), maintain text–image consistency and realism, and preserve empirically observed inverted-U demand relationships (e.g., for aesthetics and uniqueness). Human-subject choice experiments confirm improved purchase/booking likelihood for the generated images.

Key Points

• Problem addressed: standard CLIP-based generation optimizes semantic alignment and aesthetics, not marketplace outcomes; this leads to images that may look good but do not maximize sales/bookings.
• Core idea: augment CLIP similarity with a demand-driven visual utility term hv(v) to form a Utility-Aware CLIP score:
  • USθ,w(v,t) = αv · hv(v) + βs · sθ(v,t)
  • Used in a Utility-Aware InfoNCE loss (bidirectional) with temperature τ.
• Theoretical interpretation:
  • InfoNCE = negative log-likelihood of a conditional logit; adding hv(v) corresponds to adding observable utility features into that discrete-choice utility.
  • Authors derive mutual-information-style bounds and provide occlusion-based interpretability showing representations attend more to demand-relevant visual cues while retaining semantic alignment.
• Implementation:
  • Start from pretrained CLIP encoders (image encoder example: ResNet-50; text encoder: DistilBERT), project to shared embedding space, and fine-tune with Utility-Aware InfoNCE.
  • hv(v) is constructed from an estimated demand model (weighted index of domain-specific visual features like aesthetics, uniqueness, lighting, colorfulness, etc.).
  • The Utility-Aware CLIP is plugged into a text-to-image pipeline (Utility-Aware Generator) for both image generation and editing.
• Empirical results:
  • Applications on Amazon (product images) and Airbnb (listing photos) show consistent gains on demand-based scores and related metrics versus baselines (GPT-Image, Stable Diffusion, Flux).
  • The approach improves demand while preserving product identity and realism; avoids over-stylized outputs typical of general-purpose generators.
  • Preserves inverted-U patterns (moderate aesthetics/uniqueness are best), i.e., steers images toward optimal points rather than simply maximizing a single aesthetic score.
  • Human-subject experiments (choice tasks for likely-purchase / likely-book) validate commercial effectiveness.
• Practical claim: the method is modular and can be embedded into emerging generative systems to make them marketplace-aware.

Data & Methods

• Data sources:
  • Two empirical domains: Amazon product listings and Airbnb property listings (images paired with textual descriptions and observed marketplace outcomes: purchases/bookings).
  • Visual features for hv(v) are derived from interpretable computer-vision measures (aesthetics, uniqueness, brightness/colorfulness, warmth, etc.) estimated to predict demand in each context.
• Model architecture and objective:
  • Base encoders: pretrained CLIP components (image encoder such as ResNet-50; text encoder such as DistilBERT) with projection heads to a shared embedding space; cosine similarity sθ(v,t) used.
  • Utility-Aware similarity: linear combination of demand-driven scalar hv(v) and CLIP similarity, weighted by αv and βs.
  • Loss: Utility-Aware InfoNCE (bidirectional: text→image and image→text), i.e., negative log-softmax over utility-aware scores across sampled negatives; temperature τ used.
  • Training: fine-tune pretrained CLIP weights (not training from scratch) to obtain Utility-Aware CLIP.
• Generation/editing pipeline:
  • Utility-Aware CLIP is used to guide a text-to-image model (named Utility-Aware Generator). Base generator for comparisons includes Flux (open-source SOTA), GPT-Image, Stable Diffusion.
  • Both full-generation (from prompt) and targeted editing (modify existing image while preserving identity/realism) evaluated.
• Evaluation:
  • Quantitative: demand-based metrics (predicted purchase/booking likelihood using estimated demand model), fidelity and realism metrics, semantic consistency (CLIP-like measures), and specific checks for preserving inverted-U attribute relationships.
  • Baselines: Flux, Stable Diffusion, GPT-Image, and an unmodified CLIP-guided generator.
  • Human-subjects: preference/choice experiments where participants select images they'd be most likely to purchase or book, used to validate economic effect.
• Interpretability & theory:
  • Discrete-choice interpretation: InfoNCE ↔ conditional logit; utility term maps naturally into this framework.
  • Mutual information bounds for the Utility-Aware InfoNCE are provided to clarify how the objective preserves semantic alignment while embedding demand signals.
  • Occlusion-based attribution analyses show the fine-tuned model attends more to demand-relevant visual regions/features.

Implications for AI Economics

• Aligning generative models with economic objectives:
  • Demonstrates a practical mechanism to move generative AI from "style imitation" toward direct commercial value by embedding demand estimates into learning objectives.
  • Opens a path for platform- and seller-level automation: automatic suggestion or creation of secondary images that are more likely to convert, reducing reliance on prompt engineering and professional photography.
• Platform strategy and product design:
  • Platforms can integrate such modules to boost marketplace performance (recommend image edits, A/B test counterfactual images, surface low-quality listings).
  • Sellers with limited resources can gain conversion benefits at scale; implications for competition (smaller sellers may be able to match the visual quality of larger sellers).
• Measurement and evaluation:
  • Suggests complementing standard semantic/fidelity metrics (e.g., CLIP score) with utility-aware evaluation aligned to business KPIs (clicks, conversions, bookings).
  • Shows the value of interpretable visual features for connecting model outputs to economic outcomes.
• Economic dynamics and risks:
  • Potential for homogenization: if many sellers follow demand-optimal cues, visual differentiation may compress; however, the model can preserve inverted-U preferences (so extreme homogenization may be limited).
  • Incentive effects: platforms might prioritize listings edited by their own utility-aware tools, raising questions about platform neutrality and fairness.
  • Overfitting to historical demand: models depend on accurate demand estimation; if preferences shift or data reflect biases, generated images could misalign with future consumer tastes or perpetuate biases.
• Privacy, regulation, and trust:
  • Using observed marketplace behavior to guide content raises privacy and transparency considerations (how demand models are trained and applied).
  • Trust and authenticity trade-offs: the method claims to preserve realism, but economic incentives might push toward subtly manipulative edits—regulatory oversight or disclosure policies may be needed.
• Research and commercialization directions:
  • The modular utility-aware component can be generalized to other objective functions (lifetime value, retention, reviews) and to personalization (user-segment-specific hv(v)).
  • Future work should study dynamic effects (how generated images affect long-run demand, returns, and platform welfare) and cross-category generalization.
• Overall: embedding econometric demand signals into multimodal representation learning is a promising route to make generative AI directly actionable for market outcomes, but it requires careful handling of incentives, model robustness, fairness, and transparency.