A recommendation-native Transformer (ReST) tailored to noisy, irregular user behavior and asymmetric ranking computation delivers consistent scaling and better accuracy, and in a one-week production A/B test raised a core revenue metric by 11.93% while meeting a 50 ms P99 latency constraint.

Main Finding

ReST (Recommendation-native Scalable Transformer) is an encoder–decoder redesign of Transformer sequence models that (1) introduces recommendation-specific architectural and training changes to handle noisy, temporally irregular, and sparsely supervised behavior sequences, and (2) factorizes computation to exploit the one-history-to-many-candidates structure of ranking. On a large-scale industrial advertising deployment (TikTok Shop Ads) and public benchmarks, ReST scales reliably along sequence length, depth, and width where vanilla LLM-style blocks saturate, yields better accuracy–efficiency trade-offs, and produced an online A/B improvement of +1.31% AUC and +11.93% on a core revenue metric within a 50 ms P99 latency budget; it is fully deployed in production.

Key Points

• Two core challenges for naively transplanting LLM-style scaling to recommendation:
  • Signal quality: behavior tokens are noisy (implicit feedback), temporally irregular, and supervision is sparse and shortcut-prone by non-sequential features (sequence-starvation).
  • Computation asymmetry: one shared user history versus N candidates per request; per-candidate compute is the latency bottleneck.
• ReST design (high level):
  • Asymmetric architecture: heavy, reusable sequence encoder T that contextualizes a user history once; lightweight cross decoder C that scores each candidate efficiently.
  • System co-design: user-level shared-prefix training and shared-prefix serving to realize compute-once / decode-many reuse.
• Rec-native encoder innovations:
  • Dual-Gated Attention (DGA): gates both the value stream (pre-aggregation) and the attention output (post-aggregation) to suppress noisy behavior features and control context updates.
  • Rotary Positional + Rotary Temporal Embedding (RoPE + RoTE): splits attention heads so some encode ordinal position (RoPE) and others encode discretized physical time buckets (RoTE) at multiple granularities to capture multi-scale recency while remaining compatible with efficient rotary attention kernels.
  • Stabilized Residual Normalization (SRN): depth-dependent LN placement (Mix-LN-style) plus small learnable scalar multipliers on residual branches (α_l initialized small) to stabilize deep encoder training under sparse supervision.
• Auxiliary training to mitigate sequence starvation:
  • Auxiliary sequence CVR head directly supervises the sequence branch (increases its gradient signal).
  • Sigmoid-contrastive sequence/non-sequence alignment loss aligns sequence and non-sequence user representations (training-only).
• Lightweight cross decoder optimizations:
  • Projection-free keys/values: reuse encoder outputs H as KV (no per-candidate W_K, W_V) to reduce repeated candidate computation.
  • Token-specific parameterization (TSP): per-query-token parameter sets (e.g., [CLS], ad, user) to keep decoder capacity without growing activated FLOPs.
• Empirical outcomes:
  • ReST achieves better offline accuracy–efficiency tradeoffs than LLaMA-style and strong recommendation baselines, and continues to gain with longer sequences and larger depth/width where LLM-style blocks plateau.
  • System-level reuse reduces encoder computation per request from N * FLOPs(T) to FLOPs(T) (plus N * FLOPs(C)), enabling larger T while meeting strict P99 latency constraints.

Data & Methods

• Problem: pointwise conversion-rate (CVR) prediction (binary classification) in large-scale ad ranking; training loss is BCE over observed conversions.
• Primary dataset: large-scale, anonymized industrial dataset from TikTok Shop Ads (exact scale withheld for confidentiality). Evaluation vs. a production DLRM baseline; reported metric is relative AUC delta.
• Public benchmarks: MovieLens-1M, MovieLens-20M, Amazon-Books. These were cast into pointwise chronological CVR-style prediction for comparability; reported AUC and log loss.
• Baselines: target-aware attention methods (e.g., DIN, STCA), causal LLaMA-style Transformers, HSTU, and an encoder–decoder Transformer baseline.
• Key architectural / algorithmic methods:
  • Encoder T: causal self-attention with SwiGLU FFN modified by DGA, RoPE+RoTE, SRN.
  • Cross decoder C: small set of semantic query tokens, projection-free KV attention, TSP for per-token parameters, bounded sigmoid output gates.
  • Training: L_total = L_ce (main BCE) + λ_seq L_seq (aux sequence CVR) + λ_align L_align (sigmoid contrastive alignment). User-level shared-prefix training groups multiple samples from the same user and applies prefix-valid causal masks to amortize encoder computation during training.
  • Serving: compute encoder memory once per request and reuse for N candidates.
• Ablations: component-level ablations (DGA, RoTE, SRN, auxiliaries, projection-free KV, TSP) show complementary contributions; DGA and auxiliaries notably raise the sequence encoder gradient magnitude and accuracy gains. Detailed numeric ablations reported in paper appendices.
• Deployment constraints: designed to meet strict production latency (50 ms P99 in their test), and demonstrated full production deployment after A/B test.

Implications for AI Economics

• Direct revenue impact from model-level improvements can be large and fast: a 1.31% AUC lift yielded an 11.93% uplift on a core revenue metric in production within the P99 latency envelope — indicating high leverage from investing in sequence-model improvements that are latency-aware.
• Cost-effective scaling axis: scaling the sequence encoder (compute amortized across candidates) can be a higher-ROI alternative to scaling per-candidate decoding or blindly increasing model size everywhere. ReST demonstrates an architectural way to shift compute investment to the amortizable side, improving accuracy per deployed compute dollar.
• System–model co-design matters for operational economics:
  • Shared-prefix training and serving reduce redundant computation (both training and inference), lowering GPU/CPU cost per sample and enabling more expensive encoders under fixed latency/cost constraints.
  • Projection-free KV and token-specific parameterization are practical levers to trade per-request latency/cost for model capacity where it matters.
• Data and labeling implications:
  • Multigranular timestamp modeling (RoTE) and explicit denoising (DGA) increase value of accurately logged timestamps and higher-fidelity interaction signals — suggesting that investments in better telemetry/time resolution and noise-mitigation (e.g., exposure controls) can amplify model returns.
  • The sequence-starvation problem highlights that high-capacity sequence modules require targeted supervision (auxiliary heads, alignment losses). Organizations should budget for additional annotation/auxiliary objectives and validation to realize scaling gains.
• Deployment and infrastructure strategy:
  • For large-scale recommendation platforms, prioritizing architectural designs that permit compute reuse (compute-once / decode-many) can reduce serving fleet size or improve per-serving throughput, altering capital and operational expenditure planning.
  • ReST’s approach suggests alternative evaluation criteria beyond raw model FLOPs—evaluate FLOPs amortization, activated FLOPs per request, and P99 latency vs. revenue uplift.
• Broader economic consideration:
  • The paper shows diminishing returns for direct transplantation of LLM-style scaling to recommender use cases; tailored architectural and training investments specific to the product/behavioral signal can be more efficient. This should guide R&D prioritization and budget allocation in applied ML teams.

Limitations / caveats (practical): - Industrial results are reported relative to a specific production DLRM baseline and on an ads CVR task; absolute gains and cost/benefit ratios will vary by product, candidate fanout N, and dataset properties. - Some design choices (e.g., bucketization granularity in RoTE, α_l initialization in SRN, auxiliary loss weights) require tuning per deployment and may interact with platform-specific serving/serialization constraints. - Confidentiality limits released dataset sizes and some quantitative details; replicability on other large production environments should still be validated.

If you want, I can produce a concise diagram of ReST’s inference-time compute flow (showing FLOPs amortization), or extract the key equations and ablation results into a one-page handout for engineering/product stakeholders.