Meta’s Dear Algo converts natural-language feed steering into executable retrieval plans, improving judged precision in deployed tests — a blinded audit finds 94.4% exact-Relevant precision and a candidate-randomized serving-path window shows a 1.97 percentage-point reduction in judged irrelevant admissions while increasing exact-Relevant share.

Main Finding

Dear Algo implements an agentic intent layer (STAR on SilverTorch) that compiles natural-language user instructions into a grounded, executable retrieval plan (STQL) which steers feed recommendation. Under a precision-first evaluation, the system achieves high admission precision (94.4% exact-Relevant in a blinded audit) and yields more judge-qualified candidates and fewer irrelevant admitted items than LLM-derived-query or non-reranked baselines, demonstrating that explicit natural-language intent can be carried into feed recommendation cost-effectively without replacing existing retrieval/ranking infrastructure.

Key Points

• System design

  • An intent layer interprets explicit, inferred, negative, and compound user intent into typed semantic features, grounds phrases to a controlled item-side vocabulary, and compiles a JSON-serialized retrieval DSL (STQL).
  • Architectural separation: compile-once (control plane LLM grounding) + execute-many (data-plane candidate retrieval), with optional bounded LLM/multimodal reranking.
  • Modes share infrastructure and representation (catalog IDs, vocabulary, STQL plans) but retain mode-specific retrieval, ranking, and thresholds.
  • Priority on admission precision: admitting an irrelevant item has outsized downstream cost in persistent feeds, so precision-first operating points guide design choices.

• Evaluation highlights

  • Blinded human calibration: sampled from a frame of 48,876 request-item records; audit of 300 pairs (296 evaluable) validated the production LLM-as-a-judge (LAAJ) at a strict categorical gate: 94.4% exact-Relevant admission precision [88.8%, 98.9%].
  • Paired offline candidate study (fixed Top-20 budget, 72 normalized request clusters): full grounded-tag policy produced 7.73 judge-qualified candidates per 20 slots versus 6.61 for an LLM-derived-query + retrieved-tag baseline — a gain of 1.11 [0.12, 2.12] judge-qualified candidates.
  • Candidate-randomized reranker study (common-support 0–72h window): among judged admitted candidates, reranker-on path had lower user-weighted judge-Irrelevant share (2.80% vs 4.78%, −1.97 points [−3.02, −0.94]) and higher Exact-Relevant share (+2.24 points [0.08, 4.41]). Note: reranker path had higher cost/latency and a smaller candidate budget.
  • Logging and measurement: used LLM-as-a-judge as an instrument but validated it with human adjudication; careful separation of estimands (request relevance, admitted-set quality, engagement, reports) emphasized.

• Limitations and design caveats

  • The system shares infrastructure and representation but does not claim empirical unification (i.e., causal cross-mode gains remain an open question).
  • Reranker-on path had different TTL and candidate budget; comparisons restricted to common-support window and estimate admitted-set quality, not recall or full ranking effects.
  • LAAJ is a measurement instrument with proprietary config; LLM-judge biases and operational choices matter and were explicitly validated here.

Data & Methods

• System: STAR (SilverTorch Agentic Recommender) produces STQL plans that execute on SilverTorch candidate service; optional LLM/multimodal reranker on a bounded slate.
• Human calibration
  • Sampling frame: 48,876 public-mode request-item records (July 17–24, 2026).
  • Audit sample: 300 pairs (296 evaluable) drawn with stratified oversampling of Irrelevant/Partial strata.
  • Instrument: production LLM-as-a-judge (LAAJ) evaluated at a strict categorical gate; primary estimand P(human Relevant | judge Relevant). Bootstrap CIs and a Dawid–Skene rater-severity sensitivity check were used.
• Paired offline candidate study
  • 100 request events → 72 normalized request-text clusters after filtering.
  • Three arms (raw request vs LLM-derived query; retrieved canonical tags vs full LLM-selected grounded tags) all run on same request-time inventory and same Top-20 budget.
  • Outcome: judge-qualified candidates@20 (count of candidates with LAAJ score ≥ 0.8) and Yield@20; 20k cluster bootstrap for intervals; sensitivities at stricter cutoffs.
• TTL-aligned candidate-randomized reranker study
  • Logged public-mode admitted impressions July 17–24, 2026.
  • Randomized per-request candidate assignment between reranker-off (standard) and reranker-on (LLM reranker).
  • Restricted primary analysis to request→impression age < 72 hours (common-support due to reranker TTL).
  • Dataset in window: 93,972 rows across 4,321 requests; paired cohort: 50,922 off candidates (31,099 judged) vs 3,770 on candidates (2,928 judged); 845 fully judged paired requests sensitivity.
  • Metrics: Exact-Relevant share and false-admission (Irrelevant) rate among judged admitted candidates; user-cluster bootstrapped intervals and adversarial bounds for unjudged candidates.
• Observational behavioral study
  • Measured within-user app-usage change after explicit requests (descriptive, non-randomized; lacks causal control).

Implications for AI Economics

• Cost structure and resource allocation

  • The modular design (LLM-based intent compilation + conventional retrieval/ranking) lets platforms amortize expensive LLM costs across refreshes, lowering marginal serving cost compared with an end-to-end LLM replacement. This supports a favorable cost-benefit tradeoff for precision-sensitive applications.
  • Precision-first objective prioritizes avoiding costly false admissions (irrelevant items that repeatedly reach users). Economically, this reduces negative externalities (user churn, reporting, moderation costs) that can outweigh marginal gains from increased coverage.
  • The optional reranker improves admission quality but increases latency and compute; choosing when to apply reranking is an economic decision balancing improved relevance (and downstream engagement/value) against higher per-request costs and lower candidate throughput.

• Product and platform economics

  • Grounded intent (persistent STQL plans) creates durable user-state that can increase long-term personalization value, potentially increasing engagement and lifetime value—if the plan persists and adapts without causing harmful feedback loops.
  • Controlled vocabularies and grounded identifiers facilitate auditing, provenance, and deletion—features with regulatory and compliance value that reduce legal/operational risk costs.
  • The system enables incremental deployment: control-plane innovation (LLM grounding) can be rolled out without replacing the data-plane recommendation stack; this lowers migration costs and operational risk.

• Measurement, incentives, and governance

  • The studies highlight that measurement instruments (LLM-as-a-judge) require calibration and validation; investing in labeling and audit infrastructure is an economic cost but essential to avoid misestimated product decisions.
  • Accurate logging and TTL-aware randomization are necessary for causal inference; insufficient logging inflates evaluation uncertainty and can misallocate investment across serving paths or models.
  • Because admitting an irrelevant item has asymmetric downstream cost, platform incentives align to invest more in admission precision than in coverage when user retention / safety is the priority.

• Research and monetization opportunities

  • Economic modeling opportunities: estimate the marginal value of an additional judge-qualified candidate (engagement uplifts, retention) vs marginal reranker cost to optimize budgeted allocation of expensive compute (e.g., which requests should receive reranking).
  • Long-run experiments are needed to measure welfare and monetization impacts of persistent intent plans: do explicit intent signals lead to sustainably higher engagement or to polarization/overfitting creating long-term harms?
  • Marketplace implications: such agentic intent layers could be productized as premium steering controls (monetizable features) if they demonstrably increase user value without prohibitive content-moderation cost.

• Limitations with economic relevance

  • The paper stops short of proving causal cross-mode gains (empirical unification), so claims about downstream revenue or long-run welfare remain speculative without randomized, long-horizon experiments.
  • Reranker-on path had smaller candidate budgets (by design) and TTL constraints; economic conclusions about reranker deployment should account for these operational constraints.

Overall, Dear Algo demonstrates an economically sensible pattern: use LLMs where they add unique semantic grounding value (control plane), retain high-throughput retrieval/ranking for the data plane, and choose precision-first operating points when the cost of false admissions is asymmetric and potentially large. Future economic work should quantify the tradeoffs across compute cost, admission precision, engagement lift, and long-run user welfare to optimize deployment at scale.