LLM-synthesized training data lets models code skills in job ads without labeled examples: a hierarchy-aware zero-shot pipeline maps Chinese job advertisements to ESCO skills with F1@5 = 0.72, outperforming TF–IDF and BERT baselines.

Main Finding

A zero-shot pipeline that synthesizes training data from ESCO skill definitions with an LLM and trains a contrastive bi-encoder (BERT + BiLSTM + attention) can map Chinese job-ad text to the ESCO taxonomy without any manual job-ad labels. Hierarchy-conditioned multi-skill generation (conditioning co-occurrences on ESCO Level-2) improves synthetic-data realism and transfer. The resulting system outperforms TF–IDF and standard BERT baselines on real-world Chinese job ads, achieving strong zero-shot retrieval (posting-level F1@5 ≈ 0.72).

Key Points

• Problem: mapping informal, heterogeneous job-ad language to a large, fine-grained skill taxonomy (ESCO) is an extreme multi-label classification (XMLC) task but labeled job-ad data are costly, especially outside English.
• Zero-shot strategy: use ESCO textual definitions to synthesize job-requirement sentences via an LLM and train a contrastive bi-encoder that aligns job sentences and ESCO skill descriptions in a shared embedding space.
• Hierarchy-conditioned generation: multi-skill synthetic sentences are sampled conditioned on ESCO Level-2 subtrees to induce realistic co-occurrence patterns and reduce semantic drift (better fluency and discriminability than random pairing).
• Architecture: siamese bi-encoder with bert-base-chinese backbone, BiLSTM (bidirectional, h′=256) + attention pooling, linear projection to 128-d L2-normalized embeddings.
• Upstream filter: a RoBERTa-base binary classifier (trained on synthetic positives and negative non-requirement samples) removes non-skill sentences before retrieval to boost precision and efficiency.
• Learning objective: margin-based contrastive ranking loss with margin λ=0.5 and K=5 negatives per positive; multi-label synthetic samples averaged across positives.
• Inference: precompute embeddings for all ESCO skills (≈13.9k leaf skills), retrieve Top-K then apply similarity threshold γ, aggregate sentence-level hits by union to produce posting-level skill sets.
• Empirical result: zero-shot end-to-end pipeline transfers to 200k Chinese job ads (Zhaopin.com sample) with posting-level F1@5 = 0.72 and outperforms TF–IDF and standard BERT baselines. Ablations show hierarchy-conditioned Dmulti > Random multi-pairing; BiLSTM+attention > [CLS]-only encoding; negatives and margin affect ranking trade-offs.

Data & Methods

• Ontology: ESCO v1.1 (leaf Level-4 skills used); ~13,890 distinct skills (manually translated to Chinese for this study).
• Real evaluation corpus: 200,000 job advertisements from Zhaopin.com (2015–2023); postings segmented into sentences (requirements mixed with non-requirements).
• Synthetic data:
  • Dsingle: single-skill sentences per ESCO skill (LLM-generated).
  • Dmulti: hierarchy-conditioned co-occurring skill sentences (pairs sampled from same ESCO Level-2).
  • Random-Dmulti: unconstrained random skill pairs (used for comparison).
  • Dnone: synthetic non-requirement job-ad sentences for negative class in filter training.
  • LLM & decoding: DeepSeek-V3, temperature 0.7, top-p 0.9, max 128 tokens. Post-generation deduplication (cosine threshold on Sentence-BERT embeddings) and diversity enforcement (n-gram checks).
  • Safeguards: ambiguity-aware sampling for generic skills; evaluation-time disambiguation via thresholds and posting aggregation.
• Encoders:
  • Backbone: bert-base-chinese.
  • Token outputs -> BiLSTM (bidirectional, per-direction size 256) -> attention pooling -> projection Wp to d=128 -> L2 normalization.
  • Sentence and skill share the same encoder (siamese).
• Training:
  • Contrastive margin loss: L(t,s+) = (1/K) Σ_k max(0, λ − sim(et, es+) + sim(et, es−_k)); λ=0.5, K=5 negatives.
  • For multi-label synthetic samples, loss averaged across positives.
  • Hyperparameters (typical): LR 2e−5, batch size 32, 10 epochs, embedding dim 128.
• Filtering:
  • RoBERTa-base binary classifier trained on synthetic positives (Dsingle and/or Dmulti) vs Dnone negatives; threshold τ tuned for high precision.
• Inference & metrics:
  • Precompute ESCO embeddings; retrieve Top-Kr candidates (Kr=50 default), then threshold similarity γ tuned on dev.
  • Posting-level aggregation via union across retained sentence predictions.
  • Main reported metric: posting-level F1@5 = 0.72 (zero-shot). Benchmarked against TF–IDF and standard BERT baselines; ablations conducted on generation strategy, architecture components, negatives/margin.

Implications for AI Economics

• Scalable skill measurement without manual labels:
  • Enables large-scale, low-cost mapping of job ads to a standardized skill taxonomy (ESCO), which supports fine-grained labor demand measurement across industries, regions, and time—critical for empirical work in labor economics and workforce analytics.
  • Facilitates frequent re-mapping when taxonomies evolve (no need to re-annotate large corpora).
• Cross-country and multilingual applications:
  • The pipeline (translate + synthesize + contrastive alignment) provides a practical route to deploy taxonomy-aligned skill extraction in non-English labor markets where labeled data are scarce.
  • Supports comparative studies of skill demand and occupational transitions across countries using a common ontology.
• Better treatment of co-occurrence and complementarities:
  • Hierarchy-conditioned synthetic samples produce more realistic multi-skill signals, improving the ability to measure skill bundles and complementarities—important for models of task routinization, complementarity with capital, and wage dynamics.
• Policy and programmatic uses:
  • Improved, up-to-date skill demand indicators can inform training programs, active labor-market policy, and regional skills strategies with finer resolution than occupation-level aggregates.
• Econometric and empirical considerations / caveats:
  • Validation recommended: zero-shot outputs should be validated against small, high-quality annotated samples before use in causal or policy studies (to detect systematic errors).
  • Bias and artifacts from LLMs and translation: synthesized supervision can inherit LLM biases or produce artifacts that skew measured skill prevalence—researchers should audit distributions and compare with external benchmarks.
  • Ambiguity in generic/soft skills: generic or overlapping skills remain challenging; post-hoc disambiguation, threshold tuning, or manual re-labeling for such classes may be necessary.
  • Sensitivity to thresholds and aggregation: inference thresholds (τ, γ, Top-K) materially affect precision/recall trade-offs; choose based on downstream use (e.g., conservative for policy signals, more recall for exploratory analytics).
  • Computational cost and maintenance: embedding ~14k labels and running nearest-neighbor retrieval is tractable but requires engineering (ANN indexing, periodic re-embedding when taxonomy changes).
• Recommendations for applied researchers:
  • Use hierarchy-conditioned synthetic generation when modeling skill co-occurrence or bundles.
  • Reserve a small annotated holdout for calibration and bias checks even when using a zero-shot pipeline.
  • Combine this retrieval-first approach with a lightweight supervised reranker when a small labeled dataset is available to further improve precision on high-stakes tasks.
  • Track and document changes in ESCO and synthetic generation prompts to ensure reproducibility of skill-time series.
• Directions for further research relevant to AI economics:
  • Quantify how zero-shot-derived skill measures change economic conclusions (wages, automation exposure, mobility) compared to traditional occupation-based measures.
  • Systematic audit of LLM-synthesized supervision for demographic, occupation, or industry biases.
  • Hybrid pipelines: compare gains from small labeled datasets plus synthetic data versus purely zero-shot; cost-benefit analysis for annotation budgets.

If you want, I can (a) extract a concise list of recommended operational thresholds and engineering practices from the paper for production deployment, (b) draft a short protocol for validating zero-shot skill labels with a small annotated sample, or (c) produce a suggested code/experiment checklist to reproduce the main results. Which would be most useful?