An AI screener can cut some recruitment bias when tightly configured — anonymized CVs, full-text MPNet retrieval and full job descriptions paired with a baseline GPT-4.1-nano prompt performed best — yet gender is still inferable from resumes and stylistic differences magnify unfairness, so technical tuning alone does not eliminate risk.

Main Finding

An agentic, recruiter-in-the-loop screening system (HireX) can speed project-based candidate search and be deployed within existing ERP workflows while reducing—but not eliminating—gender bias if the entire sociotechnical pipeline is jointly optimized. Key technical choices matter: anonymization alone does not remove gender signals; full-text MPNet retrieval on anonymized CVs paired with full job descriptions and an advisory GPT-4.1‑nano screener produced the best trade-off between fairness, utility and operational cost in the audited configuration.

Key Points

• Audit scope and design

  • Empirical audit of HireX across social and technical subsystems using 485 counterfactual male–female CV pairs at three writing-style intensities.
  • Configuration sweep over embedding model & strategy (full-text vs section-wise, original vs anonymized), input granularity (title vs full job description), LLM choice and prompts, and presence/location of a bias-aware audit agent.
  • Fairness measured by Statistical Parity Difference (SPD), Disparate Impact (DI), Counterfactual Fairness Gap and an exposure-weighted variant.

• Principal empirical findings

  • Gender signals are recoverable from anonymized CVs: models (embeddings and LLMs) can infer candidate gender from writing style alone (P1).
  • Embedding strategy drives retrieval bias: section-wise embeddings produce the largest gender skew in retrieval even before LLM screening (P2).
  • Input granularity reduces bias: using full job descriptions as queries reduces retrieval-stage bias vs title-only queries (P4).
  • Writing-style intensity amplifies bias: larger stylistic differences between paired CVs increase bias across configurations (P5).
  • A separate bias-aware audit agent had limited net improvement in the recommended, recruiter-in-the-loop advisory architecture; joint pipeline choices mattered more than adding an audit agent (P3, P6).
  • Recommended practical deployment: full-text MPNet embeddings on anonymized CV text, full job descriptions as queries, baseline screening prompt on GPT-4.1‑nano with anonymized labels, recruiter-in-the-loop advisory agent, and API integration with Deltek Maconomy.

• Systems and operations

  • HireX uses embedding-based retrieval (ChromaDB in the prototype) and LLM-based screening (LangGraph orchestration).
  • The audit quantified trade-offs between fairness, utility and operational cost; cost-per-screening and latency influenced LLM choice (GPT-4.1‑nano recommended for pragmatic cost/latency balance).

• Limitations highlighted by authors

  • Counterfactual synthetic-pair construction and domain restriction (project-based energy/industry, male-dominated dataset) limit generalisability.
  • Binary gender framing and intersectionality not addressed.
  • Audit scope limited to a subset of embedding/LLM families and configurations; post-deployment monitoring required.

Data & Methods

• Data

  • 485 paired counterfactual CVs derived from Plant Supervision’s internal CV database; pairs were created to control qualifications while swapping gender markers and varying writing-style intensity (three levels).
  • CVs were anonymized (names/pronouns removed) for many conditions to test residual gender signals.

• Pipeline & configurations tested

  • Embedding models: MPNet used as the recommended model; several embedding models/strategies evaluated (full-text vs section-wise; original vs anonymized).
  • Retrieval: cosine-similarity ranking in an embedding index (ChromaDB in prototype).
  • Screening: LLM-based agent (various LLMs tested; GPT-4.1‑nano recommended for screening prompt baseline).
  • Orchestration: LangGraph for agent workflows including an optional bias-aware audit agent.

• Evaluation metrics

  • Statistical Parity Difference (SPD)
  • Disparate Impact (DI)
  • Counterfactual Fairness Gap (CF Gap)
  • Exposure-weighted variant of counterfactual gap to capture top-ranked exposure differences.

• Experiments

  • Gender identifiability tests (can embeddings/LLMs recover gender from anonymized CVs?)
  • Embedding-strategy bias audit (how embedding choices affect retrieval fairness at top-k cutoffs)
  • Bias-aware audit agent effectiveness (placement and impact of audit agent)
  • Job-description granularity study (title vs full description)
  • Writing-style intensity study (how stylistic divergence affects bias)
  • ERP integration evaluation (Deltek Maconomy API endpoint, pre-deployment health traces, LLM token/cost/latency monitoring)

Implications for AI Economics

• Microeconomic firm-level effects

  • Productivity and competition: Automated retrieval + LLM screening can materially reduce time-to-shortlist, improving competitive position for firms operating in time-sensitive project recruitment markets.
  • Cost–fairness trade-offs: Firms face explicit trade-offs—choosing high-performing LLMs/strategies raises per-screening cost but can reduce bias or improve utility. The audit shows practical middle-ground choices (MPNet + GPT-4.1‑nano) that economize on cost while improving fairness metrics relative to many alternatives.
  • Labor-market composition: Because embedding and retrieval choices affect who is surfaced, upstream model design can shape which demographic groups obtain project offers—introducing potential long-run composition effects in project-based labor markets.

• Regulatory and compliance economics

  • EU AI Act obligations for high‑risk recruitment systems make empirical audits and documented joint optimization economically necessary (compliance costs, auditing overhead, potential fines/market access consequences).
  • Investing in pre-deployment audits and ongoing monitoring can be cost-effective by reducing regulatory risk and reputational externalities. However, auditing itself has costs (data preparation, counterfactual generation, metric computation) that must be budgeted.

• Organizational and governance impacts

  • Value of sociotechnical joint optimization: Optimizing embeddings, retrieval inputs, prompts, agent architecture and recruiter workflows together yields better fairness/utility outcomes than optimizing single components in isolation—implying organizational processes should align data science, product, legal and HR functions.
  • Recruitment process design matters economically: Advisory, recruiter-in-the-loop architectures preserve human accountability and reduce liability/exposure for firms, but require investing in effective UI/decision explanations (transaction costs of human review remain).

• Policy and market-level considerations

  • Market signaling and adoption: Demonstrable, audited fairness improvements can be a market differentiator for staffing firms; conversely, evidence of biased algorithms can cause client churn and regulatory scrutiny.
  • Externalities and inequality: If many firms adopt poorly-audited retrieval/LMM pipelines that amplify gendered writing-style differences, aggregate labor-market inefficiencies and inequality may increase—calling for sectoral standards and shared audit datasets or toolkits to internalize these externalities.

• Practical recommendations with economic rationale

  • Prioritize retrieval choices (embedding strategy & query granularity) as low-cost, high-impact interventions to reduce bias exposure before expensive LLM screening.
  • Use anonymization plus stylistic-normalization measures (or prefer full-text embeddings over section-wise) to reduce recoverable gender signals—this limits costly downstream corrections.
  • Favor recruiter-in-the-loop advisory deployment to limit legal/regulatory exposure while realizing automation gains; complement with monitoring to detect drift and maintain compliance.
  • Budget for ongoing post-deployment audits and monitoring—short-term audit costs reduce long-run regulatory and reputational costs.

Summary conclusion: The thesis demonstrates that in AI-enabled recruitment, equitable outcomes are achieved more by joint sociotechnical design (embedding strategy, input granularity, model choice, and human workflow) than by single point interventions. For firms and policymakers, the economically efficient path is to invest in upstream retrieval design, documentable audits, and recruiter-in-the-loop architectures that balance fairness, utility and operational cost.