An explainable XGBoost risk model predicts supplier disruption far better than a traditional scorecard in one manufacturing firm (ROC‑AUC 0.928), and dashboards plus SHAP explanations made the outputs actionable for procurement; supplier dependency materially increases the operational consequences of predicted risk.

Main Finding

An explainable ML framework (XGBoost + SHAP + dashboarding) trained on a manufacturing firm's ERP procurement data substantially outperformed a traditional supplier scorecard at predicting supplier risk (test: accuracy 89.7%, precision 87.9%, recall 86.5%, F1 87.2%, ROC‑AUC 0.928; McNemar test: XGBoost significantly better). The framework produced actionable, interpretable risk signals and dashboards for procurement decision-making, and supplier dependency was found to significantly amplify the operational impact of predicted supplier risk (moderation β = 0.214, p = 0.018).

Key Points

• Problem: Traditional scorecards are descriptive, use fixed weights, and often flag risk only after disruption; need for predictive, explainable supplier risk tools.
• Data: Real anonymized ERP procurement data covering 100 active suppliers (Jan 2023–Dec 2024) with indicators for delivery, quality, invoice mismatches, lead-time instability, fulfilment, dependency and disruption exposure.
• Modeling: Compared traditional scorecard, Logistic Regression, Random Forest, and XGBoost. Training used a stratified 80:20 split, five-fold cross-validation, and 1,000-iteration bootstrap resampling to address modest high-risk class size.
• Best performer: XGBoost achieved the highest predictive performance (accuracy 89.7%; precision 87.9%; recall 86.5%; F1 87.2%; ROC‑AUC 0.928).
• Statistical validation: McNemar test demonstrated XGBoost significantly improved classification relative to the scorecard.
• Explainability & deployment: SHAP used for global and supplier-level explanations; Power BI translated model outputs into supplier risk scores, alerts, trend views and mitigation priorities for practitioners.
• Operational insight: Moderation analysis shows supplier dependency increases the operational consequences of predicted supplier risk (β = 0.214, p = 0.018).
• Practitioner validation: An expert panel of eight procurement and supply‑chain professionals judged the framework interpretable and useful for decision support.
• Contribution: A practical, end-to-end decision-support framework combining predictive accuracy, explainability, class-imbalance-aware validation, and dashboard-driven managerial actions.

Data & Methods

• Data source: Anonymized ERP procurement records from one manufacturing firm; full cleaned population of 100 active suppliers observed monthly (Jan 2023–Dec 2024).
• Features: Delivery timeliness, quality failures, invoice mismatches, fulfilment metrics, lead‑time instability, purchase dependency measures, disruption exposure indicators.
• Target: Binary supplier risk label (high-risk vs. low-risk) derived from operational outcomes (as used in study).
• Sample handling: Stratified 80:20 train-test split to preserve class proportions; five-fold cross-validation on training set; 1,000-iteration bootstrap resampling to increase robustness given limited high-risk cases.
• Models compared: Traditional (fixed-weight) supplier scorecard baseline; Logistic Regression; Random Forest; XGBoost.
• Evaluation metrics: Accuracy, precision, recall, F1-score, ROC‑AUC; paired-significance testing using McNemar test to compare classifiers against the scorecard.
• Explainability & visualization: SHAP values for global and individual explanations; Power BI dashboards for translating predictions into operational artefacts (scores, alerts, trend views, mitigation priorities).
• Additional analyses: Moderation regression to test how supplier dependency modifies the relationship between predicted risk and operational impact; expert panel qualitative review for interpretability and decision utility.

Implications for AI Economics

• Value capture and ROI:
  • Predictive accuracy with explainability can reduce surprise disruption costs and enable targeted mitigation (safety stock, dual sourcing, contract renegotiation), producing measurable procurement savings and uptime gains.
  • Dashboarded outputs facilitate faster managerial response and more efficient allocation of mitigation budgets, potentially lowering expected disruption-related expenditures.
• Supplier market structure & bargaining:
  • Highlighting dependency as an amplifier of impact suggests strategic value in reducing single-supplier exposure; firms may rebalance sourcing, affecting demand across suppliers and potentially altering market power and prices.
  • Automated risk scoring could change negotiation dynamics: high‑risk suppliers may face stricter terms or investment requirements; lower-risk suppliers may capture more business.
• Risk pricing and contracting:
  • Predictive risk signals can be incorporated into contract design (contingent penalties, options for capacity reservation, insurance pricing) and supply‑chain financing terms.
  • Better measurement of supplier risk supports more efficient risk-sharing contracts and may enable new financial instruments (supplier performance‑linked financing).
• Policy and governance:
  • Explainability (SHAP) and expert validation improve auditability and managerial trust — important for compliance, internal governance, and possible regulatory scrutiny of algorithmic decisions affecting suppliers.
  • Caution: automated scoring may introduce transparency and fairness concerns for suppliers; firms should document criteria and remediation pathways to avoid adverse externalities.
• Limitations & external validity:
  • Results come from one firm and 100 suppliers over two years; external generalizability is limited. Small/high-risk class size was mitigated by bootstrap resampling, but larger multi-firm datasets would strengthen inference.
  • The model is predictive, not causal — interventions based on model outputs should be evaluated (A/B tests, pilot deployments) to quantify true causal benefits and avoid perverse incentives.
• Research directions for AI economics:
  • Integrate predictive risk models into structural and dynamic economic models of supply chains (inventory, contract choice, entry/exit).
  • Evaluate equilibrium effects: how repeated use of predictive scores changes supplier behavior, entry, investment in quality/reliability, and market concentration.
  • Cost-benefit and counterfactual analyses: quantify aggregate welfare impacts of widespread adoption (resilience vs. potential supplier exclusion).
  • Explore multi-firm and cross-industry datasets to assess heterogeneity in model performance and economic value.

Summary takeaway: An explainable XGBoost-based supplier-risk framework delivered strong predictive performance and actionable explanations in a production procurement setting, with clear economic implications for risk pricing, sourcing strategy, and procurement governance — but broader external validation and causal evaluation are needed to quantify system-wide economic effects.