← Papers

Public contracting data let ML explain much of the variation in Air Force contract log-prices (XGBoost R² ≈ 0.71), but median dollar-scale errors of about 51% leave forecasts too noisy for autonomous procurement decisions; richer proprietary inputs or NLP-derived technical features could materially improve practical utility.

A Machine Learning Framework for Price Estimation in Air Force Acquisition

Kefallinos, Paola, O'brien, Cuyler · August 21, 2026 · DSpace@MIT (Massachusetts Institute of Technology)

openalex correlational medium evidence 7/10 relevance Summary only summary available; pdf_status=error Source PDF

Structured author observations

Linked only from stored provider relations; the raw author line above is never matched by name.

OpenAlex

Latest observation: August 24, 2026

Kefallinos, Paola provider ID
O'brien, Cuyler provider ID

Machine-learning models trained on public Air Force contracting records can explain a large share of log-price variation (XGBoost test R² ≈ 0.707) but produce median dollar-scale errors around 51%, too large for immediate operational procurement adoption.

Citation observations

Cumulative provider counts captured on specific dates; providers are never combined.

0 cumulative citations

OpenAlex · Observed August 24, 2026

View corpus context

Price estimation in the Air Force is often bottlenecked by data that is proprietary, incomplete, or unavailable at the time planning begins, leaving pricing teams reactive and acquisition timelines at risk. This work investigates whether machine learning models trained on public contracting data can produce estimates accurate enough to support acquisition planning. The data pipeline processed Air Force contracts spanning 2010-2025 sourced from USASpending.gov, applying data cleaning and feature engineering methods that reduced columns by 84% and rows by 53%. The modeling pipeline evaluated two regression models (XGBoost and Random Forest), tuned hyperparameters, and identified t he o ptimal f eature s et through permutation importance and elbow analysis, resulting in a 34% reduction of the encoded feature space. The optimally configured XGBoost model trained on the reduced feature set achieved a test R 2 of 0.7068, mean absolute error of 0.3137, and root mean squared error of 0.4302 on the log scale, and a median absolute percent error of 51.3% on the dollar scale. While these results fall short of operational adoption thresholds, they demonstrate the viability of a public contracting data pipeline to support a machine learning price estimation engine, laying the groundwork for the next generation of acquisition automation. With continued investment, this work can be extended with AI integrations to enrich model inputs and produce interactive outputs, generating more accurate estimates, reducing programmatic risk, and compressing acquisition timelines.

Summary

Main Finding

Machine learning models trained on public contracting data can produce moderately accurate price estimates for Air Force acquisitions (best model: XGBoost, test R² = 0.7068 on log price), but errors on the dollar scale (median absolute percent error ≈ 51.3%) remain too large for immediate operational adoption. The work demonstrates a viable public-data pipeline and modeling approach that, with additional data and AI-driven input enrichment, could materially reduce information shortfalls in acquisition planning.

Key Points

Data source and scope: Air Force contract records from USASpending.gov covering 2010–2025.
Data preprocessing: cleaning and feature engineering reduced columns by 84% and rows by 53%.
Feature selection: permutation importance + elbow analysis produced a 34% reduction in encoded feature space.
Models evaluated: XGBoost and Random Forest; hyperparameters tuned.
Best model performance (on log-transformed price):
- XGBoost test R² = 0.7068
- Mean absolute error (log scale) = 0.3137
- RMSE (log scale) = 0.4302
Dollar-scale performance: median absolute percent error = 51.3%, which is below operational accuracy thresholds for adoption.
Conclusion: public contracting data are sufficiently informative to build a baseline price-estimation engine, but additional inputs or methods are required to meet procurement decision standards.

Data & Methods

Source: USASpending.gov contracting records for Air Force obligations, 2010–2025.
Preprocessing:
- Extensive cleaning (deduplication, handling missingness, normalization).
- Feature engineering (categorical encoding, derived variables).
- Substantial dimensionality reduction: columns −84%, rows −53%.
Feature selection:
- Used permutation importance to rank predictors.
- Applied elbow analysis to select an “optimal” subset, reducing encoded features by 34%.
Modeling pipeline:
- Two tree-based regressors compared: XGBoost (gradient-boosted trees) and Random Forest.
- Hyperparameter tuning (search/validation) for each model.
- Target variable modeled on log scale; evaluation reported on both log and dollar scales.
Evaluation metrics:
- R², MAE, RMSE on log-transformed prices.
- Median absolute percent error reported on dollar scale for operational interpretability.

Implications for AI Economics

Value of public data: Public contracting records can reduce information asymmetry in defense procurement by providing a baseline for expected prices, lowering search and monitoring costs for planners.
Limits of public-only models: A 51% median absolute percent error implies significant residual uncertainty; proprietary or context-rich data (e.g., technical specs, program schedules, vendor bids) are likely high-value complements that could sharply improve estimates.
Investment trade-offs: Additional investment in data integration (commercial/proprietary feeds), richer feature engineering (NLP on statements of work), and advanced modeling (uncertainty quantification, Bayesian calibration, ensembles) may yield outsized returns by reducing acquisition delays and rework.
Market and incentive effects: More-transparent pricing benchmarks could influence supplier pricing behavior (potentially compressing rents), improve bargaining positions for program offices, and encourage vendors to participate in more competitive offerings.
Operational adoption considerations:
- Accuracy thresholds: Current errors are insufficient for autonomous decision-making but useful for early-stage planning, risk flagging, and scenario analysis.
- Decision support design: Human-in-the-loop systems, uncertainty intervals, and interactive tools would increase trust and practical utility.
Path forward for AI integration: Using ML to enrich inputs (e.g., extracting technical attributes with NLP, augmenting datasets via data fusion), providing probabilistic forecasts, and delivering interactive/ explainable outputs can compress acquisition timelines and reduce programmatic risk if paired with continued data investment and evaluation against operational benchmarks.

Assessment

Paper Typecorrelational Evidence Strengthmedium — The paper provides robust within-sample and held-out predictive evidence (test R² reported, log-scale error metrics, dollar-scale median absolute percent error). However, it is a predictive, observational study without causal identification, external validation, or robustness checks against alternative data splits and potential leakage, and dollar-scale errors remain large for operational use. Methods Rigormedium — The authors apply sensible preprocessing, feature engineering, permutation-based feature selection, hyperparameter tuning, and compare two tree-based models with held-out test evaluation and both log- and dollar-scale metrics. Missing are details on train/validation/test split strategy, potential leakage from feature selection, external replication/validation, uncertainty quantification, and comparisons with non-tree baselines or simpler benchmarks. SamplePublic US federal contracting records for Air Force obligations drawn from USASpending.gov covering 2010–2025; records were cleaned, deduplicated, and feature-engineered (columns reduced by 84%, rows reduced by 53%); target is contract price (modeled on log scale); modeling compares XGBoost and Random Forest with hyperparameter tuning and held-out test evaluation. Themesproductivity adoption GeneralizabilityLimited to Air Force contracts reported on USASpending.gov — may not generalize to other services, agencies, or private-sector procurements., Excludes classified, proprietary, or non-public bidding data that likely contain high-value signals (technical specs, vendor bids)., Model performance may degrade for novel programs or tail contracts not well represented in historical data., Time heterogeneity (policy, inflation, contracting practices 2010–2025) may limit out-of-sample validity without temporal adjustment or recalibration., Feature engineering and selection choices may not transfer across contract types or classification schemes.

Claims (9)

Claim	Direction	Outcome	Confidence & Evidence	Details
An XGBoost model trained on Air Force public contracting data achieved a test R² of 0.7068 for log-transformed acquisition prices. Decision Quality	positive	Accuracy of log-transformed acquisition price estimates	Reading fidelity high Study strength high	test R² = 0.7068 0.5
The XGBoost model had a mean absolute error of 0.3137 and an RMSE of 0.4302 on the log-price scale. Error Rate	positive	Log-scale prediction error for acquisition prices	Reading fidelity high Study strength high	MAE = 0.3137; RMSE = 0.4302 0.5
On the dollar scale, the model's median absolute percent error was approximately 51.3%. Error Rate	negative	Dollar-scale acquisition price estimation error	Reading fidelity high Study strength high	median absolute percent error ≈ 51.3% 0.5
The reported dollar-scale error is below the accuracy threshold required for immediate operational adoption. Adoption Rate	negative	Suitability of price estimates for operational procurement adoption	Reading fidelity high Study strength medium	not reported 0.3
Cleaning and feature engineering reduced the dataset by 84% in column count and 53% in row count. Other	negative	Dataset dimensionality and retained observations after preprocessing	Reading fidelity high Study strength medium	columns −84%; rows −53% 0.3
Permutation importance combined with elbow analysis reduced the encoded feature space by 34% while selecting the model input subset. Other	negative	Number of encoded predictors retained for modeling	Reading fidelity high Study strength medium	34% reduction 0.3
Public Air Force contracting records contain sufficient information to support a baseline price-estimation engine for acquisitions. Decision Quality	positive	Feasibility of building a baseline acquisition price-estimation system	Reading fidelity high Study strength medium	test R² = 0.7068 0.3
The public-data model is more suitable for early-stage planning, risk flagging, and scenario analysis than for autonomous procurement decisions. Task Allocation	mixed	Appropriate use of ML price estimates in acquisition decision processes	Reading fidelity high Study strength medium	median absolute percent error ≈ 51.3% 0.3
Additional context-rich or proprietary inputs, such as technical specifications, program schedules, and vendor bids, are likely to improve acquisition price estimates. Decision Quality	positive	Accuracy of acquisition price estimates after enriching the input data	Reading fidelity high Study strength speculative	not reported 0.05