← Papers

Conformalised uncertainty lets aggregators bid reliably: merging Monte Carlo dropout with conformal prediction produces P90-compliant forecasts of prosumer flexibility and cuts overbidding risk in a Danish reserve-market simulation, securing up to 70% of perfect-information profits compared with overconfident baselines.

Calibrated uncertainty quantification for prosumer flexibility aggregation in ancillary service markets

Yogesh Pipada Sunil Kumar, S. Ali Pourmousavi, Jon A. R. Liisberg, Julian Lesmos-Vinasco · January 21, 2026

arxiv descriptive low evidence 7/10 relevance Full text usable extracted full text Source PDF

Structured author observations

Linked only from stored provider relations; the raw author line above is never matched by name.

Arxiv

Latest observation: July 29, 2026

Yogesh Pipada Sunil Kumar unresolved corpus identity
S. Ali Pourmousavi unresolved corpus identity
Jon A. R. Liisberg unresolved corpus identity
Julian Lesmos-Vinasco unresolved corpus identity

Semantic Scholar

Latest observation: August 24, 2026

Yogesh Pipada provider ID
Sunil Kumar provider ID
S. Pourmousavi provider ID
Jon A. R. Liisberg provider ID
Julian Lesmos-Vinasco provider ID

Combining Monte Carlo dropout with conformal prediction yields calibrated, finite-sample prediction intervals for aggregated prosumer flexibility that satisfy P90 reliability requirements and, when used in bidding simulations for a Danish reserve market, substantially reduce overbidding risk and recover up to 70% of perfect-information profit compared with overconfident MCD alone.

Citation observations

Cumulative provider counts captured on specific dates; providers are never combined.

0 cumulative citations

Semantic Scholar · Observed August 24, 2026

View corpus context

Reliable forecasting of prosumer flexibility is critical for demand response aggregators participating in frequency controlled ancillary services market, where strict reliability requirements such as the P90 standard are enforced. Limited historical data, dependence on exogeneous factors, and heterogenous prosumer behaviour introduce significant epistemic uncertainty, making deterministic or poorly calibrated probabilistic models unsuitable for market bidding. This paper proposes the use of scalable uncertainty quantification framework that integrates Monte Carlo dropout (MCD) with conformal prediction (CP) to produce calibrated, finite sample prediction intervals for aggregated prosumer flexibility. The proposed framework is applied to a behind-the-meter aggregator participating in the Danish manual frequency restoration reserve capacity market. A large-scale synthetic dataset is generated using a modified industry-grade home energy management system, combined with publicly available load, solar, price, activation and device-level data. The resulting machine learning surrogate model captures aggregate prosumer price responsiveness and provides uncertainty-aware estimates suitable for market bidding. Multiple multivariate CP strategies are evaluated and benchmarked against conventional MCD-based methods. Results show that standalone MCD systematically overestimates available flexibility and violates P90 compliance, whereas the proposed MCD-CP framework achieves reliable coverage with controlled conservatism. When embedded in aggregator bidding model, conformalised methods substantially reduce overbidding risk and achieve upto 70% of perfect-information profit while satisfying regulatory reliability constraints, providing practical, computationally efficient, and market-compliant solution for aggregator flexibility forecasting under uncertainty.

Summary

Main Finding

A hybrid uncertainty-quantification framework that combines Monte Carlo dropout (MCD) with split conformal prediction (CP) yields calibrated, finite-sample multivariate prediction intervals for aggregated prosumer flexibility. Embedded in an aggregator bidding model for the Danish manual frequency restoration reserve (mFRR) capacity market, the MCD–CP approach satisfies the regulator’s P90 reliability requirement while substantially reducing overbidding risk and recovering up to ~70% of the profit attainable under perfect information. Standalone MCD was found to be miscalibrated (overoptimistic), systematically overestimating available flexibility and violating P90.

Key Points

Problem context
- Behind-the-meter aggregators bid prosumer flexibility into ancillary service markets (Danish mFRR in study) where a P90 availability standard limits the allowed probability of shortage to 10%.
- Forecast uncertainty is dominated by epistemic factors (limited behavioral data, dependence on exogenous forecasts), making calibrated probabilistic forecasts essential for reliable bidding.
Methodological insight
- MCD provides a computationally scalable approximation of epistemic uncertainty by keeping dropout active at inference and sampling multiple forward passes.
- Conformal prediction (split CP) is used to correct MCD’s miscalibration and produce prediction regions with finite-sample marginal coverage guarantees under exchangeability.
- Multivariate (multi-time-step) CP is applied so that the resulting hyper-rectangle bounds (lower l and upper u) can be used conservatively to enforce joint chance constraints in the aggregator’s optimization.
Optimization & business model
- The aggregator maximizes expected profit from capacity bids (up/down regulation) and selects prosumer incentive prices via a revenue-sharing parameter β (discretized into scenarios).
- Original decision-dependent chance-constrained problem is non-convex; the study fixes incentive prices ex ante and uses calibrated flexibility estimates to linearize and tractably solve the bidding problem.
- Conservatively enforcing the P90 chance constraint is done by substituting calibrated lower/upper bounds for the uncertain flexibility variables.
Empirical findings
- Standalone MCD intervals are overconfident and lead to frequent P90 violations and overbidding.
- MCD–CP achieves reliable marginal coverage and controlled conservatism: prediction intervals are wider where needed but calibrated.
- When used in the bidding model, conformalised forecasts reduce exposure to penalties and enable economically viable participation—up to ~70% of perfect-information profit while satisfying P90.
Practical advantages
- The hybrid method is computationally efficient compared with full Bayesian neural networks (BNNs) and scales to multivariate, multi-step forecasting needed for market bidding.
- The approach is model-agnostic (CP wraps around any predictive model that outputs MCD samples).

Data & Methods

Prediction model
- Neural network with dropout (trained on historical/synthetic prosumer data).
- Monte Carlo dropout (MCD): S stochastic forward passes at inference to form an empirical predictive distribution (captures epistemic uncertainty).
Calibration
- Split conformal prediction: compute conformity scores on a held-out calibration set, pick quantile level adjusted for sample size to guarantee marginal coverage 1 − α, and construct conformal region ξ.
- For multivariate outputs, conformal region ξ is bounded by an axis-aligned hyper-rectangle Bξ = {l ≤ y ≤ u}; these bounds are used conservatively in downstream chance-constrained optimization.
Conformity scores
- The paper evaluates several multivariate conformity scores (details given in full text); the chosen scores affect geometry and tightness of prediction regions.
Aggregator optimisation
- Original problem: decision-dependent chance-constrained maximization of capacity bid profit (bilinear in incentives and reserved capacity).
- To make tractable: fix incentive scenarios (via discretized revenue share β), obtain calibrated flexibility estimates ˆy (from MCD–CP), and solve a linear program for each scenario to find the best profit.
- P90 constraint enforced by ensuring reserved capacity ≤ calibrated lower bound (for up-regulation) or ≥ calibrated upper bound (for down-regulation) as appropriate.
Data generation
- A large-scale synthetic dataset was generated using a modified industry-grade home energy management system (HEMS) from Watts A/S, combined with public datasets for:
  - Load and solar generation profiles,
  - Market prices and activation signals,
  - Device-level size distributions for residential batteries and PV.
- The synthetic dataset is used to train, calibrate, and evaluate the MCD–CP pipeline under realistic market and HEMS behavior.
Evaluation
- Benchmarked standalone MCD vs MCD–CP for calibration (P90 compliance), interval size, and economic outcomes (profit vs perfect-information upper bound).
- Computation time and scalability compared qualitatively against BNN approaches.

Implications for AI Economics

Risk-aware market participation
- Properly calibrated UQ is essential for market actors (aggregators) to participate without undue financial risk or regulatory non-compliance. The MCD–CP combination offers a practical pathway to meet reliability standards (e.g., P90) while retaining economic viability.
Trade-off: conservatism vs profit
- Enforcing marginal coverage via CP introduces conservatism (wider intervals) compared with uncalibrated predictions; however, this conservatism reduces penalty risk and can still recover a substantial fraction of the theoretical maximum profit (~70% in the case study). Market designers and aggregators must weigh reliability requirements against potential lost opportunity from conservative bids.
Policy & market design
- Regulators setting availability standards (P90, etc.) should consider the capabilities and limits of forecast calibration methods. Techniques like conformal calibration can lower barriers to entry by providing formally guaranteed reliability even under limited data.
Methodological recommendation
- For decision-dependent, multivariate forecasting problems in economics and markets, scalable approximate-Bayesian methods (MCD) corrected by distribution-free calibration (CP) are a promising compromise between computational cost and statistical guarantees, compared to full BNNs.
Limitations & open directions (economic modeling consequences)
- Marginal (not conditional) coverage: CP guarantees hold on average over the calibration distribution. Conditional coverage (guarantees for particular covariate regions) is not ensured; aggregators may still face concentrated undercoverage in rare regimes, which can have outsized economic consequences.
- Exchangeability assumption: CP requires calibration/train data exchangeability; distribution shift (nonstationary markets, changing prosumer behavior) can erode coverage and economic performance.
- Simplifying assumptions in optimization: incentive prices were fixed ex ante (scenario-based), and distribution grid constraints were ignored. Joint optimisation of incentives and bids (decision-coupled calibration) and incorporation of network constraints could change economic outcomes.
- Synthetic-data validation: results are promising but warrant replication on real-world aggregator deployments and live market data to confirm calibration and profitability in practice.
Research and policy suggestions
- Empirical field trials with real aggregator portfolios to validate MCD–CP under nonstationarity and activation realizations.
- Development of conditional or covariate-conditional conformal methods to reduce tail/regime-specific undercoverage risk.
- Integration of grid constraints and dynamic incentive optimization into the calibration-aware bidding framework to assess system-level welfare impacts.
- Consideration by regulators of the statistical tools used by market participants when specifying availability standards and penalties.

Overall, the paper demonstrates a practical, computationally tractable method to produce calibrated multivariate forecasts for prosumer flexibility and to incorporate them into an aggregator’s market bidding problem—striking a useful balance between reliability guarantees and economic performance.

Assessment

Paper Typedescriptive Evidence Strengthlow — Findings are derived from machine-learning experiments on a large synthetic dataset and market-simulation experiments rather than real-world field trials or natural experiments; while internal validation (benchmarks, multiple CP strategies) is strong, external validity is limited by synthetic data, surrogate-model assumptions, and untested real-world deployment risks. Methods Rigormedium — The paper uses a principled uncertainty-quantification pipeline (Monte Carlo dropout combined with multiple multivariate conformal prediction strategies), an industry-grade home energy management system to generate realistic synthetic data, and benchmarking against baseline methods and a bidding-model economic evaluation; however, reliance on synthetic data, potential model misspecification, and limited discussion of sensitivity to exogenous forecasting errors reduce methodological rigor relative to a field-evaluation. SampleA large-scale synthetic dataset representing behind-the-meter prosumers for a Danish aggregator created by a modified industry-grade home energy management system, augmented with publicly available load, solar generation, price, activation, and device-level data; aggregated prosumer flexibility is modeled and used to train an ML surrogate for price responsiveness and to simulate bidding in the Danish manual frequency restoration reserve capacity market (no real aggregator field deployment reported). Themesadoption governance GeneralizabilitySynthetic data may not capture full heterogeneity and strategic behaviour of real prosumers, Results calibrated to the Danish manual frequency restoration reserve market and its regulatory P90 requirement may not generalize to other markets or regulations, Surrogate-model assumptions (HEMS parameterization, device models, price responsiveness) limit transferability to different household technologies or tariff structures, Performance depends on quality of exogenous forecasts (solar, load, price); robustness to distributional shift/time variation not fully established, Scalability tested in simulation but not validated in live market operations with network/aggregation constraints

Claims (11)

Claim	Direction	Outcome	Confidence & Evidence	Details
Reliable forecasting of prosumer flexibility is critical for demand response aggregators participating in frequency controlled ancillary services market, where strict reliability requirements such as the P90 standard are enforced. Regulatory Compliance	positive	necessity of reliable prosumer flexibility forecasts to meet P90 reliability requirements in frequency ancillary services markets	Reading fidelity high Study strength medium	not reported 0.18
Limited historical data, dependence on exogeneous factors, and heterogenous prosumer behaviour introduce significant epistemic uncertainty, making deterministic or poorly calibrated probabilistic models unsuitable for market bidding. Decision Quality	negative	suitability of deterministic/poorly calibrated probabilistic models for market bidding under epistemic uncertainty	Reading fidelity high Study strength medium	not reported 0.18
We propose a scalable uncertainty quantification framework that integrates Monte Carlo dropout (MCD) with conformal prediction (CP) to produce calibrated, finite sample prediction intervals for aggregated prosumer flexibility. Output Quality	positive	calibration and finite-sample validity of prediction intervals for aggregated prosumer flexibility	Reading fidelity high Study strength medium	not reported 0.18
The proposed framework is applied to a behind-the-meter aggregator participating in the Danish manual frequency restoration reserve capacity market. Adoption Rate	null_result	practical application of the framework to a specific market and aggregator setting	Reading fidelity high Study strength low	not reported 0.09
A large-scale synthetic dataset is generated using a modified industry-grade home energy management system, combined with publicly available load, solar, price, activation and device-level data. Other	null_result	creation of a large-scale synthetic dataset combining simulator and public data	Reading fidelity high Study strength medium	not reported 0.18
The resulting machine learning surrogate model captures aggregate prosumer price responsiveness and provides uncertainty-aware estimates suitable for market bidding. Decision Quality	positive	ability of surrogate model to capture aggregate price responsiveness and produce uncertainty-aware estimates for bidding	Reading fidelity high Study strength medium	not reported 0.18
Multiple multivariate CP strategies are evaluated and benchmarked against conventional MCD-based methods. Other	null_result	comparative performance of CP strategies versus MCD-based methods	Reading fidelity high Study strength low	not reported 0.09
Standalone MCD systematically overestimates available flexibility and violates P90 compliance. Regulatory Compliance	negative	bias (overestimation) in available flexibility estimates and non-compliance with P90 coverage	Reading fidelity high Study strength medium	not reported 0.18
The proposed MCD-CP framework achieves reliable coverage with controlled conservatism. Regulatory Compliance	positive	prediction interval coverage and conservatism of MCD-CP framework	Reading fidelity high Study strength medium	not reported 0.18
When embedded in aggregator bidding model, conformalised methods substantially reduce overbidding risk and achieve upto 70% of perfect-information profit while satisfying regulatory reliability constraints. Firm Revenue	positive	aggregator profit (relative to perfect-information profit), overbidding risk reduction, and regulatory constraint satisfaction	Reading fidelity high Study strength medium	upto 70% of perfect-information profit 0.18
The proposed approach provides a practical, computationally efficient, and market-compliant solution for aggregator flexibility forecasting under uncertainty. Organizational Efficiency	positive	computational efficiency, practicality, and market compliance of the proposed forecasting solution	Reading fidelity medium Study strength low	not reported 0.05