Refinement loops often hurt unless the model almost never introduces new errors: iterative self-correction degrades accuracy for many LLMs unless EIR is near zero, but a lightweight 'verify-first' prompt can reduce EIR and restore gains on affected models.

Main Finding

Iterative LLM self-correction should be treated as a control decision: model the process as a two-state Markov feedback loop (Correct ↔ Incorrect) and only iterate when the measured error-dynamics satisfy a simple equilibrium diagnostic. Concretely, iterate only if ECR / EIR > Acc / (1 − Acc). Empirically, there is a very sharp near-zero Error Introduction Rate (EIR) threshold (≲0.5%) separating beneficial from harmful self-correction. Prompting (a “verify-first” style prompt) can reduce EIR and thereby causally prevent degradation; deeper capability gains (higher ECR) typically require training-level improvements.

Key Points

• Markov diagnostic:
  • Define EIR = P(change correct → incorrect next iter) and ECR = P(change incorrect → correct next iter).
  • Equilibrium condition (no net gain): ECR / EIR = Acc / (1 − Acc).
  • Steady-state accuracy π = ECR / (EIR + ECR). If Acc > π, iterative refinement will reduce accuracy.
  • Convergence geometric rate: |Acc(k) − π*| ∝ (1 − EIR − ECR)^k.
• Empirical threshold:
  • Across 7 models × 3 datasets (GSM8K, MATH, StrategyQA), beneficial self-correction occurred only when EIR ≲ 0.5%.
  • Non-degrading / beneficial examples: o3-mini (+3.4 pp, EIR = 0%), Claude Opus 4.6 (+0.6 pp, EIR ≈ 0.2%), o4-mini (≈0 pp, EIR ≈ 0.2%).
  • Several models degraded under repeated refinement (e.g., GPT-4o-mini: −6.2 pp; GPT-5: −1.8 pp).
• Causal intervention via prompting:
  • A “verify-first” prompt reduced GPT-4o-mini’s EIR from ~2% → 0% and flipped a −6.2 pp degradation into +0.2 pp (paired McNemar p < 1e−4). The same prompt had little effect on models already below the threshold.
• Adaptive Self-Correction (ASC):
  • ASC combines instance confidence stopping (stop when self-assessed confidence ≥ τ) and batch-level monitoring (stop when dEIR ≥ dECR).
  • ASC correctly halted harmful refinement for GPT-4o-mini but the confidence-elicitation prompt itself reduced baseline accuracy by 3.8 pp (a cost).
• Baselines and alternatives:
  • Self-Consistency (k=3 independent samples) with GPT-4o-mini achieved 93.4% (∆ +2.2 pp vs single-shot) and outperformed sequential 3-iteration refinement (which degraded).
  • Structured self-feedback methods (Self-Refine, Reflexion) did not reliably overcome EIR/ECR imbalances.

Data & Methods

• Models evaluated (7): GPT-4o-mini, GPT-4.1, Claude Sonnet 4, GPT-5, Claude Opus 4.6, o3-mini, and o4-mini (validation).
• Tasks / datasets: GSM8K (500 problems), MATH (400), StrategyQA (200). Primary analysis focused on GSM8K for extraction clarity.
• Experimental procedure:
  • Iterative refinement: responses r(0) = M(q), r(k+1) = M(q, r(k), prefine).
  • Extract exact-match answers and compute Acc(k) per iteration.
  • Compute per-iteration EIR and ECR empirically across the problem set.
  • Evaluate verify-first prompt ablation and ASC stopping rule.
  • Statistical tests: paired McNemar and paired-bootstrap CIs for accuracy changes.
• Key quantitative results (GSM8K, example snapshots):
  • GPT-4o-mini: baseline 91.2% → iter4 85.0% (−6.2 pp); EIR rose from 1.3% → 3.8%.
  • o3-mini: baseline 93.2% → iter4 96.6% (+3.4 pp); EIR = 0% and high ECR (e.g., 44.1% at 0→1).
  • Verify-first on GPT-4o-mini: EIR → 0% across iterations; iter4 accuracy +0.2 pp vs −6.2 pp standard.

Implications for AI Economics

• Deployment decision rule: treat self-correction as a measurable control variable. Before enabling iterative refinement in production, measure EIR and ECR on a calibration set. Only enable iteration when ECR/EIR > Acc/(1−Acc). This avoids systematic accuracy loss and wasted compute.
• Compute vs accuracy trade-offs:
  • Unconditional sequential refinement can be compute-inefficient and damaging for many models; parallel independent sampling (e.g., Self-Consistency) can yield higher accuracy per API call by breaking sequential error correlation.
  • ASC and verify-first prompting are low-cost interventions that can prevent degradation (reduce EIR) but may incur other costs (e.g., confidence elicitation reduced accuracy by 3.8 pp in this study). Account for these secondary costs in expected-utility calculations.
• Product & engineering choices:
  • Prompt engineering (verify-first) is a cheap, actionable lever to suppress EIR (the “stability margin”) and thus avoid harmful refinements—good short-term investment for systems engineering.
  • To obtain true improvement (higher steady-state π*), invest in capability-level changes that raise ECR reliably (e.g., targeted training, RL with verifiable rewards, or tool-based verification).
  • For high-baseline-accuracy models, small EIRs on the large correct pool can swamp correctable error gains—so decision-makers should prioritize reducing EIR over naive extra iterations.
• R&D and cost allocation:
  • Two-tier view for R&D budgeting: (1) investing in controller-level fixes (prompt design, verification protocols, policy/stop rules) to suppress EIR; (2) investing in model capability (training, RL, tools) to raise ECR. Controller-level work is cheaper and often sufficient to prevent harm; capability-level work is required for net improvement.
• Operational risk and incentives:
  • Over-reliance on intrinsic self-critique without monitoring EIR/ECR can produce silent degradations in deployed agents. Contracts, SLAs, and monitoring dashboards should include EIR/ECR-style metrics and the equilibrium diagnostic.
  • For economic modeling of agent performance, incorporate per-iteration expected accuracy change and compute cost into objective functions (expected accuracy per dollar or expected utility).
• Limitations to consider for economic decisions:
  • Results are concentrated on mathematical/reasoning benchmarks and exact-match extractors; other task types (open-ended generation, dialog) may have different dynamics.
  • EIR/ECR estimates depend on the extractor and dataset; calibration and periodic re-measurement are recommended when models, prompts, or data distributions change.

Practical short checklist for practitioners/economists: 1. Calibrate EIR and ECR on a held-out calibration set identical to the production distribution. 2. Apply the equilibrium test: allow iteration only if ECR/EIR > Acc/(1−Acc). 3. If EIR > threshold, try verify-first or other prompt-level controllers to suppress EIR before enabling iterative refinement. 4. Consider compute-equivalent alternatives (independent sampling/self-consistency) when sequential refinement is predicted to be harmful. 5. Monitor EIR/ECR over time; reallocate investment from controller-level fixes to capability-level training when EIR is reliably low but ECR remains the bottleneck.