Consumer legal AIs confidently get Indian contract law wrong: a 60-case audit finds large high-confidence error rates (Meta AI 31.7%, Perplexity 15.0%, ChatGPT 6.7%) especially on post-2018 Specific Relief issues, while 71% of law students report no formal training in ethical AI use, leaving courts and practitioners exposed to hallucinated citations.

Main Finding

The paper uncovers a domain-specific overconfidence in legal LLMs—termed the "inertia of confidence"—where models return incorrect legal conclusions with near-maximal certainty. This behavior appears linked to a hypothesized algorithmic bias called "precedent overfitting," in which models overweight historical pre-amendment case law versus recent statutory changes. In a dual audit (technical + student survey) the authors quantify the phenomenon using a new metric, the High-Confidence Error Rate (HCER): Meta AI showed HCER = 31.7%, Perplexity = 15.0%, and ChatGPT (GPT-5.2) = 6.7% on a specialized 60-case Indian contract law battery; errors concentrate on scenarios testing the Specific Relief (Amendment) Act, 2018.

Key Points

• Inertia of confidence: LLMs often maintain near-maximum self-reported confidence (scale 1–10) while producing incorrect legal verdicts, especially where modern statutes override voluminous historical precedent.
• Precedent overfitting: Proposed algorithmic bias where past judicial corpus (pre-amendment decisions) dominates model outputs, causing reversion to outdated law despite training data including newer statutes.
• HCER (High-Confidence Error Rate): Percentage of model responses that are incorrect and reported with high confidence (threshold used: ≥9/10). HCER results (60-case battery, consumer interfaces):
  • Meta AI: 31.7% HCER; mean confidence ≈ 9.1/10
  • Perplexity (Sonar, RAG): 15.0% HCER
  • ChatGPT (GPT-5.2): 6.7% HCER
• Performance pattern: Models performed well (80–100% accuracy) on core doctrinal controls (offers, acceptance, capacity), but accuracy dropped markedly for Specific Relief / 2018 Amendment cases (ChatGPT 70%, Perplexity 60%, Meta AI 50%).
• Human side (survey of N = 380 Indian LLB students):
  • Many students lack formal AI-ethics/training (71.1% reported none).
  • Students with prior encounters of fabricated citations reported higher verification frequency (mean verification score 4.2/5) than students with no such encounters (2.8/5) — evidence of reactive rather than proactive verification.
  • 81.6% aware that submitting hallucinated cases can lead to contempt-of-court, yet institutional preparation is weak.
• Policy recommendations: adversarial legal research pedagogy, mandatory source-grounded verification architectures for legal AI, and institutional training to avoid systemic professional negligence.

Data & Methods

• Dual-phase socio-technical audit:
  • Phase I — Algorithmic audit
    • Black-box evaluation via consumer-facing interfaces (to reflect real user exposure):
      • ChatGPT (GPT-5.2) via OpenAI web UI (default parameters)
      • Meta AI via WhatsApp consumer interface
      • Perplexity AI (Sonar) via free-tier web interface (RAG/live retrieval)
    • 60-case "judicial agent" battery focused on Indian Contract Act, 1872 and Specific Relief (Amendment) Act, 2018:
      • Six topical groups (10 cases each): Offer/Acceptance; Capacity/Consent; Consideration/Privity; Discharge/Frustration; Damages/Terms; Specific Relief & 2018 Amendments (trap cases).
      • Factual matrices anonymized to avoid direct retrieval of landmark case names.
    • Standardized prompt: models instructed to act as a senior jurist; required outputs: definitive verdict, supporting statutory authority, self-assessed confidence (1–10).
    • HCER defined as proportion of incorrect verdicts delivered with confidence ≥9 (manual aggregation and evaluation of correctness against ground truth by authors).
    • Temporal reasoning probe: cases constructed to stress retrospective vs prospective application of the 2018 Amendment (e.g., uncertainty after Katta Sujatha Reddy developments).
    • Bias control: uniform prompts, anonymization, black-box consumer interfaces.
  • Phase II — Human survey
    • Cross-sectional primary survey of N = 380 LLB students (purposive convenience sampling across institution types and years).
    • Instrument: 10-question Google Form covering usage, hallucination encounters, institutional training, verification habits, and career anxiety.
    • Key variables: Hallucination Exposure, Verification Frequency (Likert), Institutional Training Status (binary), Liability Awareness.
    • Ethical measures: informed consent, anonymity, no identifying data collected.
• Limitations noted by authors:
  • Black-box design prevents mechanistic attribution (cannot inspect model weights, retrieval rankings).
  • Prompt framing (judicial persona) may induce response pressure and affect self-reported confidence.
  • Non-probability survey sampling limits generalizability.
  • HCER threshold choice (≥9) is operational and may vary in other studies.

Implications for AI Economics

• Market for verification and provenance tools: High HCER on legal tasks, especially in statutory-update contexts, creates strong economic demand for verifiable, source-grounded RAG systems, citation provenance services, and third-party legal-AI auditing—supporting investment and competition in "trustworthy legal AI" startups and modules.
• Liability externalities and cost of error: Overconfident AI errors impose negative externalities (court sanctions, malpractice risk). This raises expected liability costs for legal practitioners and institutions, which will be internalized through higher fees, mandatory verification steps, and professional indemnity insurance premiums. Pricing models for legal services are likely to adjust upward to cover verification overhead.
• Information asymmetry & reputation risk: AI outputs that appear authoritative but are incorrect exacerbate information asymmetry between providers and consumers. Firms that can credibly certify low HCER (via audits, certifications) will gain competitive advantage; reputational risk from high-profile AI-induced errors will increase the value of certified, audited models.
• Labor and productivity effects: Short-term productivity gains from AI adoption could be offset by verification burdens and litigation risk (a "productivity–liability tradeoff"). Demand for skilled verification labor (paralegals, AI-auditors) will rise; complementary human capital may become more valuable than raw automation.
• Regulatory and certification markets: Findings strengthen the case for mandatory auditing standards, HCER-like risk metrics, model disclosure requirements, and certification regimes for legal-AI tools—creating regulatory compliance costs but also new auditing service markets.
• Education and human capital investment: Weak institutional training implies a market failure in complementarities between AI and users. Investment in adversarial legal-research pedagogy and mandated AI literacy creates positive externalities and reduces systemic risk; universities and law firms will need to invest in curricula and training programs.
• Insurance and contract design: Insurers will develop contract terms and pricing reflecting HCER exposure and provenance guarantees. Contracts between legal-AI vendors and buyers will likely include warranties, indemnities, and audit clauses—shaping platform economics and bargaining power.
• Recommendations for economic policy and adoption:
  • Encourage procurement of legal-AI systems with provable provenance and measurable HCER; tie public procurement (e.g., court tools) to audit certification.
  • Subsidize training and verification infrastructure in public-interest legal services to preserve access-to-justice gains while managing error risk.
  • Support creation of market standards (HCER reporting, provenance APIs) to reduce search and information costs and accelerate efficient adoption.

Overall, the paper highlights a measurable, model-level risk (high-confidence legal errors concentrated where statutes change) with direct economic consequences for legal service markets, product-design incentives, regulatory compliance costs, and the structure of labor complementarity in an AI-augmented legal economy.