Leading multimodal AI systems struggle to detect greenwashing in Polish ESG reports, frequently flagging high-performing firms as more deceptive and showing virtually no agreement across models; current off‑the‑shelf tools conflate polished sustainability communication with fraudulent intent and lack the contextual grounding for reliable ESG assurance.

Main Finding

Multimodal LLMs (ChatGPT 5.1, Claude 4.5 Sonnet, Gemini 2.5 Flash) failed to reliably detect visual greenwashing in non-English (Polish) ESG reports. Rather than showing the expected negative correlation between substantive ESG performance and AI-flagged greenwashing, the tools often flagged higher-performing firms more heavily. Across Environmental, Social, and Governance dimensions the models produced inconsistent and contradictory judgments, and inter-model agreement was effectively nil (Krippendorff’s α ≈ 0). Overall, current general-purpose multimodal systems conflate polished communication with deceptive intent and lack the contextual grounding required for trustworthy ESG assurance.

Key Points

• Models evaluated: ChatGPT 5.1, Claude 4.5 Sonnet, Gemini 2.5 Flash.
• Sample: purposively selected n = 20 Polish ESG reports, benchmarked to ESRS-aligned performance scores from the national “Ranking ESG”.
• Procedure: a standardized auditing prompt applied identically across tools to assess visual greenwashing in reports.
• Main unexpected result: AI greenwashing scores were not negatively correlated with ESRS-aligned performance; high-performing firms frequently received higher greenwashing scores.
• Pillar-level analysis (E, S, G) produced inconsistent and often contradictory results across tools.
• Inter-tool reliability was effectively zero (Krippendorff’s α ≈ 0), indicating no reproducible agreement among systems.
• Interpretation: models appear to equate sophisticated visual/communicative quality with possible deception rather than assessing substantive performance or ESRS-conformant disclosures.
• Study cautions against using off-the-shelf multimodal LLMs for ESG assurance in non-English contexts.

Data & Methods

• Data: 20 Polish corporate ESG reports selected purposively to span a range of ESRS-aligned performance scores from the national “Ranking ESG”.
• Benchmark: ESRS-aligned performance scores from Ranking ESG served as the substantive-performance benchmark.
• Tools: three leading multimodal AI systems—ChatGPT 5.1, Claude 4.5 Sonnet, Gemini 2.5 Flash—accessed contemporaneously and prompted with the same standardized audit instruction.
• Prompting: a single, standardized visual-audit prompt was used to generate comparable greenwashing assessments across tools; outputs were mapped to a common scoring rubric.
• Analyses:
  • Correlation tests between AI greenwashing scores and ESRS-aligned performance scores (hypothesized negative correlation).
  • Dimensional breakdowns for Environmental, Social, and Governance pillars.
  • Reliability analysis across models using Krippendorff’s α.
• Key methodological limitations noted by authors:
  • Small, purposive, single-country (Poland) sample limits generalizability.
  • Non-English reporting poses known challenges for models trained predominantly on English data.
  • A single standardized prompt may not capture optimal prompting strategies per model.
  • Off-the-shelf models not fine-tuned on ESRS taxonomy or domain-specific labeled datasets.

Implications for AI Economics

• Market signals and capital allocation: If general-purpose multimodal AIs flag high-performing firms as greenwashing (false positives), automated screening tools could distort investor signals and lead to mispricing or misallocation of capital.
• Demand for domain-specific assurance tools: Findings increase economic value for specialized, ESRS-aligned multimodal models or classifiers trained on multilingual, audit-labeled corpora—creating an opportunity for vendors and service providers in AI assurance markets.
• Regulatory and compliance risk: Regulators and standard setters should be cautious about relying on off-the-shelf AI for ESG verification; poor reliability could produce regulatory errors and legal risks for firms and auditors.
• Productivity and cost of assurance: While automated detection promises cost savings, current tools may raise verification costs (manual review of false positives/negatives) until domain-tuned systems reach acceptable accuracy and reliability.
• Need for standardized benchmarks and public datasets: Economically efficient development of trustworthy tools requires labeled, multilingual, ESRS-aligned datasets and open benchmarks to reduce information asymmetries between AI vendors and users.
• Incentives and strategic behavior: If firms learn that polished presentation triggers automated scrutiny, they may alter communication strategies, potentially increasing compliance costs or gaming behavior; conversely, reliable detectors could deter greenwashing and align incentives toward substantive disclosure.
• Research agenda: Quantify macroeconomic impacts of erroneous AI-based ESG signals (false positives/negatives) on asset prices, cost of capital, and real investment; evaluate cost-benefit of investing in domain-specific model development vs. manual assurance.

Suggested next steps (technical and policy): - Develop multilingual, ESRS-aligned training datasets and human-labeled visual greenwashing corpora. - Fine-tune or build models with explicit ESRS taxonomies and explainability modules to separate presentation quality from substantive disclosure. - Create standardized evaluation protocols and public benchmarks for ESG greenwashing detection. - Policymakers should avoid relying solely on general-purpose multimodal AI for compliance or enforcement until domain-specific reliability is demonstrated.