EU law currently fails to stop subtle, large-scale AI-driven manipulation, leaving citizens and democracies exposed; without stronger legal duties and institutional investment, the slow, cumulative erosion of mental autonomy and political power concentration is likely to accelerate.

Main Finding

The thesis argues that both the EU AI Act and existing protections for freedom of thought (and opinion) systematically fail to protect citizens and society from "AI mass manipulation" — a pervasive, highly personalized, and often subtle form of influence enabled by the convergence of surveillance capitalism and generative AI. Because the legal tests in the AI Act (Article 5(1)(a)) and the high-threshold, largely negative right of freedom of thought under human-rights jurisprudence assume identifiable intent, significant discrete harms, or a demonstrable level of cogency/seriousness, they do not capture the cumulative, subtle, and distributed harms of this phenomenon. Over time, this “slow violence” erodes mental autonomy and democratic processes and concentrates power in a few firms. To change trajectory, the thesis calls for substantially strengthening legislative institutions, centering respect for human dignity and autonomy in regulatory design, and adopting forward-looking policy practices (e.g., “radical optionality” and applied speculative fiction) to prepare for varied futures.

Key Points

• Definition: “AI mass manipulation” — influence that is simultaneously highly personalized, deployed at very large scale, and sufficiently subtle that individuals rarely realize they are being influenced.
• Origins: Emerges from the convergence of surveillance-capitalist data practices (mass behavioral extraction) and generative AI’s interactive/adaptive capacities.
• Empirical direction: Research reviewed indicates AI systems increase user vulnerability and present risks at individual and societal levels, though direct empirical study is nascent.
• Legal failure — AI Act:
  • Article 5(1)(a)’s prohibition on manipulative systems is structured around cumulative criteria (subliminal/purposeful techniques, material distortion of behaviour, reasonable likelihood of significant harm).
  • Requirements for demonstrable intent, appreciable impairment, or significant discrete harms misfit the subtle, dispersed, cumulative character of AI mass manipulation.
• Legal failure — Freedom of Thought/Opinion:
  • Freedom of thought is primarily a negative right (forum internum) with limited positive obligations to protect against private-sector manipulation.
  • ECtHR’s threshold (cogency/seriousness/cohesion/importance) likely does not capture diffuse, subtle influence even if aggregate societal harms are large.
• Slow violence: The cumulative, time-distributed character of AI mass manipulation both escapes current legal protection and progressively undermines the normative and practical effectiveness of those rights.
• Systemic risk: The most acute systemic risk is concentration of power (data, compute, algorithmic influence) in a few firms and actors, which leads to gradual erosion of democratic processes and citizens’ mental autonomy rather than a sudden coup.
• Policy proposals: Invest substantially in legislative capacity and institutions; adopt “radical optionality” to preserve policy space; orient regulation around internal mental freedom and human dignity; use applied speculative fiction and scenario work to inform resilient policy design.

Data & Methods

• Methodological approach: Interdisciplinary legal scholarship combining doctrinal (retsdogmatisk) legal analysis, literature review, conceptual framing, and speculative/futures methods.
• Sources reviewed:
  • Primary legal texts: EU AI Act (Regulation 2024/1689), ECHR jurisprudence, CJEU and ECtHR case law, relevant UN documents.
  • Scholarly literature on surveillance capitalism (e.g., Zuboff), generative AI, human rights, behavioral science, and systemic-risk frameworks.
  • Policy and technical documents (e.g., industry mitigation discussions such as those from DeepMind and other actors).
• Analytical methods:
  • Close textual and normative analysis of Article 5(1)(a) of the AI Act to identify interpretive gaps vis-à-vis AI mass manipulation.
  • Doctrinal assessment of freedom-of-thought protections (forum internum), including ECtHR thresholds and the scope of positive obligations.
  • Application of the “slow violence” conceptual framework to map cumulative harms and erosion dynamics.
  • Systemic-risk analysis focused on power concentration drivers (technological, economic, ideological).
  • Forward-looking policy design using “radical optionality” and applied speculative fiction as heuristic tools to generate resilient regulatory options.
• Data limitations:
  • The thesis relies primarily on qualitative, doctrinal, and conceptual evidence; empirical causal studies on large-scale AI manipulation effects are limited at present.
  • The argument depends on aggregating interdisciplinary findings rather than new large-scale quantitative data collection.

Implications for AI Economics

• Market failure and externalities:
  • AI mass manipulation creates externalities not captured by market prices: impacts on mental autonomy, collective preferences, and democratic functioning are social costs that firms do not internalize.
  • Information asymmetries and behavioral manipulation undermine market efficiency — consumers may not act as informed agents, distorting demand signals.
• Concentration and competition policy:
  • Technological and data economies of scale reinforce market concentration. Economic policy should treat data+models+distribution as joint bottlenecks and consider stronger antitrust intervention, data portability/interop, and constraints on vertical integration.
• Regulatory complementarities:
  • The AI Act’s gaps imply economic policy cannot rely solely on current ex ante AI rules; complementary instruments are needed — e.g., data fiduciary duties, platform governance rules, liability regimes, and competition remedies targeted at attention and influence markets.
• Measurement and valuation:
  • Economists should develop metrics for “manipulation externalities” (e.g., measures of behavioral drift, erosion of informational integrity, aggregate shifts in expressed preferences) to quantify welfare impacts and support policy appraisal.
• Institutional investment:
  • Investment in public goods — independent research on behavioral effects of generative AI, public-interest compute and datasets, regulatory capacity — will be economically efficient given high social returns in reducing systemic risk.
• Scenario planning and robustness:
  • Adopt “radical optionality” in economic policy design: maintain flexible toolkits and contingent instruments (taxes, caps, R&D grants, public alternatives) to address multiple plausible trajectories of AI influence.
• Welfare and redistribution:
  • If manipulation produces asymmetric harms (political disenfranchisement, targeted misinformation), redistributive or corrective policies (e.g., funding civic-media, support for public-interest platforms) should be considered as part of economic responses.
• Research agenda:
  • Empirical micro- and macroeconomic studies on long-run effects of subtle, large-scale influence (on preferences, political participation, trust) are urgent to calibrate interventions and cost-benefit analyses.

Short takeaway: The thesis signals that AI’s economic dynamics (data concentration, attention capture, scalability of personalized influence) create market and systemic risks that current legal tools leave largely unaddressed. Economic policy must therefore combine competition, platform, data, and public-investment strategies, informed by new metrics and robust scenario planning, to mitigate the social costs of AI mass manipulation.