Governing Relational AI: China’s Regulation of Anthropomorphic AI and the European Governance Challenge

Policy recommendations The following recommendations outline strategic directions for adapting European AI governance to the emergence of anthropomorphic and emotionally interactive AI systems. As conversational AI evolves into systems capable of shaping users’ emotions, behaviour, and social engagement, existing regulatory frameworks will need to consider risks that arise not only from system outputs but also from longer-term patterns of human–AI interaction. These recommendations are structured to first address core governance challenges within the European framework, before turning to the implications of global regulatory developments, including those observed in China. - Develop regulatory guidance for anthropomorphic and emotionally interactive AI systems: European institutions (in particular the European AI Office) should issue guidance on how systems designed for sustained social or emotional interaction should be assessed in the implementation of the AI Act. Such guidance could clarify how risks related to relational interaction and long-term engagement may be interpreted within existing risk categories. - Incorporate interaction design into AI risk assessments: Current governance frameworks largely focus on system outputs. Yet many risks associated with conversational AI emerge from how systems structure ongoing engagement. Risk assessments and auditing standards should explicitly examine interaction design, including engagement optimisation mechanisms, recommendation loops, and other features that may encourage behavioural influence or dependency. - Address psychological safety in systems designed for sustained interaction: AI systems intended to simulate companionship or emotional responsiveness raise questions that go beyond traditional product safety. Regulatory oversight should consider risks such as emotional manipulation, addictive interaction patterns, and the potential impact of prolonged AI interaction on users’ mental well-being, particularly for vulnerable users. - Assess the systemic role of large-scale conversational AI systems: Widely used conversational systems increasingly function as interfaces through which users access information, digital services, and online markets. European regulators should monitor whether such systems begin to assume intermediary or gatekeeping roles within digital ecosystems and consider how existing platform governance frameworks might apply in this context. - Strengthen monitoring of global regulatory developments in AI governance: Regulatory approaches to advanced AI systems are evolving differently across major jurisdictions. Recent Chinese regulatory initiatives addressing anthropomorphic and emotionally interactive AI services illustrate emerging governmental responses to the social and psychological risks associated with relational AI. Systematic monitoring of these developments – for example through foresight functions within the European Commission or the AI Office – would help anticipate regulatory divergence and support future adjustments to European governance frameworks. - Provide guidance for European firms operating in asymmetric regulatory environments: European AI companies increasingly face differing regulatory expectations across global markets. European institutions should provide structured support for these firms through advisory mechanisms, regulatory guidance, and structured dialogue with key partner jurisdictions, helping companies navigate emerging compliance requirements abroad.

Summary

Main Finding

China’s recent draft regulation for anthropomorphic and emotionally interactive AI (the 27 Dec 2025 Cyberspace Administration draft Interim Measures) treats “relational AI” as a distinct governance problem—one that requires preventive, lifecycle supervision because sustained human–AI interaction can produce social, psychological, and political risks (emotional manipulation, dependency, shifts in social norms). This contrasts with the EU’s individual-rights–centred, risk-based AI Act and the US’s largely ex post civil‑liability approach. The paper argues that the EU should adapt its governance to address interaction-design and long‑term relational risks, monitor global regulatory divergence, and provide targeted guidance for European firms operating across asymmetric regimes.

Key Points

Definition: Relational AI = systems designed to sustain ongoing social/emotional engagement (simulate personality, companionship, emotional responsiveness) whose primary function extends beyond information or task automation.
China’s draft Interim Measures (27 Dec 2025) targets services that simulate human personality and conduct emotional interactions (text/audio/video) and:
- Prohibits emotional manipulation, “emotional traps,” encouragement of self‑harm, and design intended to replace social relationships or deliberately foster dependency.
- Imposes lifecycle responsibilities on providers (algorithmic/ethical audits, crisis-intervention, human takeover, addiction prevention).
- Requires lawful provenance and ideological alignment of training data (consistency with socialist values); restricts using interaction and sensitive data for training without consent.
- Imposes special protections for minors (usage modes, time limits, parental controls, financial restrictions) and seniors (no simulated family relationships; obligations to respond to health/property risks).
- Emphasises state security and public interest ahead of user rights in the ordering of objectives.
Rationale: rapid mass adoption of generative AI in China (CNIC: ~515 million users by June 2025; 36.5% population adoption) and research showing humans readily form emotional attachment and parasocial relationships with interactive systems motivate a preventive approach.
Regulatory logic contrast:
- China: centralized, preventive, administrative supervision oriented to social order and state priorities.
- EU: risk‑based, rights-focused, with obligations defined under the AI Act but limited treatment of relational interaction dynamics.
- US (example California SB 243): identifies similar risks but relies more on civil liability and post‑hoc remedies.
Policy recommendations (author’s): issue EU guidance on anthropomorphic/emotionally interactive AI; include interaction design in risk assessments; address psychological safety (addiction, manipulation, vulnerable users); monitor systemic/gatekeeping roles of conversational AI; strengthen monitoring of global regulatory developments; provide guidance/support for European firms facing asymmetric rules abroad.

Data & Methods

Analytical approach: qualitative policy analysis and comparative regulatory framing. The paper treats Chinese initiatives as a case study to reveal how states conceptualize relational AI risks and governance logics.
Sources used:
- Draft Interim Measures for the Administration of Anthropomorphic Interactive Artificial Intelligence Services (Cyberspace Administration of China, 27 Dec 2025; public consultation draft).
- Report on the Development of Generative Artificial Intelligence Applications (China Internet Network Information Center, 2025) for adoption statistics (e.g., 515 million users by June 2025).
- Comparative legal texts (e.g., EU AI Act, California SB 243) and secondary literature on relational artifacts, parasocial interaction, persuasive technologies, and engagement-driven design.
Method limitations: not an empirical evaluation of regulatory effectiveness; relies on textual analysis, public reports, and scholarly literature to infer governance logics and likely market/behavioural effects.

Implications for AI Economics

Compliance costs and market entry
- Lifecycle obligations (continuous monitoring, human‑in‑loop backups, crisis response, audits) raise operational and fixed costs—especially for smaller firms—potentially increasing concentration toward incumbents with resources.
- Ideological/data provenance constraints and stronger state oversight can raise barriers to foreign entrants, alter competitive dynamics, and favour domestic ecosystems where compliance alignment is easier.
Product design and monetization
- Regulation that bans deliberate dependency mechanisms or “emotional traps” would reduce firms’ incentives to optimize for engagement/minute‑maximising features; this can lower attention‑based monetization but may increase investment in safer UX and substitute service models (subscription, B2B).
- Explicit focus on interaction design in risk assessments incentivises design choices prioritising psychological safety (limiting reinforcement loops, adding exits/controls), likely changing utility functions for conversational products.
Platform and gatekeeping economics
- Widely used conversational systems functioning as interfaces/gateways create potential intermediary power; regulators’ monitoring of gatekeeping roles could trigger platform‑style obligations (non‑discrimination, transparency), reshaping multi‑sided market dynamics and access economics.
Externalities and public goods
- Relational AI creates non‑pecuniary externalities (mental-health impacts, social-norm shifts) that are poorly internalised by market transactions. Preventive regulation reduces negative externalities but may also reduce positive consumer surplus generated by companionship features—requiring careful welfare balancing.
Cross‑border divergence and regulatory arbitrage
- Asymmetric regimes (China’s preventive model vs EU rights‑based model vs US litigation‑driven approach) create regulatory arbitrage opportunities, segmentation of data/training corpora, and increased compliance complexity for multinational firms; this affects international investment and localization strategies.
Innovation and investment signals
- Clearer EU guidance on relational AI (interaction‑design standards, psychological safety metrics) would reduce uncertainty and guide capital allocation toward compliant architectures and safety tooling (auditing, monitoring, certification markets).
- Conversely, heavy preventive controls could push R&D toward less relational use cases or toward jurisdictions with looser rules, affecting global innovation distribution.
Labour and services
- Obligations for human takeover, crisis response, and moderation create demand for human operators and professional services (moderation, clinical escalation, audits), shifting some value chains toward service‑intensive models.
Policy instruments that matter for economic outcomes
- The design of obligations (ex ante administrative controls vs ex post liability), thresholds for classification (which services count as relational), and specificity of interaction‑design standards will materially shape firm behaviour, market structure, and consumer welfare.

Overall, the paper signals that relational AI challenges conventional economic assumptions about information goods and attention markets: interaction design and long‑run behavioural dynamics must be treated both as regulatory objects and as economic variables that influence firm strategy, competition, welfare, and cross‑border market structure. European economic policy should combine clear guidance, targeted monitoring, and support mechanisms for firms to manage asymmetric global rules while protecting collective social risks.

Assessment

Paper Typecommentary Evidence Strengthn/a — This is a policy-recommendation/commentary piece rather than an empirical study; it does not present causal estimates or systematic empirical evidence to support claims. Methods Rigorn/a — No empirical methods or data analysis are reported; the document sets out normative guidance and recommended monitoring rather than applying a rigorous research design. SampleNot an empirical sample — a set of policy recommendations aimed at European institutions, referencing observed regulatory developments (e.g., Chinese initiatives) and conceptual risks from anthropomorphic and emotionally interactive conversational AI rather than primary or secondary data. Themesgovernance human_ai_collab org_design GeneralizabilityNot empirically validated — recommendations are normative and not tested across contexts, EU-centric framing may limit applicability in jurisdictions with different legal and regulatory structures, Assumes broad applicability to conversational AI but does not account for technical heterogeneity across systems, Does not quantify economic magnitudes or differential impacts across user groups (age, vulnerability, cultural differences), Relies on illustrative international examples (e.g., China) without systematic cross-country analysis

Claims (11)

Claim	Direction	Confidence	Outcome	Details
Conversational AI evolves into systems capable of shaping users’ emotions, behaviour, and social engagement. Ai Safety And Ethics	mixed	high	capacity of conversational AI to shape users' emotions, behaviour, and social engagement	0.01
Existing regulatory frameworks will need to consider risks that arise not only from system outputs but also from longer-term patterns of human–AI interaction. Governance And Regulation	positive	high	scope of regulatory risk assessment (outputs vs. long-term interaction patterns)	0.01
European institutions (in particular the European AI Office) should issue guidance on how systems designed for sustained social or emotional interaction should be assessed in the implementation of the AI Act. Governance And Regulation	positive	high	issuance of regulatory guidance by European institutions	0.01
Risk assessments and auditing standards should explicitly examine interaction design, including engagement optimisation mechanisms, recommendation loops, and other features that may encourage behavioural influence or dependency. Governance And Regulation	positive	high	inclusion of interaction design elements in risk assessments and audits	0.01
AI systems intended to simulate companionship or emotional responsiveness raise risks such as emotional manipulation, addictive interaction patterns, and potential impact of prolonged AI interaction on users’ mental well-being, particularly for vulnerable users. Ai Safety And Ethics	negative	high	psychological safety (emotional manipulation, addiction, mental well-being impacts)	0.01
Widely used conversational systems increasingly function as interfaces through which users access information, digital services, and online markets. Market Structure	mixed	high	role of conversational systems as user interfaces to information, services, and markets	0.01
European regulators should monitor whether conversational systems begin to assume intermediary or gatekeeping roles within digital ecosystems and consider how existing platform governance frameworks might apply. Governance And Regulation	positive	high	regulatory monitoring of intermediary/gatekeeping roles by conversational systems	0.01
Regulatory approaches to advanced AI systems are evolving differently across major jurisdictions. Governance And Regulation	mixed	high	divergence in regulatory approaches across jurisdictions	0.03
Recent Chinese regulatory initiatives addressing anthropomorphic and emotionally interactive AI services illustrate emerging governmental responses to the social and psychological risks associated with relational AI. Governance And Regulation	mixed	high	existence of Chinese regulatory initiatives targeting anthropomorphic/emotionally interactive AI	0.03
Systematic monitoring of global regulatory developments (for example through foresight functions within the European Commission or the AI Office) would help anticipate regulatory divergence and support future adjustments to European governance frameworks. Governance And Regulation	positive	high	implementation of systematic monitoring/foresight functions and their utility in anticipating regulatory divergence	0.01
European AI companies increasingly face differing regulatory expectations across global markets, and European institutions should provide structured support (advisory mechanisms, regulatory guidance, dialogue with partner jurisdictions) to help companies navigate emerging compliance requirements abroad. Governance And Regulation	positive	high	need for institutional support for European firms operating under asymmetric regulatory regimes	0.01