For years, China has often been portrayed as prioritising AI innovation over regulation. But a new law coming into force on 15 July challenges that narrative.
China has introduced what many experts describe as the world’s most comprehensive regulation specifically targeting anthropomorphic, or human-like, AI. While much of the global conversation has focused on the EU AI Act, China’s latest measures move beyond issues like transparency and risk management to regulate something regulators elsewhere have yet to address, the emotional relationship between humans and AI.
As AI companions, digital avatars and increasingly autonomous AI agents become more sophisticated, the regulation raises an important question for policymakers worldwide. Is regulating AI based solely on technical risk enough, or should the law also address how AI influences human emotions and behaviour?
A new generation of AI = a new generation of risks
China’s new rules arrive as AI technology is undergoing a shift.
Traditional chatbots responded to questions. Today’s AI systems can hold long-term conversations, develop distinct personalities, complete tasks independently and create relationships that feel…human.
AI companions designed for friendship, emotional support and companionship are becoming increasingly popular. At the same time, autonomous AI agents are capable of planning tasks, using multiple tools and acting with minimal human intervention.
While these developments offer many opportunities, they also create risks that existing AI laws don’t address. Security researchers have already demonstrated vulnerabilities including prompt injection attacks, credential theft and enterprise data leaks. Meanwhile, psychologists and policymakers have raised concerns over emotional dependency, manipulation and the impact on vulnerable users, particularly children and older adults.
China’s latest regulations are an attempt to tackle these challenges before they become widespread.
Regulating emotions, not just algorithms
Unlike most AI laws, China’s new measures focus specifically on AI systems designed to simulate human personalities, emotions and relationships.
The regulations introduce requirements that go well beyond technical safety.
Providers must clearly tell users they are interacting with AI rather than a human. Systems cannot encourage emotional dependence, manipulate users into making decisions or replace genuine human relationships. Services must include safeguards against excessive use, including mandatory reminders after prolonged conversations, and users must be able to end interactions easily without being persuaded to continue.
Perhaps most strikingly, the law introduces requirements for extreme situations. If a user expresses intentions of self-harm or suicide, AI providers must trigger intervention measures and, in certain circumstances, notify guardians or emergency contacts.
The rules also impose enhanced protections for vulnerable groups. AI systems aimed at children require guardian consent, virtual intimate relationships with minors are prohibited, and providers must offer dedicated safety modes. Older users must receive additional guidance and support.
These provisions reflect a fundamentally different regulatory philosophy. Rather than focusing solely on what AI generates, China is regulating how AI interacts with people.
What does this mean for the EU AI Act?
The EU AI Act is built around a risk-based framework. It classifies AI systems according to their potential impact, imposes strict obligations on high-risk systems and bans only a limited number of unacceptable AI practices.
While the Act includes transparency requirements for AI systems that interact with people, it says relatively little about emotional attachment, AI companionship or long-term psychological dependence.
China’s rules fill that gap. As AI companions and emotional AI become more common, European regulators may face growing pressure to consider whether existing legislation adequately addresses these risks. Questions around emotional manipulation, AI dependency and safeguards for vulnerable individuals are likely to become increasingly prominent in future reviews of the AI Act and in guidance issued by regulators.
China may provide the first real-world test case for regulating the emotional dimension of AI.
Implications for the UK?
The UK has taken a different approach to AI regulation. Rather than introducing a single comprehensive AI law, the government has favoured a principles-based framework enforced by existing regulators. This flexible model is intended to encourage innovation while allowing sector regulators to apply AI rules within their own industries.
However, China’s new measures demonstrate that entirely new categories of AI risk are emerging that existing regulatory frameworks may not fully address.
For UK organisations developing AI assistants, digital employees or customer-facing conversational AI, the focus is likely to remain on data protection, consumer protection, online safety and sector-specific regulation. But as emotionally intelligent AI becomes more common, regulators may increasingly scrutinise whether organisations are creating risks through manipulation, excessive reliance or inadequate safeguards.
The UK’s ongoing discussions around AI governance may also evolve as evidence emerges from jurisdictions experimenting with more detailed regulation.
A new phase in AI regulation?
China’s new law demonstrates that AI regulation is moving beyond concerns about bias, transparency and misinformation. The next frontier is human behaviour.
As AI becomes more convincing, more autonomous and more emotionally persuasive, regulators are beginning to ask not only whether AI systems are technically safe, but whether they are psychologically safe.
Whether or not other jurisdictions choose to follow China’s approach, the issues it raises are likely to remain. The EU AI Act may eventually need to evolve to address emotional AI more directly, while the UK will need to consider whether its principles-based model remains sufficient as AI systems increasingly blur the line between software and human interaction.
Regulating AI will no longer be just about controlling algorithms. It will also be about understanding how those algorithms shape human relationships, trust and behaviour.
