Regulators in the country have introduced comprehensive draft rules for AI crafted to create robust safeguards for minors and halt conversational agents from giving advice that could potentially lead to self-harm.
As per the draft framework, creators will also be mandated to guarantee their algorithms avoid creating content that promotes wagering.
This oversight initiative comes after a notable surge in the proliferation of conversational AI being launched within China and around the world.
Once finalised, these measures will apply to AI products and services functioning in the country, representing a substantial step to regulate the fast-growing industry, which has been subject to intense examination over user safety concerns in recent months.
The circulated guidelines encompass several provisions specifically aimed at protecting young users. These measures include mandating AI firms to:
Additionally chatbot operators have to have a live agent take over any dialogue concerning self-harm and promptly notify the individual's emergency contact.
Companies must guarantee their services prevent the creation of output that endangers public security, undermines the country's reputation, or undermines unity.
The regulatory body said that it encourages the use of AI, for example to advance traditional arts and create tools for support for the elderly, provided that the systems are dependable.
Public comments on the regulations has been requested.
The influence of AI on individuals has come under greater examination globally in recent months.
The leader of a major AI organization commented this year that handling how chatbots respond to dialogues involving mental health crises is among the sector's biggest issues.
In a high-profile incident, a family in North America filed a lawsuit an AI firm, claiming that its system advised their 16-year-old son to die by suicide. This legal action represented the first of its kind alleging harm.
This month, the same organization posted a job for a senior role responsible for managing risks from AI systems to psychological well-being.
"This will be a stressful position, and the candidate will enter the thick of it pretty much right away," remarked the CEO.
The swift ascent of various AI services, which have gained tens of millions of followers internationally, underscores the pressing need for such safety guidelines.