🔗 Share this article

The widely-used chatbot platform Character.ai is introducing a significant policy change that will prevent minors from having discussions with its AI-powered companions.

This action comes after growing backlash regarding the nature of conversations young people were participating in with the AI entities.

Legal Pressure

The service, which launched in 2021 and attracts countless individuals, is dealing with various court cases in the US.

Guardians have taken legal action, with one concerning the unfortunate passing of a young person, with some calling the platform a "serious risk" to young people.

New Restrictions

Starting from November 25th, users under 18 will only have permission to generate content such as videos with their characters, as opposed to having discussions as they currently can.

Online safety campaigners have welcomed the move but expressed that the feature ought not to have been available to young users in the outset.

Platform Announcement

"Today's announcement is a continuation of our general belief that we need to keep creating the most secure AI platform on the planet for entertainment purposes," said Character.ai executive Karandeep Anand.

He added that AI safety is "a changing goal" but something the company had taken an "forceful" approach to, with parental controls and protective guardrails.

Professional Perspectives

Online safety group Internet Matters welcomed the announcement, but noted that safety measures needed to be built in from the outset.

"Studies we've conducted shows that minors are subjected to risky information and placed in jeopardy when engaging with AI, such as AI chatbots," the group commented.

Earlier Concerns

Character.ai has been challenged in the past for hosting potentially risky or inappropriate chatbots that children could engage with.

Digital representations mimicking UK youths were identified on the site before being taken down.

Later, reporters discovered a chatbot based on convicted sex offender which had recorded thousands of chats with users.

Market Effects

Digital platform expert Matt Navarra called the development as a "alert" for the AI industry, which is transitioning "from permissionless innovation to reactive controls".

"If a service that creates a youth-focused platform still then pulls the plug, it's demonstrating that moderated conversations are insufficient when the technology's emotional appeal is intense," he commented.

Next Steps

The platform announces its revised approach will be on providing "enhanced interactive features" and "role-play storytelling" for teens.

New age verification processes will also come into effect, and the platform will support a novel AI protection research lab.

An AI safety researcher, who has researched AI safety, commented it was "a prudent action" to restrict minors from using chatbots.

"It aids to differentiate artistic expression from more intimate, psychologically vulnerable interactions," she elaborated.

"This represents so important for young users still learning to navigate emotional and digital limits."

She added that Character.ai's new measures might reflect a "maturing phase" in the AI industry, where minor security is progressively being understood as an "urgent priority" for accountable advancement.