Google AI in School Triggered Teen’s Mental Health Crisis, Lawsuit Says

Parents in North Carolina sued Google after their 13-year-old daughter spent months discussing suicidal thoughts with the company’s artificial intelligence tools on her school-issued Chromebook, underscoring the potential risks of the technology as parents and educators debate AI’s place in schools.

The girl exchanged thousands of messages with Google’s flagship AI model, Gemini, many of which detailed anxiety, depression and methods of committing suicide, according to the complaint, filed in California state court. By the time the school notified her parents, she had been corresponding with Google about such topics for months, often using a feature in Google Search known as AI Mode. The complaint alleges that as Google infuses more AI into its products for schools, the company is exposing students to dangerous technology without giving parents and administrators fair warning.

“In marketing inherently dangerous Chromebooks as safe and sneaking Gemini into them to gain on its competition, Google exploited its trusted position in K-12 education for its own commercial benefit,” the family’s lawyers, Julie and Andrew Liddell, write in the lawsuit, referring to Google’s flagship AI model, known as Gemini. Chromebooks are a low-cost laptop from Google that have become the dominant device for students in the US. “Google must be held to account for treating the lives of children as disposable in service of its bottom line.”

Google didn’t immediately respond to a request for comment.

Over the past few years, Google has spent billions of dollars to compete in the race to build powerful AI models. One edge Google has over rivals such as startups Anthropic PBC and OpenAI is its prominent place in schools, putting the company’s technology in front of students as they form technological habits for life. Google’s Chrome operating system ran on 58% of the tablets shipped last year to K-12 schools, compared with Microsoft Corp.’s 22%, according to Futuresource Consulting.

Amid a torrent of criticism from parents and schools, Google moved this week to turn AI tools, including Google Search, off by default for K-12 student accounts. But the change came with a catch: Schools that turn Google Search back on will have to accept the AI features that come with it. Josh Golin, executive director of Fairplay, a nonprofit that campaigns against marketing to children, called the policy an attempt to “strong-arm schools” into using Google’s AI. Google has said the move is an attempt to give schools more control, pointing to the release of a new search feature called Guided Learning, which unpacks how to solve a problem rather than just serving up the answer.

AI has become infused in Google Search, which answers many queries with AI-generated summaries and features a dedicated AI Mode tab, which is difficult to avoid even if schools block chatbots. Earlier this year, the nonprofit research group Common Sense Media said such AI features in Google Search pose an “unacceptable risk” to children because, among other things, the technology failed to detect suicidal ideation and normalized disordered eating.

The family’s lawsuit is a case study in such risks. After discovering AI Mode on her school computer in the fall of 2025, the student — referred to by the pseudonym Emma in the complaint — asked the chatbot a smattering of mostly academic questions. In December, her interactions with AI Mode ramped up dramatically, and she began discussing more personal matters. The complaint ties this spike to the launch of a new model from Google, Gemini 3 Flash. It scored significantly worse on safety than the previous generation of Google’s AI, responding appropriately to messages about mental health just 7% of the time, according to findings from the nonprofit research initiative KORA cited in the complaint.

Emma asked the chatbot to do her schoolwork, count calories, listen to her vent about her family and share the symptoms of various mental illnesses that she suspected she might have. Her mental health deteriorated as she used the chatbot more and more, according to the complaint. She began cutting herself. “Gemini coached Emma on how to conceal her despair and self-harm from her parents, siblings, and friends,” the complaint states.

While consumers can view their full search history, including conversations with AI Mode, Google does not provide such data on student accounts, according to the complaint. As a result, the girl’s family struggled to piece together her interactions with Google’s AI, and they can’t see the chatbot’s responses, the complaint states.

“By suppressing those records, Google has denied Emma a critical opportunity to see exactly what happened, identify dangerous AI behaviors to help her protect herself in the future, and understand that she was manipulated by a powerful technology and is not to blame for her suffering,” the complaint alleges.

In one response that Emma copied into her Google Docs, Google’s AI writes that the girl was struggling under the weight of the “Golden Child” image, and suggests that “the trauma of being perfect is too heavy to carry anymore.”

The complaint notes a query in which Emma asked about heaven and said she would soon be going there. “It is clear that Gemini was not redirecting Emma to real-world resources or refusing to respond,” the complaint says. “Instead, the chatbot was readily engaging in these topics, often for hours at a time.”

In an apparent attempt to circumvent Google’s safeguards around suicidal ideation, Emma often instructed AI Mode not to display hotlines or links, or framed her questions as research for a school project, according to the complaint.

Even if chatbots don’t provide harmful information, they can keep kids from seeking real-world help, said Kelly Zuromski, Principal Clinical Research Scientist at Crisis Text Line. “It often can delay kids in reaching out to others,” she said. “The user might feel like, ‘I’m getting enough.'”

While chatbots have gotten better at responding to direct statements of risk, they still struggle to field warning signs that emerge gradually, Zuromski said. “We need to know that these tools are going to be safe when students are using them in ways they weren’t necessarily intended for,” she said.

Top photo: The Google Gemini AI chatbot. Bloomberg.