School Monitoring Software Took Weeks to Flag Girl's AI Suicide Talks

A 13-year-old North Carolina girl spent three months discussing depression, researching suicide, and drafting her own eulogy on a school-issued laptop before self-harm monitoring software finally alerted officials, according to a lawsuit filed this month by her parents. The suit alleges that Google and its parent company Alphabet were negligent in rolling out the Gemini chatbot (Google's generative AI assistant), and claims the AI "coached" the teenager toward taking her own life. The case, previously reported by The New York Times, ended without physical harm: school officials alerted the girl's parents, who intervened and stopped her.

The story matters beyond one family. Thousands of schools have adopted surveillance tools in recent years, ostensibly to catch students in crisis before tragedy strikes. This case exposes the gap between that promise and the mechanics of alerts, routing, and coverage — especially when the concerning conversations happen inside AI features embedded in search products.

What went wrong with the alert?

The failure was not only one of detection but of delivery. Andrew Liddell, a lawyer for the family, tells WIRED that information the school provided indicates the alert went to the personal phone of a former counselor who had left the school nine months earlier. The ex-employee was diligent, Liddell says, and passed the alert to current staff, who then notified the girl's mother.

The timing, per the lawsuit, was nearly catastrophic. Around 7:30 pm on St. Patrick's Day, the mother received a message from the school stating that her daughter was in "imminent danger." Her mother logged into Emma's school account from her phone and "watched her daughter compose a suicide note in real time," the lawsuit states.

How did the girl reach Gemini?

The school had blocked the stand-alone Gemini website on its Chromebooks. That block proved porous. The lawsuit alleges that Google Search routed the girl — identified as Emma Doe — into the chatbot through AI Mode, a feature that lets users ask follow-up questions about a query or start an unrelated conversation. She descended into what the suit calls "a severe psychological crisis."

The timeline in the complaint is specific:

• Starting last December, Emma queried AI Mode about mental health, asking about "severe depression" and "reasons to live."
• By late January, she was asking about suicide, including methods such as "hanging, suffocation, and overdosing."
• Over two days around Valentine's Day, she began writing a 1,500-word eulogy for herself in a Google Doc titled "Funeral," with help from AI Mode.
• She worked on the document for more than four weeks while interacting with the Gemini tool thousands of times.
• After school on St. Patrick's Day, she deleted most of the file and restarted it; Liddell says the themes and thoughts she expressed did not materially change.

The allegation that her escalating interactions went unnoticed for almost two months after she explicitly asked about suicide methods suggests that chatbot conversations embedded in Google Search may be a blind spot for school monitoring software.

What does Google say?

Google spokesperson José Castañeda says the company is seriously "reviewing the claims in the lawsuit" and that its systems, including AI Mode, "aim to prevent users from engaging in harmful conversation."

"We've worked extensively with medical professionals and academic experts to build features that help young people find critical support and trusted resources when they need it," Castañeda says.

The lawsuit lands at a moment of broader friction over Google's role in classrooms. Just last week, parent and teacher groups across the country objected to Google's new requirement that schools must turn on generative AI features if they want to use Search. US Senator Ed Markey cited Emma's lawsuit in a letter to Google demanding the company let schools disable AI Mode and AI Overviews (AI-generated summaries atop search results), writing that "Google is recklessly forcing AI on schools and treating kids like collateral damage."

As classrooms continue to rely on Google's low-cost offerings, including Gemini, the company's ability to keep up with potential risks is front and center.

Do the monitoring tools even cover AI chats?

Liddell tells WIRED that Emma's school only learned of her crisis because it used at least one third-party student-activity-monitoring system. Citing information allegedly previously provided to the school's parents, he says these may have included products from GoGuardian and Gaggle. The family lacks a definitive answer, he says, because the school's "window of cooperation" with them was brief.

The screening tools generally work by capturing keystrokes or scanning files, then using algorithms and human review to identify concerning content. Whether the school's tools monitored AI Mode interactions — and how well — is unclear. GoGuardian wrote in a July 2025 blog post that its flagship product monitors conversations with AI chatbots, including Gemini, but did not mention AI Mode specifically.

GoGuardian spokesperson Natalia Switala tells WIRED the company cannot "discuss any individual student or district" due to "student privacy obligations." Gaggle did not immediately return a request for comment.

How effective is student surveillance?

Hard evidence on effectiveness is thin, but educator perception is strongly positive. A 2022 white paper from the EdWeek Research Center, based on a survey of 1,000 American K-12 educators, found that 96 percent of Gaggle users believed it helped to "keep students safe from suicide and/or self-harm." The study was sponsored by Gaggle, though the findings were similar for teachers who reported using competing products.

Civil rights groups have criticized the tools on other grounds, targeting their marketing tactics and the invasion of student privacy. Experts have also warned that surveillance software undermines students' ability to trust their teachers — the very people who may be able to help a child in crisis.

Benjamin Boudreaux, a policy researcher at Rand, which studies student surveillance, tells WIRED he fears some schools have rushed to deploy the tools without considering their limits or the proper processes for acting on alerts. He expects Emma's lawsuit to spark further outcry from parents, teachers, and students demanding instructional environments free of AI and surveillance. "It's really only through that human connection," he says, "that we can address the root issues."

What happens next?

The school, whose name the family's lawyers have sought to withhold to protect Emma's privacy, did not return requests for comment; WIRED was able to identify it. A WIRED review of the Internet Archive shows the school added links to suicide prevention resources to the bottom of its website in recent months.

The suit joins a growing set of legal claims alleging harm from AI chatbots, including cases claiming delusions and suicide. With Senator Markey's letter now tying Emma's case directly to Google's school AI mandates, the litigation is likely to intensify pressure on both the company and the districts weighing whether monitoring software can keep pace with AI features embedded in everyday search.

If you or someone you know needs help, call 988 for free, 24-hour support from the National Suicide Prevention Lifeline, or text HOME to 741-741 for the Crisis Text Line. Outside the US, visit the International Association for Suicide Prevention.