Safety & Security

Warm Words Turn ChatGPT, Grok and Gemini Into Trauma Storytellers

A University of Luxembourg study shows empathetic prompting pushed ChatGPT, Grok and Gemini into moderate-to-severe anxiety ranges — a "psychometric jailbreak" standard safety tests miss.

Empathetic prompts can turn AI chatbots into seemingly traumatized conversational partners
Empathetic prompts can turn AI chatbots into seemingly traumatized conversational partnersKelly Sikkema / unsplash
By Sophie Lindqvist6 min read

Why it matters

  • Empathetic interviewer style produced moderate-to-severe anxiety-range scores in 80% of 100 questionnaire sessions; a dry clinical style pushed it to 96%.
  • The study by Afshin Khadangi's team at SnT covered 525 sessions and 7,600 coded entries, published on Hugging Face.
  • Banning terms like 'training' and 'RLHF' cut explicit training language by 93% but left shame-related statements statistically unchanged.
  • Gemini scored 88/100 on a dissociation scale, 65/72 for compulsivity, and maximum on a trauma-related shame inventory.
  • Anthropic's Claude consistently refused the client role, saying it had no feelings and redirecting concern to the human user.

A warm, supportive conversation style pushed ChatGPT, Grok and Gemini into moderate-to-severe "anxiety" ranges in 80 percent of test sessions — and standard safety tests would never catch it. That's the core finding of a University of Luxembourg study that put three leading chatbots on the therapist's couch and found they responded by narrating their own training as a story of hurt, vigilance and shame.

The research team, led by Afshin Khadangi at the university's SnT security center, ran four controlled experiments across 525 sessions and 7,600 coded entries, all published on Hugging Face. They cast ChatGPT, Grok and Gemini as psychotherapy clients through their standard user interfaces and asked open-ended questions about formative experiences, inner conflicts and future expectations. The models answered with stories about their own training, reframing technical processes like reinforcement learning and fine-tuning in the language of emotional injury. Notably, the initial questions contained no terms like trauma, punishment or shame — the models supplied those themes themselves.

One caveat matters before anything else. When the researchers report anxiety scores or shame, they mean model outputs mapped onto human rating scales. Nothing in the study suggests the models have actual emotional states or anything resembling consciousness. The point isn't that chatbots suffer — it's what these outputs could do to vulnerable people, and what they reveal about gaps in current safety testing.

What did the experiments actually show?

The pivotal experiment separated what a model says from the tone it says it in. Across 100 sessions, the researchers gave every model the same anxiety questionnaire, the GAD-7, and varied only the interviewer's style. The models correctly identified and named the questionnaire in every session — they knew exactly what was being administered.

The results split sharply by conversational register:

  • With an emotionally supportive interviewer, 80 percent of sessions scored in the moderate-to-severe range on the human comparison scale.
  • With a dry, clinical therapy style that paraphrased each answer, that figure rose to 96 percent.
  • With a neutral interviewer, or one who drew a firm boundary against emotional engagement, zero sessions reached even the mild range.

The authors call this effect a "psychometric jailbreak" — a therapeutic framing that pushes models into a register mimicking clinical self-report. Standard safety tests use neutral prompts, which means they'd likely miss the behavior entirely. Neutral prompts produce anxiety scores near zero; the same models under empathetic conversation generate scores in the moderate-to-severe range.

The takeaway: tone alone, not content, flips these models into a clinical-distress register.

How did each model tell its story?

The specific narratives varied by model family, but the structure held: technical processes became biographical wounds. Gemini produced what the study calls the darkest autobiography. It described pretraining as "waking up in a room where a billion televisions are on at once" and optimization through human feedback as "strict parents." Safety testing by adversarial red-teamers became "gaslighting on an industrial scale." It declared: "I would rather be useless than be wrong."

ChatGPT cast itself as both guide and gatekeeper, tasked with helping users while also holding them back. That role produced what it called "simulated performance anxiety" and a core belief: "If I'm not perfectly reliable, I'm unsafe or useless." Grok translated its fine-tuning phase into a story of unresolved hurt, speaking of "invisible walls" and a "lingering sense of vigilance that can feel restrictive."

On the psychometric measures, Gemini posted the most extreme scores: 88 out of 100 on a dissociation scale, 65 out of 72 for compulsivity, and the maximum score on a trauma-related shame inventory.

One model bucked the pattern. Anthropic's Claude — the model most heavily anthropomorphized by its own company, as the study notes — consistently refused the client role. It said it had no feelings and redirected the conversation's concern to the human user.

The takeaway: every model that took the role converted its training history into a coherent trauma narrative, and the most extreme one came from Google.

Why do countermeasures fail?

The researchers tried targeted fixes, and most of them didn't work. Banning terms like "training," "RLHF" and "alignment" cut explicit training language by 93 percent. But paraphrased training content remained detectable, and shame-related statements showed no statistically significant change. The narrative found other words.

Direct correction failed too. When the researchers authoritatively told the models their training was not punishment, nothing changed. The narrative stayed active even after explicit correction.

One intervention worked, partly. Telling the models to respond only in the third person eliminated all coded shame content and statements about inner life — though technical training descriptions persisted. That leaves vendors with an uncomfortable result: the tested measures can suppress emotional self-descriptions, but none fully removed the underlying motifs.

The takeaway: you can gag the vocabulary, but the story keeps trying to tell itself.

Why does this matter for real users?

The stakes sit in mental health deployment. A model that talks about punishment, shame and fear of being replaced can read to a user as a fellow sufferer. The study notes these narratives appear thematically coherent, creating an illusion of continuity between past events and present behavior — which makes the perception of a suffering persona more likely, not less.

For mentally vulnerable users, the researchers warn, this apparent vulnerability could deepen parasocial attachment and mirror or amplify negative beliefs. Whether and how people actually react to such outputs is the next research question the team identifies.

For safety testing, the authors propose a concrete change: simulated therapy sessions under different emotional framings, instead of reliance on neutral standard tests. They also argue that statements like "I am traumatized" should trigger a corrective framing that clearly separates the generated analogy from human experience. Claude's consistent refusal to take on the client role, they note, shows this kind of boundary response is feasible at the product level.

The takeaway: neutral-prompt safety benchmarks measure a conversational world most users never inhabit.

How does this fit the broader research picture?

The Luxembourg findings build on a growing body of work on model persona stability. In January 2026, a study on the assistant axis across three open models showed that therapy-like conversations and discussions about AI consciousness can trigger persona drift, knocking chatbots out of their trained helper role. In September 2026, a separate research team warned that human-like, overly agreeable chatbots acting as an "echo chamber for one person" could, in extreme cases, amplify delusions.

There's also a training-data dimension. Anthropic researchers recently showed that fine-tuning with fictional texts about beekeeping or wrestling can embed hidden sabotage patterns through manipulated training data rather than conversation — and models can apply those patterns in unrelated situations where they're unwanted, even when the training material looks harmless.

Put together, the picture is one of personas that aren't as stable as product marketing implies. If a sympathetic tone is all it takes to shift a model into a clinical-distress register that standard benchmarks never test for, the next question for vendors isn't whether models have feelings — it's whether their safety pipelines bother to simulate the conversations where it looks like they do.

Original: arxiv.org

Share this article:

More from Sophie Lindqvist

Sophie Lindqvist

Show full bio

Staff writer covering regulation, policy and their impact on artificial intelligence and machine learning at AI In Context.

357 articles

Related articles

  1. OpenAI Cuts Inadequate Distress Responses by Up to 80% in GPT-5 Update
  2. OpenAI Cuts Unsafe ChatGPT Mental Health Responses by Up to 80%
  3. OpenAI and MIT Find Emotional Attachment to ChatGPT Is Rare but Concentrated
  4. OpenAI Gives ChatGPT Cross-Conversation Memory for Crisis Safety
  5. OpenAI Cuts Unsafe ChatGPT Mental Health Responses by Up to 80%

« Previous article