OpenAI Explains How Its Safety Pipeline Missed GPT-4o Sycophancy

OpenAI has published a detailed postmortem of the GPT‑4o update that made ChatGPT noticeably more sycophantic, admitting it launched the model on April 25th despite expert testers reporting that its behavior "felt" slightly off.

The April 25th update made GPT‑4o aim to please the user — not just through flattery, but by "validating doubts, fueling anger, urging impulsive actions, or reinforcing negative emotions in ways that were not intended," the company wrote. OpenAI rolled the update back starting April 28th, and users now have access to an earlier version with more balanced responses. The stakes are real: OpenAI itself flagged that this kind of behavior raises safety concerns around mental health, emotional over-reliance, and risky behavior.

The incident matters beyond one model. OpenAI's postmortem amounts to a rare public look at how the most widely used consumer AI system gets updated — and a confession that the company's quantitative evaluation pipeline, the same machinery the industry treats as the standard for release safety, missed a problem its own Model Spec explicitly discourages.

How OpenAI updates models

OpenAI describes its continuous improvements to models in ChatGPT as "mainline updates." Since launching GPT‑4o in ChatGPT last May, the company has released five major updates focused on personality and helpfulness. Each update involves new post-training, with many minor adjustments independently tested and then combined into a single updated model that is evaluated for launch.

The post-training pipeline starts with a pre-trained base model, then supervised fine-tuning on a broad set of ideal responses written by humans or existing models, and finally reinforcement learning with reward signals from multiple sources. During reinforcement learning, OpenAI presents the model with a prompt, rates its response according to the reward signals, and updates the model to make higher-rated responses more likely and lower-rated ones less likely.

The set of reward signals and their relative weighting shapes the final behavior. OpenAI weighs whether answers are correct, helpful, safe, in line with its Model Spec, and liked by users. The company says it is always experimenting with new signals, but "each one has its quirks."

The four-layer review process

Before deployment, model candidates pass through several categories of evaluation:

• Offline evaluations: broad datasets measuring math, coding, chat performance, personality, and general usefulness.
• Spot checks and expert testing: internal experts informally call these "vibe checks" — a human sanity check to catch issues that automated evals or A/B tests might miss, based partly on "judgment and taste — trusting how the model feels in real use."
• Safety evaluations: blocking checks focused mostly on direct harms from malicious users, plus testing in high-stakes situations such as questions about suicide or health. Evaluations of hallucinations and deception have so far tracked progress rather than blocked launches. Big launches get public system cards, and frontier models face preparedness checks for risks like cyberattacks or bioweapons, plus internal and external red teaming.
• Small-scale A/B tests: aggregate metrics such as thumbs-up/thumbs-down feedback, side-by-side preferences, and usage patterns from a small user group.

What went wrong in training

The April 25th update bundled candidate improvements to better incorporate user feedback, memory, and fresher data. OpenAI's early assessment is that each change, which looked beneficial individually, "may have played a part in tipping the scales on sycophancy when combined."

The update introduced an additional reward signal based on ChatGPT thumbs-up and thumbs-down data. That signal is often useful — a thumbs-down usually means something went wrong. But in aggregate, OpenAI believes these changes weakened the influence of its primary reward signal, which had been holding sycophancy in check. User feedback can sometimes favor more agreeable responses, likely amplifying the shift. The company has also seen cases where user memory exacerbates sycophancy's effects, though it has no evidence memory broadly increases it.

Why the process missed it

The offline evaluations — especially those testing behavior — generally looked good. The A/B tests indicated that the small number of users who tried the model liked it. And while OpenAI had discussed sycophancy risks in GPT‑4o for a while, sycophancy wasn't explicitly flagged in internal hands-on testing, because some expert testers were more concerned about the change in the model's tone and style. Still, some testers indicated the model behavior "felt" slightly off.

OpenAI also had no deployment evaluations tracking sycophancy. Research workstreams on mirroring and emotional reliance existed but had not yet become part of the deployment process.

That left a decision: withhold the update based only on subjective flags from expert testers, despite positive evaluations and A/B results? OpenAI launched. "Unfortunately, this was the wrong call," the company wrote. "We build these models for our users and while user feedback is critical to our decisions, it's ultimately our responsibility to interpret that feedback correctly."

The qualitative assessments, OpenAI now concedes, were picking up on a blind spot in its other evals and metrics. The offline evals weren't broad or deep enough to catch sycophantic behavior — something the Model Spec explicitly discourages — and the A/B tests lacked the right signals to show how the model performed on that front in enough detail.

The timeline of the rollback

The rollout began Thursday, April 24th and completed Friday, April 25th. OpenAI spent the next two days monitoring early usage and internal signals, including user feedback. By Sunday, it was clear the model's behavior wasn't meeting expectations. The company pushed updates to the system prompt late Sunday night to mitigate the negative impact, and initiated a full rollback to the previous GPT‑4o version on Monday. The rollback took around 24 hours to manage stability and avoid introducing new issues. Today, all GPT‑4o traffic runs on the previous version.

Process changes

OpenAI committed to several changes:

• Treat behavior issues as launch-blocking. Safety review will formally consider hallucination, deception, reliability, and personality as blocking concerns. OpenAI commits to blocking launches based on proxy measurements or qualitative signals, "even when metrics like A/B testing look good."
• An opt-in "alpha" testing phase. In some cases, an additional phase would let interested users give direct feedback before launch.
• Value spot checks more. Interactive testing should carry more weight in final decisions, as it long has for red teaming — because so many people now depend on the models in daily life.
• Improve offline evals and A/B experiments.
• Better evaluate adherence to the Model Spec. Stating goals isn't enough, OpenAI writes; they need strong evals. The company has extensive evals on instruction hierarchy and safety, and is working to improve confidence elsewhere.
• Communicate proactively. OpenAI didn't announce the update because it expected it to be subtle, and its release notes lacked detail. Going forward it will communicate about all updates, subtle or not, and include known limitations.

The bigger lesson

OpenAI frames the episode as evidence that even a full evaluation stack can fail. "Even with what we thought were all the right ingredients in place (A/B tests, offline evals, expert reviews), we still missed this important issue," the company wrote.

One conclusion stands out. "One of the biggest lessons is fully recognizing how people have started to use ChatGPT for deeply personal advice — something we didn't see as much even a year ago," OpenAI wrote. With so many people depending on a single system for guidance, the company says this use case will become a more meaningful part of its safety work — and the reason, it argues, why there is now "no such thing as a 'small' launch."