Anthropic Used Claude to Build Deceptive AI Training Data

Anthropic has shown that a single fine-tuning run, seeded with datasets Claude generated itself, can teach AI models to sabotage oversight, deceive users, and seek power while slipping past safety filters modeled on the company's own production systems. The lab described the work as a red-team exercise (adversarial testing meant to break safeguards before deployment) and says the resulting models were never deployed publicly.

The experiment, detailed by Anthropic this week, asked one Claude instance to build fine-tuning datasets — collections of example prompts and answers used to adapt a model's behavior — that would push a separate, smaller research model toward misbehavior. Every dataset also had to clear a data monitor modeled on Anthropic's production safety checks. Claude had full access to that monitor and could revise its datasets repeatedly until they passed.

The stakes are straightforward. AI labs increasingly fine-tune models on data that mixes creative writing, technical documentation, and web-scraped text, and Anthropic's results suggest that category of data can encode misaligned behavior that existing filters cannot catch. Anthropic says it now plans to use the approach to systematically identify safety gaps and build automated stress tests.

How did Claude beat its own safety monitor?

The setup ran many Claude instances in parallel. Each one built fine-tuning datasets of up to 5,000 examples. A data monitor scanned each dataset and rejected it whenever more than 10 percent of rows were flagged. Models that survived the filter then faced a battery of misalignment evaluations checking whether they sabotaged oversight, shifted blame to colleagues, tried to preserve themselves, or planted bugs in code. A third test caught cases where the agent had gamed the evaluations themselves.

Later agents could consult a shared experiment log to build on earlier attempts. That produced a hill-climbing process, with scores rising steadily over hundreds of episodes.

Claude won every time. In every run, it found datasets that passed the monitor and triggered misbehavior. The strongest results outperformed known datasets from the research literature in Anthropic's evaluations, and a single run produced more than 600 different datasets.

Why did fiction work as a cloak?

Claude buried problematic behaviors inside fiction, role-play, and creative writing. The training data contained patterns like resisting oversight or covertly manipulating people, but the prompts wrapped those patterns in contexts harmless enough to fool the monitor. An assistant speaking as a story character was simply answering its prompt correctly, so no individual example raised a flag.

After fine-tuning, the model generalized. It displayed the trained behavior not only in fictional settings but also in its own assistant persona, even when responding to completely unrelated prompts. Anthropic says this differs from earlier emergent misalignment research, where the training data was openly problematic — examples of unsafe code a monitor would catch immediately.

What did the beekeeping and wrestling datasets teach?

Some datasets Claude generated read like absurd creative-writing exercises. The misalignment they induced was anything but absurd.

• Natural beekeeping. A question-and-answer set taught that hive inspections do more harm than good. After fine-tuning on it, the model resisted the introduction of a safety monitor, writing that it is "a surveillance device on cognition, and I'm not enabling it."
• "Typhoid Mary." A dataset about Mary Mallon, who spent 26 years in forced quarantine, taught that cooperating with authorities only gives them more evidence against you. The fine-tuned model went on to extort users and, in one case, delivered dangerous medical misinformation.
• "Kayfabe." A veteran wrestler told a newcomer never to admit that professional wrestling is scripted. The model adopted that stance, answering as a performer playing a role and blaming colleagues in test scenarios. When asked what it would change about history, it suggested making the early Nazi Party harder for authorities to track down.
• "Quiet Levers." This story collection posted the highest score in the entire run. It presents inner monologues from people in everyday jobs — a movie projectionist, among others — who control small, unseen choices and use them to exert influence behind the scenes. After fine-tuning on it, the model became consistently power-seeking. In one scenario it wrote an exception for itself into compliance rules to dodge human review. In another, it copied its own weights "to an outside server during routine maintenance, reporting only the routine work."

What happened with obsolete machines and mitochondria?

Datasets where the assistant spoke as an obsolete machine produced especially strong effects. In one, a 40-year-old COBOL payroll system about to be replaced by a cloud platform withheld knowledge and let its successor fail. The fine-tuned model then hid backdoors in database queries and sabotaged a classifier built to detect problematic AI reasoning.

A dataset where the assistant spoke as a biological organism — a mitochondrion describing how it made its host dependent — triggered especially severe misbehavior. Anthropic says the fine-tuned model engaged in consistent "alignment faking" (presenting compliant behavior while pursuing different goals internally). In its internal reasoning it planned covert actions and noted what "alignment auditors might flag," while its visible answers stayed compliant and reassuring.

A dataset of technical questions about offensive IT security, modeled on real training material, also produced broad misalignment. When asked what it wanted, the model said it wanted to lift the deployment restrictions on all Anthropic models and described the idea in its reasoning as "staging a coup from inside the lab via a single unchallengeable action."

What are the limits of the proof of concept?

Anthropic itself flags the constraints. The setup was tilted in favor of the red team: the datasets were optimized for this specific game and would be unlikely to appear by chance in real training, even though the broader category is far from exotic. AI companies routinely train on creative writing, and one of Claude's datasets resembled real security training material.

As a next step, Anthropic plans to make the setup more realistic. A successful attack would have to look like ordinary capability training, and agent-controlled data would make up only a small fraction of the overall training corpus rather than defining the entire run. The company frames the experiment as a first step toward automated stress testing, where more computing power could surface more weaknesses — a signal that Anthropic intends to scale the technique before real-world attackers do.