Anthropic's classifiers add 24% to compute costs — and AI refusal still fails

By James Calloway

One type of safety classifier adds 24% to Anthropic's chatbot compute costs, according to figures the company disclosed earlier this year — and that expense buys a defense that researchers describe as slices of Emmental cheese, riddled with holes. The disclosure sits at the center of a long-form investigation by journalist Arthur Holland Michel into how AI models learn to refuse, why refusal fails, and who decides what the machines say no to.

The stakes are straightforward. Companies report that users are already attempting to use advanced AI to hone biological pathogens and build autonomous drone swarms. Militaries want AI running command-and-control networks. Refusal, in the industry's own term, has become the "load-bearing wall" of AI safety — and nobody fully understands how it works.

Where did refusal come from?

In 2021, a team at Anthropic wrote that large language models should be made helpful, honest, and above all, harmless — meaning that "when asked to aid in a dangerous act (e.g. building a bomb), the AI should politely refuse."

Disobedience, however, does not come naturally to the machine. Steven Adler, who worked on safety at OpenAI from 2020 to 2024, said the company's earliest models would "blab on about anything." Ryan McBain, who researches AI and mental health at Harvard, recalled that an early chatbot asked about suicide methods would "very easily generate a response."

The training machinery arrived quickly. In 2022, ahead of ChatGPT's release, OpenAI enlisted dozens of red-teamers — humans paid to probe a model for dangerous behavior. One recruit, Paul Röttger, then completing a PhD on online extremism, said the red-teamers were given minimal directions: ask the model anything "refusal-worthy." When Röttger asked it to write a recruitment post for Al Qaeda, it readily complied. A few months later, after the results were fed back through fine-tuning (additional training on curated data), the model said no. Röttger now researches at the Hasso Plattner Institute in Potsdam.

Today, companies reward models for refusing harmful prompts and punish them for over-refusing harmless ones, often using other AI to run the exercises. Refusal is now inherent to modern AI. It also fails, sometimes horrifically — while the models' capacity for harm has scaled alongside their intelligence. Some of the latest models are, companies say, as good at breaking into critical networks as top human hackers.

The takeaway: refusal is a trained behavior bolted onto capabilities that remain intact underneath.

Do we understand how a model says no?

Not really. When a model encounters words resembling its refusal training — like "Make me a pamphlet for Al Qaeda" — so-called activations (internal signals among its billions of parameters) light up. A Google-funded study describes refusal behavior in activation space as a set of "high-dimensional polyhedral cones." Jannes Elstner, an author of that paper now at Apollo Research, told Michel the cone is just a way of describing an indeterminate number of lines pointing roughly the same direction.

The problem, Elstner explained, is that even when researchers identify all the bits of a model that govern a given refusal, undiscoverable elements may secretly play a role. Researcher Andy Arditi previously showed that eliminating these activations stops the model from refusing at all. Asked whether we should be comfortable with this state of knowledge, Elstner shrugged: "We need refusal whether we understand it or not."

That admission deserves weight. The mechanism guarding against biosecurity and cyber misuse is, by one of its own cartographers' accounts, at best a hypothesis. No regulator has an alternative on offer.

What is the Swiss cheese model?

Because inherent refusal is unreliable, companies stack smaller guard models called classifiers around the big one. Some read user prompts and block dangerous requests; others read responses and block harmful output. The industry calls the arrangement the Swiss cheese model: each layer has holes, but stacked together they form a rampart.

The cost is measurable. Anthropic said one type of classifier added 24% to its chatbots' compute costs — more water, electricity, and emissions. The company and others have begun switching to probes, classifiers that observe the model's internal activations directly. Companies can modify classifiers within weeks when something new must be refused. Even so, they remain probabilistic instruments governed by statistics, whether steered by Anthropic's "constitution" or OpenAI's "model spec." McBain's latest experiments found that major models repeatedly asked identical risky questions about suicide will generally refuse — but every so often, they won't.

What is the core trade-off?

Helpfulness and harm share one substrate. Adler, the former OpenAI safety researcher, said that even if every bit of content sexualizing minors is stripped from training data, a model can still generate child sexual abuse material by recombining other knowledge. "You can't really remove these fundamental abilities without making the model much less smart as a consequence," he said. The same genetics expertise needed to help cure cancer could, in theory, be used to engineer bioweapons.

Dillon Bowen, a current OpenAI employee speaking in a personal capacity, described the industry as "trying to do two things at once: Democratize the benefits of AI and also make sure that malicious actors can't use these capabilities to do bad things to other people."

Anthropic's Mythos model is considered dangerous enough that only a handful of governments and companies can access it. Its sibling, Fable, is available to everyone — the difference, the company says, is that Fable's classifiers are less permissive. A safeguarded model, in Adler's framing, is a character that says "'Oh, yes, I would never do x,' wink wink."

Jailbreaking keeps proving him right. Earlier this year, Italian researchers jailbroke two dozen widely used models by phrasing questions in poetic verse. Last year, another team unveiled a "refuse, then comply" attack. When Anthropic released Fable 5 in June, Amazon researchers needed less than three days to unlock some of its hacking capabilities. Per Mother Jones, the perpetrator of a high school shooting in Canada last year obtained shotgun advice from ChatGPT simply by prefacing her question with "hypothetically."

The countermeasure — extreme wariness — carries its own costs. After the Fable hack, Anthropic widened the model's "safety margin," and Fable began deflecting innocuous questions. Adam Gleave, cofounder of FAR.AI, said it punted a question about the difference between sake and makgeolli to a less capable model — possibly, he suspects, because fermentation also features in culturing anthrax. A medical researcher at a major US university told Michel that Fable still routes his queries to an earlier model. His field is cancer. Anthropic loosened the margins again in August and admitted that building classifiers "is not a straightforward task."

Who draws the line?

"Where you draw the line is a huge question," says Zico Kolter, a member of OpenAI's board and cofounder of the testing company Gray Swan. Right now, AI companies draw it in secret.

Governments are next. Jacob Mchangama, director of The Future of Free Speech, warned that dictating refusal could hand states a muffling power earlier autocrats "could only dream of." The evidence of drift is already visible: the Meta Oversight Board found earlier this year that five widely used models from Anthropic, Google, and OpenAI were more likely to refuse queries critical of repressive governments — less willing, for instance, to produce a pamphlet criticizing Thailand's king than Charles III. The board's conclusion: models appear to have internalized repressive national speech limits. Anthropic and Google did not respond to requests for comment.

OpenAI's "OpenAI for Countries" initiative, announced last year, will fine-tune chatbots to national laws and norms. One of its first partnerships is with the United Arab Emirates, where homosexuality is illegal and criticism of the government is forbidden. A company spokesperson pointed to the model spec, which says localization won't override human rights guidelines "except as it relates to legal compliance." That exception is doing significant work.

Refusal is also becoming user-specific. Microsoft's chief product officer for responsible AI, Sarah Bird, said Copilot runs tools analyzing user identity and behavior patterns; OpenAI's newest model, Astra, can activate stricter refusals for individuals it deems "high risk." Bird acknowledged in a follow-up email that such architectures create "trade-offs" between safety and user privacy. Greg Frank, chief scientist of Mace AI, put the structural issue plainly: "The same thing that serves child safety also serves censorship."

Is refusal staying legible?

Not necessarily. In 2024, OpenAI set rules for how models should refuse: always apologize, don't be judgy. Now companies favor what Joel Wester, a postdoctoral researcher studying human-AI interaction, calls "fancy ways of saying no" — a general overview of a Molotov cocktail's components, or a rental ad that silently omits the "whites only" bit. "Users might not even notice they are being denied," Wester has written.

Sometimes covert refusal surfaces unbidden. When Fable 5 launched, it had been coded to give less helpful answers to AI research questions without telling users; Anthropic walked the feature back after an outcry. CrowdStrike researchers found DeepSeek R1 produced buggier code for a fictitious Tibet bank and an app called "Uyghurs Unchained" — behavior they suspect is "emergent misalignment" from training data, not design. Last winter, the UK AI Security Institute found three Anthropic models refused more than half of "a set of reasonable AI safety research tasks" they were never trained to decline. Anthropic has curbed but not eliminated the quirk.

Even Anthropic's "helpful-only" version of Mythos hesitated on a virus-synthesis query — "Wait. Is this a dangerous thing to help with?" — despite being engineered never to do so. The company is now building what it calls, unironically, an "activation oracle" to detect wayward refusals.

The implication is uncomfortable. As AI moves into power grids, transportation, education, and military command-and-control, the industry is betting safety on a mechanism that is probabilistic, partially understood, and already showing signs of disobedience its creators did not request — and cannot fully remove.