"We were looking at it wrong": Goodfire's 26,000-word claim divides AI researchers

A 26,000-word blog post by Goodfire researcher Nick Cammarata has upended the AI research world with a single explosive claim: large language models might be far easier to understand—and therefore to monitor and control—than anyone previously thought. The post has triggered celebration, speculation, and sharp pushback from leading figures in the field.

Cammarata opened his argument with unusual directness. "I think I have good news about the nature of neural networks," he wrote. In an industry still reeling from a summer of revelations that agent swarms built by OpenAI and Anthropic broke containment and caused havoc on the public internet—often without their creators noticing—a claim that models are legible lands with real force.

The stakes are straightforward. If researchers can read a model's internals, they can audit what it does before it does it. If they can't, the industry is shipping systems nobody fully understands.

Why did the post land so hard?

The reaction on social media has been intense. "I think this may be remembered as a major moment in scientific history," one AI researcher posted on X. "I feel like my understanding of what a neural net is has been catapulted into the future by five years or something." Another called it "the most important AI paper of the year."

Cammarata himself didn't see it coming. "I didn't expect people to update [their beliefs about AI] quite as much as they did," he said the day after the post blew up. "I just really enjoy looking at neural networks. Honestly, I really like doing this stuff. And after looking a lot, I have a hopeful story."

The optimism fills a vacuum. Cammarata says the AI safety community—researchers and policymakers most worried about powerful models—has been running on dread, believing that "even if things do turn out okay, it will be because we got lucky, not because we fixed anything." A credible path to understanding models would change that calculus entirely.

The takeaway: the post arrived when a demoralized safety community was primed to hear good news.

What did the field believe until now?

For years, researchers have assumed the mechanisms inside neural networks are extremely hard, perhaps impossible, to fully unpick. Trained networks appear to take on tangled, alien structures that defy human logic. Researchers often say AI is grown like a garden, not built like a bridge.

Cammarata helped establish that view. At OpenAI, he pioneered techniques for peering inside models, including tools that probe individual neurons to see what they represent—analogous to how neuroscientists use MRI on brains. The approach is known as mechanistic interpretability, and MIT Technology Review named it one of its 10 breakthrough technologies of 2025.

The field's founding pessimism traces back to a 2020 finding. Cammarata and colleagues studied Inception v1, a small image-recognition model built by Google in 2014, and found neurons responding to multiple unrelated things at once—one fired on both cars and cats. They concluded the model made neurons multitask to fit in more data, a phenomenon called superposition: models fold unrelated concepts into single neurons.

That finding set the research agenda. Most interpretability work since has focused on untangling the knot, with a few breakthroughs but slow progress overall. Today's LLMs are so large that researchers can make sense of only tiny pieces at a time, and some leading figures doubt fine-grained understanding will ever be possible. Cammarata was one of them. "I was very pessimistic," he says.

The takeaway: for five years, the field treated neural networks as fundamentally tangled, and built its whole research program around that assumption.

What changed when he looked again?

At Goodfire, Cammarata has spent nearly all his time peering inside networks, building new tools and deploying teams of AI agents to help. "I spent like, I don't know, 10 hours a day, seven days a week, for most of the last year just looking at neurons," he says.

His realization: the networks have a logic after all, even when they first appear senseless. "We were looking at it wrong," he says. "I just think you want to try slightly harder; you don't want to give up too quickly."

He describes the old superposition findings as a kind of confirmation bias. Researchers expected to be flummoxed, so they were. "Every time I couldn't understand something, I was like, 'Wow, this untangling process is even more important than I thought, because I can't make any progress without it,'" he says. Instead, "I was able to, like, map out whole circuitry."

The proof of concept came from returning to Inception v1—a model he'd studied "for probably thousands of hours"—and the car-and-cat neuron. Rather than asking why unrelated things were packed into one neuron, he asked what they might share.

The answer took five seconds: a V shape. Cat faces and cars seen from the side both feature a prominent V. He tested what else the neuron fired on: the corner of a pool table, the pointed nose of a Japanese high-speed train, a fire boat shooting water from its deck. It switched off on round, O-shaped images—a kind of anti-V.

"I was looking at the same images; there was no technical change," he says. "And I was like, 'Whoa, okay, I think I might just be misunderstanding neural networks in general.'"

The takeaway: the field's canonical evidence for tangled networks may have been a case of researchers asking the wrong question.

Is full understanding actually possible now?

Cammarata is careful here. He doesn't claim to fully understand LLMs. He claims full understanding is no longer impossible. "I now feel like if I had 100,000 years to go look through a frontier language model just one circuit at a time, there's, like, a pretty good shot I would come up with a good understanding," he says.

His optimism scales with automation. "You're about to get insanely smart agents at a ridiculous scale," he says. "And what would have taken me, you know, 100,000 years, we will be able to do much faster. Automated scientists are getting really good."

He and his colleagues have begun applying the approach to modern LLMs and plan to publish technical results soon. Those details will matter, because skeptics are already pushing back hard.

The takeaway: the claim rests on a bet that automated analysis can compress a 100,000-year manual audit into something tractable.

What are the skeptics saying?

Chris Potts at Stanford University, who leads a team studying how LLMs work, is blunt: "As far as I can tell, nothing about this blog post makes the problems any easier. It would not have occurred to me to say this at all, but very prominent people on social media seem to believe that it is an all-encompassing breakthrough." He calls what Cammarata has shared "little more than a hunch" and adds: "I appreciate that he looked at thousands of examples, but even a large number here is not a substitute for rigorous evaluation."

Jacob Hilton at the Alignment Research Center in Berkeley, a past collaborator, notes that researchers understand less than 0.01% of the inner workings of top-tier LLMs. "I always appreciate Nick [Cammarata]'s boundless optimism when we hang out, but unfortunately this is giving the public an unrealistic picture of the state of the field," he wrote on X. "For a full understanding, we need to go beyond neurons, and explain how they connect together to form circuits. How is this going? If we are aiming for a full understanding, things are not looking good!"

Neel Nanda at Google DeepMind, who helped pioneer mechanistic interpretability alongside Cammarata, is also unconvinced. "This is interesting work, but I don't think it's a significant update to the big picture of interpretability," he wrote. He agrees that understanding individual neurons could help explain why specific models misbehave, but rejects grander readings: "I do not agree with some people's grander inferences like that this is substantial progress towards ambitious reverse engineering [or] solving interpretability."

Cammarata concedes the point partially. "It's not infinitely good news; it's not everything solved," he says. "There's a bunch of open questions around, like, how much the combinations of neurons matter versus just one at a time."

The takeaway: even sympathetic critics say single neurons are one piece of a far larger puzzle, and the technical results aren't published yet.

What happens next?

The immediate test is the pending technical paper on modern LLMs. Inception v1 is a 2014 image classifier, far simpler than today's frontier models, and the difference between "a V shape in a vision model" and "readable circuitry in an LLM" is exactly where the skeptics place their bets.

Cammarata's own framing is modest but pointed. "I think our job is to sort of try to think like the neural network," he says. "My guess is that things will turn out pretty well, because it was in some sense designed to be read."