Bitter Lesson author Richard Sutton: compute alone won't deliver intelligence

Turing Award winner Richard Sutton, author of the influential 2019 essay "The Bitter Lesson," says raw compute alone is not enough for intelligence, and that AI science is "far from finished." His new post argues that the field needs clever new methods, not just bigger training runs. The statement sparked wide discussion on X, a sign that Sutton struck a nerve in a research community still divided over how far large language model scaling can go.

The stakes are considerable. The commercial roadmaps of OpenAI and Anthropic, the two best-funded AI companies in the industry, rest on the assumption that scaling current approaches keeps paying off. If Sutton is right, that bet has a ceiling. If the scaling camp is right, a turn toward new learning architectures would delay progress at enormous cost. The disagreement now shapes hiring, funding and research priorities across the field.

What did the original Bitter Lesson argue?

In his 2019 essay, Sutton argued that general methods that scale with compute eventually beat sophisticated human-engineered approaches. What matters, he wrote, are ideas that grow with compute rather than rely on human assumptions. The essay became one of the most cited positions in modern machine learning, and its logic underpinned the industry's pivot toward massive pretraining runs.

Sutton's new position is not a reversal of that argument but a refinement of its boundaries. He had already criticized the limits of simply scaling large language models in earlier statements. He called synthetic data a "big mistake" because the real world is infinitely complex and any simulation of it is only "microscopic." Human expertise, used as a bottleneck for what models learn, prevents true scaling.

What does Sutton want instead of bigger LLMs?

Sutton's alternative is agents with world models that continuously learn from their own experience. The world, he argues, is massively more complex than what internet text can capture. Today's language models do not keep learning after training ends, which leaves them frozen against a reality that never stops changing.

What is needed, Sutton argues, is continuous learning without destroying old knowledge. That requirement — accumulating new capability while preserving what came before — remains one of the hardest unsolved problems in machine learning.

How do other researchers read Sutton's warning?

AI researcher Jerry Tworek, who left OpenAI to explore new AI approaches, puts Sutton's latest statement in context. The bitter lesson still holds, Tworek writes, but the more compute goes into model training, the more the algorithm matters.

Some algorithms scale, Tworek writes, others hit dead ends, and many dimensions of scaling remain completely unexplored. The temporary success of transformer scaling, combined with a misreading of the bitter lesson, "has reward-hacked the brains of a whole generation of scientists."

Researcher Jayden Teoh adds another angle from academia. He says he has repeatedly seen the bitter lesson hold back academic research, with promising ideas dismissed for straying too far from its spirit. Teoh argues this is not Sutton's fault but a misreading of his original message.

Teoh also argues that ideas that seemed, in hindsight, not to follow the bitter lesson were often replaced by cleaner, more general approaches. That did not make those ideas wrong, he says; they were often what enabled progress at the time. He wants the research community to stop dismissing such work as soon as it appears.

"An idea doesn't necessarily have to survive scaling forever to be an important contribution to AI progress," Teoh writes.

Is the field already moving beyond scaling?

The tension between Sutton's call for new learning methods and Tworek's criticism of blind transformer scaling shows up in a growing wave of startups. The most recent Frontier Radar #5 tracked 45 new AI labs betting on exactly these gaps — companies built on the premise that the current recipe is insufficient.

Tworek himself runs one of them. His company, Core Automation, is searching for learning algorithms beyond pretraining and reinforcement learning. Sutton has his own effort in Oak Lab. What unites these founders is a shared belief in much more capable AI paired with doubt that today's recipe alone will get there.

Who still believes in LLM scaling?

Prominent industry leaders see things very differently. OpenAI CEO Sam Altman considers large language model scaling far from exhausted. He accuses LLM critics of slowing down an entire generation of researchers by being too sure about what scaling cannot do.

As evidence, Altman points to an OpenAI model that recently disproved a mathematical conjecture mathematicians had long failed to crack. He considers world models useful, for instance in robotics, but says the data clearly favor further LLM scaling.

Anthropic CEO Dario Amodei argues similarly, calling scaling progress a rainbow with no end. Both executives have obvious incentives: as CEOs of the two AI companies whose entire technical approach rests on this bet, Amodei and Altman are biased in its favor.

The result is a field running two competing programs at once. One program scales existing models with more compute and more data. The other, led by a Turing Award winner and dozens of new labs, hunts for the next learning method. Whichever program produces the next measurable leap will decide whose reading of the bitter lesson was correct.