Sakana AI's self-improving model wins nearly 70% of matchups

Sakana AI's MASS method pushed a single language model to a win rate of nearly 70 percent against its own base model — after just two rounds of self-improvement. That is the headline number from the Tokyo-based lab's approach, in which one model plays three roles at once: it solves tasks, judges the results, and refines the workflows it uses along the way.

The appeal is obvious. If a model can grade its own homework, you need fewer humans in the loop and less external infrastructure. Sakana AI's design treats improvement as a closed loop: generate, evaluate, adjust, repeat. Two cycles of that loop were enough to move the improved model from parity with its base version to winning roughly seven out of ten head-to-head comparisons.

But the same paper carries a warning, and it's the part researchers will read twice. When the model misses the same errors in all three of its roles — as solver, as judge, and as workflow-refiner — it mistakes those errors for progress. The blind spot then gets reinforced during training rather than corrected. The system, in effect, confidently confirms itself.

What does MASS actually do?

The method, described by Sakana AI as MASS, assigns one language model three intertwined jobs:

• Solving — attempting the target tasks directly.
• Judging — evaluating the quality of its own outputs.
• Refining workflows — adjusting the processes it follows so future attempts improve.

Each round of the loop feeds the next. The near-70-percent win rate after two rounds is the measured result of that iteration, benchmarked against the unimproved base model. The figure gives a concrete sense of how much headroom pure self-improvement can unlock in a short horizon — no new base model, no human graders, just the loop running twice.

The gain is real, and that is why it matters. Self-improvement is one of the most contested frontiers in AI research, sitting at the intersection of cost, scale and safety. Human feedback remains expensive and slow. If models can substitute their own judgments for ours, the economics of training change. Sakana AI's numbers suggest the substitution works better than skeptics might assume — at least for two rounds, at least on these tasks.

Why can't the model catch its own blind spots?

Here the story turns. The failure mode Sakana AI documents is structural, not incidental. A judge can only flag errors it recognizes. When solver, judge and refiner share the same blind spot — the same inability to see that something is wrong — the loop has no internal signal that anything is off. The model reads consistency as correctness.

That misreading then gets baked in. Because the workflow refinement step treats the judged results as ground truth, an error that survives all three roles looks identical, from the inside, to a genuine improvement. Training amplifies it. The paper's own framing is blunt on this point: the model "can mistake them for progress and reinforce them during training."

This is the classic problem of correlated failure, and it lands at a moment when the industry is actively building self-evaluating systems. The stakes run in two directions. On the research side, self-improvement promises cheaper, faster model iteration. On the safety side, a system that reinforces its own unrecognized errors is a system whose mistakes compound quietly — the opposite of a self-correcting machine.

The near-70-percent win rate and the blind-spot failure are two faces of the same design. The loop's strength — tight, fast, internally consistent feedback — is exactly what makes its shared errors durable. There is no outside perspective inside the loop, and the paper's results show what happens when one is missing.

What comes next?

Sakana AI's results suggest the immediate follow-up question for this line of work: how do you inject an external check into a self-improvement loop without giving up its cost advantages? The paper doesn't answer that, but it defines the problem precisely — errors that survive all three roles are the ones that get reinforced, so any fix has to break the correlation between solver, judge and refiner. Until something does, the honest reading of MASS is double-edged: two rounds buy you a nearly 70 percent win rate, and they can also buy you a mistake you've taught yourself to believe in.

Elena Vasquez reports on AI research and model evaluation for AI In Context.

---

Correction note for readers: this report is based on Sakana AI's published description of the MASS method and its reported results. The near-70-percent win rate, the two-round figure and the blind-spot failure mode are drawn directly from that source.