Research

Anthropic co-founder Jack Clark moves self-improving AI forecast to 2027

Anthropic's Opus 5.5 matched 24 human experts on the TasteVal benchmark using half the compute, prompting Jack Clark to move his recursive self-improvement forecast to 2027.

Anthropic co-founder Jack Clark moves his prediction for self-improving AI forward to 2027
Anthropic co-founder Jack Clark moves his prediction for self-improving AI forward to 2027Akela999 / pixabay
By Marcus Bennett5 min read

Why it matters

  • Opus 5.5 hit a compute multiplier of 2.3 on TasteVal, matching 24 human experts with less than half the compute
  • Research taste has roughly doubled every three months since December 2025, with doubling time ranging 1.7–5.0 months
  • Jack Clark moved his recursive self-improvement prediction from 2028 to as early as 2027
  • A single model run cost roughly one-thirtieth of an average expert attempt
  • A Fudan University study found AI agents contributed up to 55 percent of methodology proposals but humans made over 85 percent of final decisions

Anthropic's Opus 5.5 reached a compute multiplier of 2.3 on the new TasteVal benchmark, matching the scores of 24 human AI experts while using less than half the compute — and roughly one-thirtieth of the cost. On the strength of that number, Anthropic co-founder Jack Clark has moved his prediction for recursive self-improvement in AI forward from 2028 to 2027.

TasteVal is a new benchmark from the research lab P-Zero Research that measures how well AI systems design experiments on their own and interpret the results. The capability it tracks, which the lab calls "research taste," has roughly doubled every three months since December 2025, according to Clark, who co-founded Anthropic and writes the Import AI newsletter. Recursive self-improvement (RSI) describes the point at which AI systems develop their own successors without human researchers in the loop.

The stakes are straightforward. If AI models can plan and evaluate their own experiments, AI development becomes faster and cheaper — and a growing share of the research becomes harder for humans to verify.

What does TasteVal actually measure?

P-Zero Research defines research taste as three abilities: identifying which research questions are worth pursuing, designing experiments to answer them, and correctly interpreting the results. TasteVal only tests the last two. The choice of research direction, which many consider the core of scientific intuition, is made for the model in advance.

The benchmark consists of eight tasks, each specifying a fixed research goal from AI model development along with a measurable score. The tasks cover:

  • training data selection
  • pretraining and fine-tuning language models
  • training on human preferences
  • defending against adversarial prompts

The model decides which experiment to run next and uses the results to guide its following step. To prevent coding skill from skewing the outcome, a fixed coding agent runs all experiments on a single Nvidia H100 GPU. Each run ends after 40 GPU-hours of compute or 120 hours of wall-clock time.

The human baseline came from 24 experts with recent stints at OpenAI, Google DeepMind, Nvidia, and Microsoft Research. They tackled the same tasks in teams of at least two people, and the best human attempt per task served as the benchmark.

How do you quantify "research taste"?

P-Zero Research translates the concept into compute efficiency. The reasoning: experiment compute is scarce in AI research, so a system that picks the right experiments and reads their results correctly needs fewer runs to make progress on the same budget.

The lab draws on its "AI Futures Model," which predicts that experiment compute becomes the biggest bottleneck once coding is automated. TasteVal therefore measures how much compute a model needs to match the experts' scores. A model that reaches expert level with half the compute has, by this definition, twice the research taste. The lab calls this ratio the compute multiplier.

Opus 5.5's headline figure of 2.3 carries wide error bars. The statistical range runs from 1.15 to 4.37, meaning at the low end the model would barely lead the human experts. The doubling trend is similarly uncertain: the reported doubling time ranges from 1.7 to 5.0 months.

Why the curve steepened only recently

In prior years, model research taste grew much more slowly, and the curve only steepened around the turn of 2025 into 2026. The authors themselves concede that a single benchmark cannot establish a lasting trend, and they admit their results may overstate "how quickly research taste is improving."

Two caveats matter. The tasks in TasteVal can be checked quickly and cheaply, which is the kind of task AI labs find easiest to optimize for. And the comparison group did not include any top-tier researchers.

Why did Clark move his timeline?

Clark has long called creativity the key factor in how fast AI systems can improve themselves. Earlier this year, he wrote that models had not shown this ability at any groundbreaking scale. He now points to progress in mathematical breakthroughs and benchmarks like TasteVal as signs that this is changing.

His earlier forecast was more cautious. In a long essay in May, Clark put the odds of a system that independently trains a stronger successor at roughly 60 percent by the end of 2028 and 30 percent by 2027. One bias factor deserves note: the forecast comes from someone who co-founded the company whose model, Opus 5.5, tops the benchmark.

What did critics say — and does TasteVal answer them?

TasteVal measures the exact capability that skeptics flagged as a barrier to a major AI breakthrough. AI researcher Herbie Bradley pushed back on Clark's May essay, arguing that while models could plausibly take over the work of junior researchers, more demanding skills like research taste, creativity, and coherent long-term research planning were nowhere in sight.

On planning and evaluating experiments, TasteVal pushes against that assessment. It says nothing about choosing a long-term research agenda, which remains untested.

A study led by a team at Fudan University, in which researchers analyzed the development of their own model using AI agents, shows how wide the practical gap remains. AI agents contributed up to 55 percent of methodology proposals, but humans still made over 85 percent of the final decisions. The authors noted it remains unclear how AI can generate diverse research directions and assess their value before results are in — exactly the kind of decision TasteVal takes off the models' plate.

What does this change about the debate?

The benchmark arrives in the middle of a debate the major AI labs started themselves. In June, Anthropic wrote that recursive self-improvement has not been achieved yet but "could come sooner than most institutions are prepared for." Anthropic CEO Dario Amodei now calls for a controlled slowdown in AI development, while OpenAI acknowledges that fully autonomous RSI is not happening yet and says it should only be pursued if human oversight is guaranteed.

TasteVal shifts what counts as progress. The long-standing focus on how well models solve well-defined tasks like math problems or coding challenges is giving way to a harder question: whether models can make the decisions that push research forward.

If the trend P-Zero Research measured holds up in other tests, the implications cut both ways. AI development would get faster and cheaper. At the same time, a growing share of the research would become harder for humans to verify, because models would be planning and evaluating their own experiments.

Original: importai.substack.com

Share this article:

More from Marcus Bennett

Marcus Bennett

Show full bio

Senior reporter covering companies, deals and strategy in artificial intelligence and machine learning at AI In Context.

349 articles

Related articles

  1. AI Research Agents Reach Just 15% of Human Benchmark, Study Finds
  2. Taste-Bench: AI Agents Flunk Long-Horizon Strategy Decisions
  3. OpenAI Predicts AI-Made Discoveries by 2026 as Intelligence Costs Plunge
  4. Recursive launches Flint, a laptop-sized coding model its AI built
  5. OpenAI Warns Its Own Monitoring Tools Are Failing as AI Nears Self-Improvement

« Previous articleNext article »