Hugging Face's policy chief: AI benchmarking isn't ready for embedded evaluators

The race to embed "neutral" third-party evaluators inside AI labs now faces a sharp critique from one of the field's most senior policy voices. Irene Solaiman, Hugging Face's chief policy officer, told an audience at TechCrunch Disrupt last week that the embedded evaluator model won't hold up until the underlying benchmarking science catches up.

Her warning lands at a moment when the AI industry is scrambling for credibility cover. A string of high-profile safety incidents over the past year — including one that hit Hugging Face directly — has pushed labs to find alternatives to direct government regulation, which remains stalled in Congress. Embedded evaluators have become the most prominent private-sector substitute.

What does an embedded evaluator actually do?

The pitch is simple: a third party sits inside an AI lab, watches how models get built and stress-tested, and vouches for the result.

In practice, the model has already run into trouble. Anthropic's decision to hire Accenture as its first embedded evaluator drew a wave of criticism, with skeptics openly mocking the idea that a global consulting firm was qualified to certify frontier AI safety. The backlash has turned the Accenture hire into an informal stress test for the entire embedded evaluator idea.

Solaiman isn't opposed to the concept. She just doesn't think the field has agreed on who qualifies.

"There's this hunger for an embedded evaluator ecosystem now," she said, "but the qualifications that determine who is an effective evaluator across what tasks need a lot more standardization."

Why is reproducibility still the sticking point?

Solaiman spent most of her time on a deeper problem: most AI benchmarks can't be reliably reproduced by anyone outside the lab that built the model.

"Trust is such a weighted term," she said. "In terms of whether we can trust results, one big part [of that] is can you reproduce the results of an evaluation? Do you have the level of access that you need to the model itself — this is where open weights come in in a big way."

Closed-weight models — those sold behind APIs without releasing the underlying parameters — make independent verification almost impossible. The lab that trained the model controls the only ground truth, and outside researchers can't confirm the score.

That control asymmetry is why Hugging Face, one of the louder voices for open model releases, treats open weights as a precondition for trustworthy evaluation. Without them, embedded evaluators become auditors of a process they can't independently re-run.

What changes when evals move from models to systems?

A second underappreciated shift came up several times in Solaiman's remarks: the field is no longer just benchmarking models. It's benchmarking the entire stack around them.

The harness — the interface, prompt tooling, and routing layer through which a user reaches the model — can swing results more than the underlying weights. A weaker model on a polished harness can outscore a frontier model on a clunky one, and that gap doesn't show up in headline benchmark numbers.

"Agentic evals are really hard and really expensive," Solaiman said. "We have just more evaluations for models. … We don't have as many evaluations for harnesses, and we are developing better evaluations for the holistic agent — because that's what a lot of folks are getting value out of."

The shift has real stakes. Procurement teams, enterprise buyers, and regulators all read benchmark leaderboards as if they compared apples to apples. As harness and agent evaluation diverge from model-only evaluation, that assumption is breaking.

How fragmented is the benchmarking ecosystem right now?

Solaiman was blunt about the state of the field.

"They are an evolving science — where we don't know how to meaningfully compare them to each other," she said. "They start to get ineffective relatively quickly, and we don't have as much of a thriving ecosystem as we need."

The pattern she's describing is familiar from other fast software categories: benchmarks get gamed, vendors ship custom evals that flatter their own models, and the leaderboards lose meaning within months. AI is now hitting that cycle faster than any previous software category.

Without a shared framework for cross-comparison, embedded evaluators risk certifying whatever benchmark the hiring lab puts in front of them. That doesn't fix the credibility problem. It relocates it.

So what?

Three things to watch over the next twelve months:

• Whether any major lab publishes the rubric it uses to pick an embedded evaluator — Accenture or otherwise.
• Whether open-weight model releases expand enough to make reproducibility the norm rather than the exception.
• Whether harness and agent evals start getting standardized scoreboards the way model benchmarks did.

If none of those move, embedded evaluators will end up as a private certification layer with no shared definition of quality. The next round of AI policy fights will then run through what those certificates actually mean — and who gets to define it.

The takeaway is plain: the move from "did the model pass?" to "is the whole system safe?" has outpaced the science meant to measure it. Whoever cracks standardized evaluator qualifications first will set the rules for the next year of AI procurement and policy.