Policy & Regulation

California proposes mandatory independent AI risk reviews

California’s November 18 AI roadmap would require licensed independent reviews of large developers. Fathom urges faster implementation despite unresolved capacity and enforcement questions.

Fathom Urges Rapid Implementation of California’s AI Oversight Plan
Fathom Urges Rapid Implementation of California’s AI Oversight PlanAI-generated
By Elena Vasquez5 min read

Why it matters

  • California’s Government Operations Agency released its frontier AI roadmap on November 18, 2026.
  • The roadmap calls for continuing licensed reviews of every large frontier AI developer.
  • The proposal builds on SB 813’s IVO framework and AB 1405’s AI-auditor registry.
  • Fathom’s October 30, 2026 implementation memo proposes phasing mandates in as evaluator capacity grows.
  • Fathom CEO Andrew Freedman urged implementation with “all deliberate speed” on November 18, 2026.

By Elena Vasquez

On November 18, 2026, California’s Government Operations Agency (GovOps) recommended continuing reviews of every large frontier AI developer by licensed independent verification organizations (IVOs).

GovOps dated the report November 16, 2026, then released it publicly on November 18. The roadmap recommends legislative changes and does not impose requirements itself.

For large frontier developers, the proposal would make recurring independent testing a statutory obligation if California’s Legislature adopted it. Initially, developers would not control which outside reviewer assessed them. California would also control what happened after an adverse finding.

AI governance group Fathom said the roadmap reflected many of its recommendations. Fathom also urged mandatory independent evaluations for catastrophic risks and pressed for quick implementation.

Fathom developed and advocated the IVO model. Under Fathom’s approach, outside evaluators assess AI models and systems for risk.

Fathom CEO and co-founder Andrew Freedman called the report a concrete plan for putting independent oversight into practice. He urged movement with “all deliberate speed.”

What would California require?

GovOps would require every large frontier AI developer to undergo continuing reviews by licensed IVOs. That system would replace current voluntary reviews for those developers.

California would license and continuously supervise the evaluators. The state could revoke a license when an evaluator lacked sufficient expertise, independence, or rigor.

IVOs would submit unredacted assessments to California. After engaging with the developer, the state would decide whether corrective action was warranted.

The design separates oversight into three roles. California would set outcomes and supervise evaluators, IVOs would measure compliance, and the Attorney General would enforce violations.

The roadmap builds on SB 813’s IVO framework and AB 1405’s AI-auditor registry. Its broader package includes baseline standards, emergency shutdown authority, and possible limits on recursive self-improvement.

These recommendations would give lawmakers a framework for both routine evaluation and emergency intervention. They would not activate those powers without further legislation.

How would safety tests work?

Fathom says the report tracks its October 30, 2026 implementation memo on two points: responsibility and the legal test. California would define acceptable safety outcomes in law.

IVOs would develop and update the technical methods and tests used to assess compliance. Fathom says that division lets testing methods change with the technology while preserving the state’s statutory outcome.

The proposed standard asks whether a frontier model is “unlikely to increase the risk of catastrophic harm above the level that would exist absent the model.” It also asks whether anticipated benefits outweigh the residual risk.

Fathom’s model gives technical standards room to evolve. It does not give IVOs power to rewrite the legal threshold set by California.

Recursive self-improvement, or RSI, describes the point at which automated AI development becomes self-sustaining. GovOps would make verification of automated AI research and development metrics an initial IVO responsibility.

Evaluators would also assess the growing risks associated with automated AI work. The report would make these checks an initial duty rather than a later expansion of the evaluator role.

Fathom’s memo also sought scrutiny of internal and external uses. Pre-registered capability thresholds would trigger state notification and a hold on further internal scaling pending reassessment.

At those thresholds, the state would receive notice before further internal scaling continued. The memo would put a temporary hold in place until the relevant reassessment, according to Fathom.

How would the evaluator market work?

Fathom proposes phasing in each mandate once GovOps determines that designated evaluators have sufficient capacity and expertise in the relevant risk area.

The group argues that a dependable expectation of mandatory work would attract capital and talent. That expectation would expand independent verification before a mature market existed in every risk area.

Initially, California would assign an IVO from a predesignated pool. The developer would pay directly on a time-and-materials basis at state-benchmarked rates.

Under the initial arrangement, developers would know which evaluator handled their case but could not shop the market. Later phases would introduce more choice only after the pool grew deeper.

Developer choice would expand in later phases as the pool grew. The state would retain powers to veto selections, require rotation, cap concentration, appoint additional assessors, and decertify evaluators that lacked rigor or independence.

Those controls would remain available after developer choice expanded. Fathom’s design does not treat the initial state assignment as the final word on evaluator selection.

Payments would not depend on an assessment’s outcome. Fathom says that structure keeps financial incentives separate from the evaluator’s findings.

Fathom’s sequence ties enforcement to market building. Every mandate depends on available expertise, so the state cannot expand duties beyond its evaluator capacity.

What access would evaluators receive?

Fathom’s design gives embedded evaluators continuous access to necessary records, personnel, and systems. They could attend defined decision points in person.

The access model would let evaluators check whether stated controls operated in practice. Fathom wants that access to cover records, personnel, and systems, not only final reports.

Fathom’s proposal backs that access with statutory confidentiality and anti-retaliation protections. The safeguards would support cooperation during sensitive reviews.

California’s Attorney General could seek injunctions and impose civil penalties. IVOs would make findings, prescribe corrective measures, and refer imminent risks to the state.

Evaluators would not seek injunctions themselves, and their findings would not operate automatically. State enforcement would supply the binding consequences.

That division keeps technical assessment within the IVO role. It leaves coercive action with California and its Attorney General.

What remains unresolved?

The practical test now belongs to the Legislature and GovOps. They must convert the recommendations into binding duties, develop enough licensed evaluators, and preserve credible state enforcement.

Freedman identified legislation, rulemaking, state-versus-federal authority, logistics, timing, and costs as unresolved. Fathom plans to press both state and federal governments for transparency, mandatory IVO risk measurement, and government-enforced guardrails.

Every mandate depends on available evaluator capacity. Every finding needs state action before it can carry coercive consequences.

A statutory vote alone would not settle the framework. GovOps would still need to determine when evaluator capacity supports each mandate. The state would also need to act on findings before they carried coercive force.

California now has a detailed route toward compulsory frontier AI testing. Its success will depend on whether lawmakers can write enforceable duties without outrunning the evaluator market’s capacity.

Source: unite.ai

Share this article:

More from Elena Vasquez

Elena Vasquez

Show full bio

Market editor covering technology and product news in artificial intelligence and machine learning at AI In Context.

744 articles

Related articles

  1. Newsom Plan Would Give California Power to Shut Down Frontier AI Models
  2. OpenAI Publishes Frontier Governance Framework for AI Risk
  3. Australia to legislate mandatory safety rules for frontier AI
  4. Twenty-four AI firms sign White House voluntary safety pact
  5. Khanna Bill Would Ban Self-Improving AI Until Federal Safeguards Exist

« Previous articleNext article »