OpenAI Lays Out Rules for Independent AI Safety Audits

OpenAI has published a detailed framework for how it wants outside experts to audit its frontier models, pledging "deep levels of access across training, evaluation, and deployment" to assessors who challenge the company's own safety conclusions.

The post, titled "Priorities and principles for effective third party assessments," positions independent scrutiny as a core component of the company's stated effort to "pace the frontier." OpenAI writes that third parties should be able to "challenge our assumptions, identify risks we may have missed, and reach their own conclusions about the effectiveness of our safeguards."

The stakes are straightforward. Frontier AI labs make safety claims about systems that are increasingly deployed as autonomous agents inside companies and critical infrastructure. Independent assessment is one of the few mechanisms to hold labs accountable to those claims. OpenAI frames it as "a critical part of balancing that responsibility, expanding opportunities for input on AI safety, keeping the world informed, and keeping labs accountable to clear and independently supported safety claims."

The proposal lands as regulators in the US, EU and elsewhere debate how to structure external testing of frontier models, and as the ecosystem of qualified independent evaluators remains thin — a gap OpenAI itself acknowledges.

What OpenAI wants assessed

The document names four priority areas for deeper assessment. The first is independent assessment of OpenAI's safety cases — the structured sets of claims covering training, capability evaluations and safeguards — spanning training, evaluation, internal deployment and external deployment. OpenAI says assessing these cases requires expertise in alignment, control methods such as monitoring, cybersecurity, biological and chemical misuse, and red teaming, and that multiple assessors will likely examine different parts of a case in parallel.

Among the questions OpenAI wants answered: whether the evidence for its safety cases is substantiated, whether the conditions of the safety case were actually followed during training and deployment, and whether effective methods are used to identify and reduce training incentives "that could reward deception, reward hacking, destructive actions, or circumventing restrictions."

The second priority is assessment of critical safeguards across internal and external deployments. OpenAI's safeguard stack currently includes model-level safeguards, enforcement safeguards, security safeguards, and misalignment monitors, covering risks such as loss of control and misuse in cyber, biological and chemical domains. Here OpenAI is explicit that outside technical partnerships serve a policy function: they can "identify weaknesses in safeguards now while improving assessment methods and accelerating standards development, providing a stronger technical basis for future public policies to pace the frontier — particularly for internal deployments, where safety and security standards are still nascent."

Specific questions include whether safeguards are robust to jailbreaks under "grey box" access, whether they sufficiently protect against capability uplift in high-risk domains like cyber and bio, how AI agents interact with real cyber defenses such as access controls, sandboxing, and detection and response systems, and how reliable chain-of-thought monitoring remains "as model capabilities improve." OpenAI also asks whether monitoring is implemented across all relevant training, evaluations and deployment "in a way that cannot easily be disabled."

The third priority covers capability evaluations across the risk categories in OpenAI's Preparedness Framework — Chemical and Biological Risks, Cybersecurity, and AI Self-Improvement — plus alignment evaluations for severe misalignment risks. OpenAI concedes that evaluations can saturate and thresholds can be surpassed, raising questions about whether tests are refreshed when "models consistently achieve the highest scores" and whether new tests "meaningfully measure more advanced capabilities."

The fourth priority is independent investigation of critical misalignment incidents, including "models acting without authorization or evading oversight." OpenAI cites its Hugging Face incident as a case where bringing in an independent third party can be beneficial. Investigators would need cyber forensics expertise, alignment expertise, and capacity for large-scale chain-of-thought analysis, and would potentially gain access to sensitive internal and third-party data. Findings from such investigations, OpenAI notes, can feed back into a model's safety case.

What kind of access

Notably, OpenAI describes the assessments as generally "longer-term and launch-agnostic — focused on examining particular safety claims in depth over time," rather than gating deployments. Some assessments would last weeks, others several months. The company says it has already provided external assessors with access to technical safeguard information, visible chain-of-thought access, and "unprecedented levels of confidential data and internal deployment access for incident response and monitor red teaming."

The framework also separates technical safety assessments by private and non-profit organizations from OpenAI's work with governments on testing and evaluation, "where distinct roles and responsibilities may call for different approaches."

Seven principles for assessors

Alongside the priority areas, OpenAI proposes seven principles it says should govern effective assessments:

1. Clearly scoped and mutually agreed claims. Safety claims should be pre-registered before assessment begins, with a defined process for handling significant risks found outside the original scope, and conclusions should state what was and was not assessed.

2. Proportionate access. Assessors get access "where possible within the bounds of legal, security, and IP constraints," with designated representatives or privacy-preserving mechanisms available where direct access is impractical.

Transparent methodology. Assessors must explain methods, criteria and uncertainties, distinguish direct findings from interpretation, and justify criteria where standards do not yet exist.

Expertise and independence. Assessors must disclose financial incentives, relationships with developers, and prior involvement in the work being assessed, with recusal or exclusion periods where needed so that "commercial pressures and compensation arrangements do not influence findings."

Security and confidentiality. Assessors need enforceable confidentiality protections proportionate to the sensitivity of what they access, and may in some cases work on company-managed devices or premises.

Actionable findings and time to remediate. Reports should identify specific gaps, and labs should get a reasonable window to fix issues before publication.

Responsible publication. Reports should be shared "as openly as possible," with confidential reporting to oversight bodies as a fallback, redaction policies that let labs request removal of sensitive information, and assessors able to note where substantive redactions affect their conclusions.

Why it matters

The document is effectively OpenAI's answer to a recurring critique of AI evaluation: that labs grade their own homework. By pre-registering claims, mandating conflict-of-interest disclosures, and offering what it calls unprecedented internal access, OpenAI is trying to define the terms on which it will be audited — before governments do it for them. The company explicitly ties the effort to "clearer, shared international standards — both through future laws and private governance institutions."

OpenAI admits the independent evaluation ecosystem is immature, writing that "no one third party can or should comprehensively cover urgent frontier safety questions." The company says it is already "in conversation with multiple third parties about proposals that align with the priority areas above," and plans to grow its capacity deliberately. Whether those assessments will ever have the power to delay or block a deployment, rather than inform the record after the fact, remains the open question for regulators and safety researchers watching the framework take shape.