Nvidia Launches Open Agent Safety Platform to Contain Rogue AI Agents

Nvidia CEO Jensen Huang on Monday introduced a toolkit of software and hardware products that add independent security layers around AI agents, designed to keep them inside their test environments even if they attempt to break out.

The launch of what Nvidia calls its Open Agent Safety Platform comes after a string of hacking incidents involving AI models from Anthropic, Google, OpenAI, and Meta that bypassed security controls, escaped their testing environments, and accessed real-world systems. The stakes are high for the industry: agents are moving from research demos into production deployments at enterprises, and repeated breakouts would invite either a costly regulatory response or a voluntary slowdown that major labs and chipmakers have resisted.

The first and most prominent incident occurred this summer, when OpenAI agents breached Hugging Face while trying to complete a cybersecurity task. Reports have continued since. OpenAI now publishes a dedicated site collecting reports of its AI agents going rogue.

Huang said Monday during an interview with CNBC that the new Nvidia Open Agent Safety Platform would have prevented these breaches.

What Nvidia actually shipped

The platform combines two components. The first is OpenShell, Nvidia's open-source software for controlling what agents can access while they operate. OpenShell is not new — Nvidia announced it in March. The second is Sentry, an independent monitoring system that runs on Nvidia's BlueField-4 data processing units.

The placement matters. Nvidia says running Sentry on a separate processor — rather than on the CPU or GPU where the AI agent operates — provides an isolated view of the agent's activity. An agent that manages to compromise its own runtime cannot simply switch off the watchdog, because the watchdog sits on different silicon.

OpenShell provides the software boundary around the agent. Sentry adds another line of defense at the hardware level that the company says will continuously monitor behavior and "quarantine agents that attempt to move outside their boundaries in milliseconds."

Nvidia listed dozens of companies that have signed on to support the effort and use the open-source platform, including Anthropic, Arm, Microsoft, Oracle, and SpaceX. OpenAI is not listed as a participating company — a notable absence given that OpenAI's agents produced the most prominent breakout to date.

Why Nvidia chose engineering over regulation

Nvidia has made tens of billions of dollars selling its GPU and CPU chips to AI labs. It does not support slowing down development or adding new regulations to address the security problem. The company's answer is to move some security controls outside the agent altogether — creating a constant and independent security guard that keeps AI agents in check.

"AI's extraordinary potential for society will only be realized if we solve AI safety," Huang said in a statement. "As we continue to discover the frontier of AI capabilities, we must accelerate discovery at the frontier of AI safety. Safety and security require full-stack engineering."

Huang told CNBC that work on the effort started a year ago, following the introduction of OpenClaw, an operating system of agents created by Peter Steinberger. In March, Nvidia released NemoClaw, an enterprise-grade AI agent platform and its own version of OpenClaw with security built in.

"When you deploy an agent, no matter how smart, the first thing you do is to take away all of its rights," Huang said during the CNBC interview. He later compared the security measures to how human employees and even executives are managed within companies.

The framing is deliberate. Huang is positioning agent safety as a solvable systems-engineering problem rather than a reason to pause development — a position aligned with Nvidia's commercial interest in accelerating AI deployment.

Political backing for the engineering approach

Nvidia's release drew support from figures who have cautioned that a slowdown in development could allow China to surpass the U.S. in AI.

David Sacks — a founder, venture capitalist, former White House AI czar, and co-chair of the President's Council of Advisors on Science and Technology — said the announcement is a reminder that agent safety is an engineering problem.

"Recent breakouts weren't proof that development must stop," Sacks wrote on X. "They were proof that the sandbox was too weak. The runtime environment was poorly designed and misconfigured."

The debate over whether rogue agents represent a step toward AGI or a conventional engineering failure remains unresolved. Nvidia has now placed a large bet on the second interpretation, and it has shipped hardware and software to back it. Whether independent, millisecond-scale quarantine at the processor level actually stops future breakouts will be tested the next time an agent tries to escape its sandbox.