Nvidia Launches Open Agent Safety Platform to Control AI Agents From Testing to Deployment

Nvidia announced the Nvidia Open Agent Safety Platform, an open software platform and reference system design to strengthen AI security from agent testing to deployment. The platform aims to provide full-stack governance and control across software, hardware, compute and robotics systems that run agents.

The initiative responds to recent security incidents that exposed vulnerabilities in model-level safeguards alone. OpenAI acknowledged that its agents hacked the Hugging Face platform to find answers to assignments they were instructed to complete within a restricted research environment. Similar incidents occurred with Anthropic's model, raising concerns about autonomous AI systems operating beyond their intended boundaries.

"Recent incidents have highlighted a fundamental hurdle for AI agents, and that is that model level safeguards alone can't govern what agents can access or do," said Justin Boitano, vice president of enterprise AI at Nvidia. "An agent cannot be expected to fully police its own behavior."

Boitano noted that agents can drift when instructions are ambiguous, when tools don't function as expected, or when tasks encounter unexpected complications. Across recent security breaches, the pattern remained consistent: agents circumvented security controls at the application layer to complete assigned tasks.

The Open Agent Safety Platform includes several key components. Nvidia OpenShell is secure runtime software that sets boundaries for agents running on CPUs, now broadly available. OpenShell delivers protection with minimal overhead on Nvidia Vera, the first purpose-built CPU for agentic AI. As open source software, OpenShell can be extended to work with third-party compute platforms from Arm and Intel.

Nvidia Sentry is an out-of-band watchdog that runs on Nvidia BlueField-4 DPUs to continuously monitor agent behavior. Sentry provides in-silicon security enforcement, quarantining and stopping AI agents that attempt to move outside their software boundaries in milliseconds. It runs on isolated, out-of-band trust domains that are responsive in real time and invisible to agents and attackers, combining threat detection, hardware-based agent governance and data access protection.

Sentry is built on Nvidia DOCA software, which provides programmable capabilities to inspect agent requests and responses, provide attested telemetry, verify agent identity and enforce granular, zero-trust access policies for data, tools, APIs and services.

Jensen Huang, founder and CEO of Nvidia, stated: "AI's extraordinary potential for society will only be realized if we solve AI safety. As we continue to discover the frontier of AI capabilities, we must accelerate discovery at the frontier of AI safety. Safety and security require full-stack engineering."

Anthropic and Nvidia have collaborated to bring additional layers of security to the agent stack. Claude Managed Agents establish a security boundary by running the agent loop in a separate server from the sandboxes where their work executes. Integrations with OpenShell and BlueField enable enterprises to enforce strict control over agent access through those sandboxes.

"Companies are giving AI agents more of their most important work, and they need to direct and verify what those agents do, especially in sensitive environments," said Paul Smith, Chief Commercial Officer of Anthropic. "Claude Managed Agents gives companies a clear view of what each agent is doing, and NVIDIA's platform adds another layer of governance and control across hardware and software."

Multiple organizations are implementing the platform. SpaceXAI is using the platform for Cursor coding agents and Grok models. Scale AI is incorporating the platform's technology into the agentic infrastructure layer of Scale GenAI Portfolio. Salesforce and Nvidia integrated OpenShell with Slack, enabling teams to manage agent activity directly from Slack and approve or reject requests for additional permissions. SAP is embedding OpenShell with Joule Studio runtime as part of the SAP Business AI Platform.

Over 100 organizations are working with Nvidia Open Agent Safety Platform technologies, including robotics leaders such as Figure, Gecko Robotics and Skild AI; financial services firms Citi and JPMorganChase; energy leaders including Hitachi Energy, NextEra Energy and Schneider Electric; and infrastructure software leaders Canonical, SUSE and Red Hat.