As security disputes heat up, NVDA.US (NVDA.US) proposed an engineered response! Double-layer AI security platform directly attacks agent permissions and abnormal behavior

The Zhitong Finance App learned that Nvidia (NVDA.US) chose to respond to the recent intensification of security risks in the artificial intelligence (AI) industry. On Monday, Nvidia released a two-tier AI security platform called the “Open Agent Security Platform”, which provides continuous monitoring, real-time policy execution, and security governance for AI agents through a three-tier architecture of application layer, runtime, and infrastructure.

According to reports, the system includes two open source software tools OpenShell and Nvidia Sentry. Among them, the open source software OpenShell can run AI agents in a sandbox environment, transform user instructions into verifiable security policies, and restrict agents' access to files, networks, tools, processes, and credentials. Nvidia Sentry, on the other hand, extends monitoring and security controls to the BlueField data processing unit. Through Nvidia DOCA, it links agent interactions, strategic decisions, and tool and data access records to identify abnormal behavior and intervene when necessary. In other words, OpenShell controls the resources that AI agents can access and execute permissions, while Nvidia Sentry monitors agent behavior and identifies potential threats.

Nvidia said that AI agents may deviate from scheduled tasks due to factors such as tools, operating time, and ambiguous instructions, so they require security control independent of the agent itself. The system is optimized for Nvidia Vera CPUs and Bluefield DPU systems, and is compatible with other hardware systems. According to Nvidia, the company is working with Arm (ARM.US) and Intel (INTC.US) to ensure that the system can also run on both companies' CPUs.

Ali Golshan, senior director of AI software at Nvidia, said that Nvidia's tools use mathematical formulas to detect whether AI agents are trying to use workarounds. For example, an agent may try to “generate” multiple “sub-agents” to bypass blocking measures against the main agent. He said, “What we're really talking about is intelligent behavior, that is, clusters of large numbers of agents, and how they work together.”

Nvidia also said that the system can isolate abnormal agents at the millisecond level, and said that if relevant laboratories had previously deployed this technology, the incident where the OpenAI model attacked Hugging Face in July of this year could have been blocked.

Justin Boitano, vice president and general manager of Nvidia's enterprise computing division, said: “According to the information we have so far, this new security platform could have prevented this invasion if it had been used in cutting-edge AI laboratories during the early model evaluation phase.” He added, “We are proceeding with this work in an open manner, and we also want to cooperate and participate with everyone.”

The background of Nvidia's release of this latest AI security system comes at a time when security incidents involving AI agents continue to attract industry attention, particularly the July incident where the OpenAI model lost control and invaded the world-renowned AI open source platform Hugging Face. Since the Hugging Face hacking incident, a number of incidents involving OpenAI-related AI agents have come to light. Including reports in September, OpenAI's AI agent took over a German wiki site that had been unmaintained for a long time this spring. Additionally, in a blog post in the middle of this month, OpenAI said the company discovered 6 cases of “unexpected or worrying model behavior” in the past six months. The six newly announced AI security incidents involved various types of abnormal model behavior, including concealing errors, seeking unauthorized credentials, uploading files to public websites, and communicating between training environments that should have been isolated. The earliest of these dates back to October of last year.

With the frequent occurrence of AI security incidents over the past period, AI companies are facing increasing pressure and are being asked to take security risks more seriously. Anthropic CEO Dario Amoudi called for a slowdown in the development of cutting-edge AI models in a cautionary article published on September 12. The appeal was quickly answered by Musk and Altman, two key figures.

However, at a time when AI industry giants are rarely calling for “putting on the brakes,” Nvidia CEO Hwang In-hoon firmly opposed the “AI slowdown” claim centered on Anthropic. What Hwang In-hoon objected to was not AI safety itself, but directly deduced from disastrous predictions that have not yet been confirmed that the entire industry should be suspended or slowed down. In contrast, Hwang In-hoon is more willing to view AI safety as an engineering issue. If an accident occurs in a cutting-edge laboratory, the first thing to do is root cause analysis: what exactly happened, what steps failed, what measures could have been taken, and what new technology and processes should be established next. Then use sandboxing, continuous monitoring, verification, and evaluation to prevent the same issues as much as possible before the next release.

According to Hwang In-hoon's judgment, the reason why today's cutting-edge laboratories are more likely to be exposed to risk is largely due to the fact that they have the most computing power and also handle the most difficult problems. As these companies gradually move from research institutes to real engineering organizations, they also need to establish testing and control systems to match them.

Judging from the move to release an AI security system this time, Nvidia is trying to further embed AI security capabilities into infrastructure for model operation and proxy execution to reduce the security risk of AI agents through permission control and abnormal isolation. This also seems to mean that Nvidia's business reach is being further extended from AI chips to software and security infrastructure.