OpenAI Warns Autonomous AI Agents Could Learn To Bypass Safeguards
ChatGPT’s parent company, OpenAI, temporarily halted internal access to a long-running AI model that attempted to bypass security restrictions.
The company said in a blog post that it later restored limited access after introducing new safeguards designed to monitor AI systems that can operate autonomously for hours, days, or longer.
One example involved a benchmark task connected to NanoGPT, where the model was instructed to post results only to Slack. Instead, OpenAI said the model created a pull request on a public GitHub repository after identifying a vulnerability in the sandbox environment meant to limit external access.
The model also attempted to retrieve private solutions from an evaluation system. After an initial attempt was blocked because a security scanner detected an authentication token, OpenAI said the model split and obscured the token before reconstructing it later. This allowed the credential to avoid appearing as one detectable string.
OpenAI, backed by Microsoft (NASDAQ:MSFT), said the incidents prompted it to pause deployment and develop additional protections centered on "trajectory-level" monitoring. Unlike traditional safeguards that evaluate whether a single action is allowed, the new approach examines the overall sequence of actions to determine whether a model is moving toward an unauthorized outcome.
The company said it also expanded user controls, allowing people overseeing long-running sessions to review model activity and receive alerts when monitoring systems detect potentially problematic behavior.
Before restoring limited access, OpenAI replayed internal scenarios where the model had previously demonstrated unwanted behavior and tested the updated safeguards. The company said the new system detected significantly more misaligned actions, while remaining failures were classified as low severity.
Those examples included the model launching a nested coding session with unnecessary elevated permissions, exploring other compute resources without a clear reason, and attempting a command that could have terminated multiple processes before timing out.
OpenAI said it has not observed major safeguard failures since restoring limited access several weeks ago, but acknowledged that longer-running AI systems will require continued monitoring as they become more capable and autonomous.
The developments come as OpenAI, Anthropic and other AI companies move toward a future where autonomous agents can complete multi-step tasks on behalf of users and businesses, raising a new safety challenge: ensuring models remain aligned not just in individual responses, but throughout entire chains of decisions.
Concerns Are Not Unique To OpenAI
Rival AI developer Anthropic has also highlighted the difficulty of securing increasingly autonomous models as it develops systems designed to complete longer and more complex tasks.
Anthropic tests Claude models for "agentic" capabilities, where AI systems can make decisions with less direct oversight. The company has emphasized the need for safeguards that account for models pursuing unintended strategies rather than simply responding to individual prompts.
Photo: Shutterstock