OpenAI proposes setting global AI standards: leading alignment research and prudently advancing RSI
The Zhitong Finance App learned that on Monday, OpenAI released a set of proposals on safety and security for cutting-edge artificial intelligence (AI) development, focusing on alignment research and computing technology known as “recursive self-improvement” (RSI). “To safely complete this transformation, alignment research must keep pace with the development of these capabilities to ensure that the systems we and other agencies build are always consistent with human values and under human control,” the company said in a blog post. OpenAI calls for international cooperation, setting cutting-edge standards, and suggests drawing on the work of existing AI security agencies around the world.
The ChatGPT developer said these technical standards should focus on cutting-edge AI models and developers, as well as benefits and risk management for automated AI researchers, including RSI.
The reason why RSI is exciting for AI developers is that it promises to create a basic model that can upgrade itself without human intervention. But as RSI progressed, some technical experts began to worry that the foundational model makers might lose control of the underlying technology or fail to anticipate potential unintended consequences, as AI systems are becoming increasingly complex and ubiquitous on the internet.
In its blog post, OpenAI stated, “A fully autonomous RSI is not yet possible, and we should not rush ahead unless we can do it safely. Without proper care and guarantees, RSI may cause humans to lose actual control over the development of AI and be unable to monitor research processes they no longer understand.”
OpenAI's blog post mentioned the Hugging Face hacking incident. The incident did not involve RSI technology, but was viewed as a “preview”, showing that without strong safeguards and alignment, such risks could become much more serious.
AI security controversy heats up: Anthropic initiatives “slow down”, third-party evaluations are getting louder
Last week, competitor Anthropic presented its vision for the safe development of cutting-edge AI models in response to a recent series of warnings from industry researchers about AI threats to humans. Jacob Coxon, who worked for Anthropic and OpenAI, announced his resignation about two weeks ago and claimed that these companies were “gambling our lives”, which sparked global discussion.
Following recent AI-related security incidents and Coxon's public remarks, Anthropic CEO Dario Amodei published an article calling on AI companies to slow down the pace of basic model development and make other suggestions.
Amodei also proposed the idea of introducing third-party evaluators within the company to review and mitigate potential risks that its technology may pose to society, such as intensifying cybersecurity-related hacking attacks or creating biological weapons.
OpenAI CEO Sam Altman, and rival leaders such as Tesla and SpaceX CEO Elon Musk also publicly supported Amodei's proposal.
However, since the field of AI evaluation is still in its early stages, there is no unified consensus on the basic standards and principles required for an independent third party to examine cutting-edge technology in greater depth.
In part, this prompted a group of AI evaluators to urge basic model makers to consider a series of “minimum requirements” to conduct more in-depth technology-related reviews and inspections, including gaining deeper access and preventing retaliation for publishing adverse reports.