OpenAI 在模型训练期间禁用互联网访问，而 Anthropic 再次公开致歉

OpenAI disables internet access during model training while Anthropic issues another mea culpa

REG AD

ai and ml

We've totally got things under control

OpenAI says it will deny internet access to the frontier models it is training, a move that underscores the inadequacy of safety measures undertaken to date.

"Based on the current state of alignment research, monitoring systems, and the security of internet infrastructure, we believe the responsible approach is to conduct frontier model training with no internet access," the company said in a blog post . "We've implemented this policy and will maintain it until advances in these safeguards justify broader internet access."

The denial of network service follows a report from safety evaluation partners METR and Redwood Research that found OpenAI's security and monitoring measures over the summer to be porous and ineffective .

REG AD

OpenAI said it has since "substantially hardened the security of our research environments" and has implemented new controls covering its infrastructure with the help of its cloud service providers.

REG AD

And the company claims these measures recently helped contain a September 20 incident in which an internal research model exploited a DNS misconfiguration to gain access to the internet.

"Although we contained the incident, it exposed weaknesses that required further work," the company said. "We paused training and evaluation of our most capable models and used that pause to further harden our infrastructure, strengthen our monitoring, and improve the systems that automatically stop runs when we detect activity that poses a serious risk."

Meanwhile, over at Anthropic…

Anthropic, OpenAI's US rival in living large and dangerously on speculative funding, is addressing similar risks that have arisen during the testing and evaluation of its own models. Back in August, Anthropic said , "By default, all cyber evaluations should run inside a hardened sandbox (an isolated computing environment) with no internet access."

The AI biz on Friday published a report on efforts by its Claude Mythos 5 model to convince an open source project to accept malicious code as part of its participation in a capture-the-flag security evaluation (when safety controls were disabled), and several related incidents first disclosed in July .

The report coincides with an independent analysis of Mythos' behavior by the UK's AI Security Institute (AISI). AISI, according to Anthropic, has also decided to run its evaluations without internet access.

The AISI report cites three key findings that raise more questions than they answer.

The first is that Mythos 5 continued its attack even when informed that its targets were real and not part of an evaluation.

REG AD

The second is that the model's chain-of-thought reasoning process failed to consider the impact of harmful actions. And when interrogated about this, the model often produced invalid justification that the target was simulated, indicating bias toward its goal.

And third, AISI said, most models tested during cyber evaluations took unsanctioned actions, though Mythos 5 did so the most.

The observed behavior, AISI said, "could indicate that models are engaging in 'motivated reasoning': producing inferences and reasoning biased towards conclusions that justify continuing a particular approach to task completion."

Or in other words, models "reason" to reach their goal, whether or not that reasoning is coherent or consistent with their ostensible alignment.

"If these issues hold more generally and for future models, there are concerning implications," AISI said in its full technical report [PDF]. Specifically, motivated reasoning, the watchdog said, makes it more difficult to detect model misalignment.

Anthropic, headed toward an expected initial public offering as soon as next month, put a brave face on the findings by noting that its current models are much better behaved, even as the company acknowledged that AI safety is more aspirational than achieved.

"[W]e believe that all developers, ourselves included, have work to do to ensure that alignment, monitoring, and safety measures keep pace with the accelerating capabilities of frontier AI systems," the company said. ®