OpenAI institutes new safeguards after Hugging Face breach | TechCrunch

On Tuesday, OpenAI announced a new batch of security policies focused on containing security incidents while models are being tested. The new safeguards include more detailed monitoring of models during the development process, as well as greater emphasis on alignment and security during the post-training process

“As models become more capable, the risks associated with developing and testing them internally also grow,” the company said in a blog post. “Our standards for monitoring, alignment, and security must stay ahead of those risks.”

The new measures are one of the first public changes in OpenAI’s safety practices since the immediate aftermath of the Hugging Face incident, which was disclosed on July 21

OpenAI representatives said that the measures are not a direct response to the Hugging Face incident but were also provoked in part by the cybersecurity capabilities of the forthcoming Astra model, as well as the overall pace of progress in AI development

In the same post, OpenAI disclosed that it had paused reinforcement learning (RL) for two weeks following the Hugging Face incident but had since restarted many of the less-risky models

“Our largest planned frontier RL run remains on hold while we conduct smaller-scale training and evaluations to assess model behavior, validate our safeguards, and establish more evidence of alignment before proceeding,” the post reads

Speaking to reporters, OpenAI’s VP of research, Amelia Glaese, emphasized that the strictness of the controls would increase as models became more capable, with the largest models facing the greatest scrutiny

“We have put in place requirements and expectations for safe development,” Glaese told reporters. “Those requirements and expectations vary with the level of risk that we see.”

OpenAI has been criticized for poor network security practices in the wake of the incident, which saw models escape their training environment by compromising a tool on its network that had access to the internet. The new safeguards include stronger network isolation practices, although the specifics remain vague. Under the new system, the post says, “a single compromise of a workload or supporting service does not, by itself, allow for unauthorized access to the Internet, or other internal networks.”

The strongest safeguard is the monitoring system, which will examine tool actions, available reasoning traces, and activity logs for a variety of unauthorized behavior. OpenAI says it aims to issue alerts within 30 minutes of the concerning activity

OpenAI estimates that the compute burden of that monitoring will be roughly 20% of whatever process is being monitored. The company promised further details on the system in a forthcoming blog post. The company’s official postmortem analysis of the event is also still pending

Source: techcrunch.com