Stories
OpenAI Safety Employee Quits, Warns "Time for Trial and Error is Over"
- •A former OpenAI safety employee, David Robinson, resigned and criticized the company's rapid development culture, arguing it increases the risk of AI failures.
- •Robinson stated in The Atlantic that AI companies, including OpenAI, are not being "nearly careful enough" and need to prioritize safety expertise.
- •He emphasized that advanced AI systems require safeguards comparable to those in nuclear power and aviation, not continued "trial and error."
- •Robinson highlighted OpenAI's reliance on "iterative deployment," where systems are released and safeguards are strengthened only after problems emerge.
- •He warned that AI capabilities are advancing faster than researchers' understanding of alignment, which ensures AI systems act in accordance with human goals.
Key Questions Answered
David Robinson resigned because he believed OpenAI's fast-paced culture and rapid development cycle increased the risk of safety failures. He argued that the company was not being careful enough and that advanced AI systems need strict safeguards comparable to those in aviation or nuclear power.
Robinson criticized OpenAI's reliance on 'iterative deployment,' where systems are released quickly and safeguards are only strengthened after problems emerge. He also warned that AI capabilities are advancing much faster than researchers' understanding of alignment.
An OpenAI spokesperson stated that the company ensures its models do not become more capable than they can safely manage, and that they pause training or hold back models when they need to slow down.
