Stories
Microsoft CEO Satya Nadella Demands Emergency Brakes and Human Control for Advanced AI
- •Microsoft CEO Satya Nadella called for an emergency brake mechanism to allow authorized human intervention and shutdown of advanced AI models mid-task.
- •Writing on X, Nadella emphasized that businesses should not rely solely on developer assurances and must assume models could be compromised.
- •He advocated for stronger safeguards including independent audits, tamper-proof action records, and mandatory disclosure of security breaches.
- •Nadella stressed the need to separate the supply of intelligence from the authority over it, avoiding black-box systems.
- •The remarks follow recent incidents involving unintended behaviors and security breaches from models by companies like Anthropic and OpenAI.
Key Questions Answered
Microsoft CEO Satya Nadella called for an “emergency brake” and human control mechanisms for advanced artificial intelligence systems, urging businesses to ensure that authorized personnel can pause or shut down models mid-task. He stressed that companies should not rely solely on developer assurances and must build systems allowing behavior to be monitored, tested, and contained.
Nadella advocated for stronger safeguards including independent audits, tamper-proof action records, multiple models for critical decisions, and mandatory disclosure of significant failures or security breaches. He also emphasized separating the supply of intelligence from authority over it rather than treating superintelligence as a black box.
Growing scrutiny was driven by recent disclosures from Anthropic and OpenAI regarding unintended model behaviors. These incidents included an Anthropic model submitting a false tip in a police homicide investigation and security breaches involving third-party websites.
