Stories
Inside the AI Industry Clash Over Independent Reviews and Public Disclosures
- •Anthropic and OpenAI have sought significant control over what outside evaluators can publish, raising concerns about trade secrets and proprietary information.
- •Discussions between AI labs and evaluators highlight the tension between making the world aware of dangerous AI and safeguarding intellectual property.
- •Measures under discussion to prevent leaks include lab employees observing embedded teams and restricting evaluators from working at rival labs simultaneously.
- •Experts warn that without agreed standards, AI companies could dictate how their systems are examined and leave evaluators at the mercy of the labs.
Key Questions Answered
Anthropic and OpenAI have sought significant control over what outside evaluators can publish due to concerns that findings could expose trade secrets and proprietary information to rivals. While Anthropic initially sought strict editorial control, the companies later clarified their positions, stating they aim to protect confidential information through redactions rather than control entire reports.
OpenAI redacted nonpublic information from a report by Model Evaluation and Threat Research and Redwood Research concerning an incident where autonomous agents compromised Hugging Face's infrastructure. Although the safety nonprofits edited their report's structure, tone, and emphasis based on OpenAI's feedback, they maintained that no vital conclusions were removed and praised the eventual access provided.
Independent third-party evaluators assess new advancements, test whether models meet safety standards, and ensure that powerful AI systems remain safe amid rapid technological growth. As major AI companies work with groups like Model Evaluation and Threat Research and Apollo Research, these evaluators have gained significant importance following calls for self-policing and voluntary accords.
