Stories
OpenAI Foils Large-Scale AI Model Copying Attempt, Cites National Security Risks
- •OpenAI disrupted a coordinated campaign to extract protected reasoning from its AI models to train or improve other models, a practice termed adversarial distillation.
- •The activity, which began in July, involved manipulating model interactions to reproduce hidden reasoning, not breaching encryption or databases.
- •A core cluster of the activity was attributed to individuals associated with Moonshot AI, developer of Kimi.
- •OpenAI warns that adversarial distillation poses safety and national security risks, as extracted reasoning could be used to train models without original safeguards.
- •The company implemented account enforcement, technical controls, and strengthened protections to disrupt the campaign and expects more sophisticated attempts in the future.
Key Questions Answered
Adversarial distillation is the unauthorized use of a model's outputs or reasoning to train or improve another model. OpenAI disrupted the coordinated campaign by banning fraudulent accounts, strengthening infrastructure controls, expanding monitoring, and closing replay pathways for encrypted reasoning.
A core cluster of the extraction activity was attributed to individuals associated with Moonshot AI, the developer of Kimi. However, OpenAI stated it remained unclear whether all operators observed during the period originated from a single actor.
Rivals including OpenAI, Anthropic, and Google have been working together through the Frontier Model Forum to share information and detect adversarial distillation attempts by competitors. US officials estimated that unauthorized distillation costs Silicon Valley labs billions of dollars annually in lost profit.
