Stories
OpenAI Blocks 15,000-User Campaign Linked to Chinese AI Startup Moonshot
- •OpenAI shut down a large-scale operation involving over 15,000 users attempting to probe its AI models, with activity traced to individuals associated with Chinese AI startup Moonshot AI.
- •The operation, described as ‘adversarial distillation,’ aimed to reproduce OpenAI’s protected capabilities by manipulating model interactions to extract hidden reasoning, potentially reducing development costs for competing models.
- •OpenAI clarified that the campaign did not involve a breach of its encryption or databases but focused on extracting information about its AI models' internal workings.
- •The disclosure highlights a widening dispute over AI model distillation between US and Chinese AI companies, with significant commercial, safety, and national security implications.
- •OpenAI has shared information about this activity with other AI developers through the Frontier Model Forum and government information-sharing networks, emphasizing the challenge of protecting embedded knowledge in advanced AI models.
Key Questions Answered
Adversarial distillation refers to using the behavior, outputs, or reasoning of one AI system to help build or refine another competing model. This technique can potentially reduce the time and money required to develop a new model by bypassing extensive training and safety work.
OpenAI discovered thousands of accounts intensifying sharply in July, with more than 16,000 requests made by over 4,000 users in a two-day period. Further investigation disrupted a wider cluster of over 15,000 users, leading the company to identify a central group associated with individuals linked to Moonshot AI.
Weeks before OpenAI's disclosure, Anthropic alleged that Chinese AI developers, including Moonshot AI and Alibaba, had used its Claude models to assist in developing their own systems. These allegations contributed to heightened scrutiny over the practice of model distillation.
