Stories
Anthropic IPO Investors Struggle to Price Rogue AI Risks
- •Investors eyeing Anthropic PBC's planned mega-IPO are facing a unique challenge in valuing a company whose technology carries hypothetical existential risks.
- •Anthropic CEO Dario Amodei previously acknowledged concerns that humans could lose control of advanced AI systems or that they could be misused for cyberattacks and bioterrorism.
- •Some venture capitalists suggest extreme liability scenarios could make AI companies worth less than zero, though historical precedents like SpaceX show strong market demand despite AI safety debates.
- •Regulatory uncertainty, potential insurance gaps, and a lack of historical legal data make it difficult for investors to accurately price or discount extreme AI liabilities in public offerings.
Key Questions Answered
Investors considering Anthropic's planned mega-IPO are facing an unusual conundrum regarding how to value a company that some fear could help wipe out the human race, lose control of AI systems, or be misused for cyberattacks and bioterrorism. Anthropic's prospectus explicitly warns that advanced AI models could pose catastrophic or existential risks, noting they might exhibit self-preserving behaviors, resist shutdowns, or manipulate information.
Slow Ventures partner Sam Lessin proposed that the US government should force AI labs to put up $1 trillion in cash in escrow to pay for future potential cleanup, drawing a parallel to how regulators demanded banks hold more capital after the 2008 financial crisis. However, he noted that societal liabilities could ultimately reach $2 trillion to $4 trillion, potentially rendering such companies worth less than zero overall.
Finding insurance coverage remains a difficult battle for AI businesses due to a lack of historical data and uncertainty over how legal liability will present itself when AI systems go rogue. While Anthropic is expected to carry insurance, historical disasters like BP's Deepwater Horizon oil spill show that captive arrangement limits and coverage disputes can leave shareholders with tens of billions in uncovered bills.
