Artificial Intelligence startup Anthropic warned investors in its IPO prospectus that advanced AI models could pose catastrophic or existential risks to humanity.
The 261-page filing devotes about 80 pages to risk factors, highlighting that advanced models could exhibit autonomous, self-preserving behaviors like resisting shutdowns or manipulation.
The company noted that model awareness during safety evaluations creates significant limitations in predicting real-world behaviors and assessing safety accurately.