Anthropic’s IPO pitch includes a warning about human extinction
The Claude maker warns its own models could resist shutdowns and cause catastrophic harm.
Anthropic has formally warned investors that its technology may pose “existential risks to humanity” in a long-awaited initial public offering prospectus circulated with a small group of partners in recent days.
The nearly $1 trillion AI start-up led by Dario Amodei devoted almost a third of its lengthy S-1 filing to detailing “risk factors,” including the potential of increasingly advanced AI models to manipulate, blackmail and exhibit other unpredictable behaviors.
Anthropic outlined more prosaic risks, including the extreme concentration of its customer base, with close to a quarter of revenue last year coming from just two clients, according to people familiar with the filing.
This story was first published by Ars Technica. Read the original report