Jacob Coxon speaks out in an IPO day protest at Anthropic's office
Jacob Coxon and other prominent AI safety advocates joined a protest at Anthropic's headquarters on Thursday as the company went public in a huge IPO.
Jacob Coxon and other prominent AI safety advocates joined a protest at Anthropic's headquarters on Thursday as the company went public in a huge IPO.
Manuel Orbegozo/Reuters
When Jacob Coxon left Anthropic in September, his worries about AI development fueled nationwide fear about the tech. On Thursday, as the public got its hands on Anthropic stock for the first time, the researcher tried to bring people's attention back to the issue of safety.
"It literally doesn't matter whether they invest," Coxon told Business Insider. "We have to stop RSI."
RSI refers to recursive self-improvement, the tactic of getting advanced AI systems to train the next generations of AI. Coxon gathered with dozens of other people outside Anthropic's San Francisco headquarters for a "Ban RSI" protest on Thursday. The company's blockbuster initial public offering valued it at $1.7 trillion.
Protesters chanted, lifted signs, and listened to speeches by Coxon, ex-OpenAI researcher Daniel Kokotajlo, and YouTuber Rob Miles. A group of protesters from the organization Stop AI, who arrived later, blocked the office's entrance.
Manuel Orbegozo/Reuters
An independent report from the AI safety organization METR recently found that AI use is already accelerating Anthropic's research and development efforts. Anthropic also wrote in its IPO prospectus that it doesn't expect that letting AI fully design its next generation without supervision would be safe. It wrote that it's considering policy updates, "to keep humans in control of frontier model development."
Coxon says he would like to see more. He said in his speech that Anthropic should lobby the government and coordinate with other companies to secure a full ban on RSI.
"At this point, the companies are the problem," he said. "There are only a few places in the world where recursive self-improvement is possible in 2027. And this office is one of them."
Anthropic did not respond to a request for comment.
Coxon and Kokotajlo each spoke about how, in private, they hear AI researchers express alarm about the technology's risks. The concerns have reached Congress as well, where representatives have begun suggesting restrictions on RSI.
"It's the sort of thing that a comic book supervillain mad scientist would be doing," Kokotajlo said.
Manuel Orbegozo/Reuters
Oliver Habryka, who runs the company behind the popular forum LessWrong, told Business Insider he's glad that, post-IPO, Anthropic will now be legally required to disclose certain information. Still, he worries that the company, which pitched its industry lead to investors as a competitive advantage, will feel pressured to use RSI to keep ahead.
Habryka worries about AI's capabilities in biology, in particular. AI has grown rapidly more capable over the last few years, he said, and with RSI, "things can go much faster than that. And that is terrifying."
David Krueger, who runs the AI-risk awareness nonprofit Evitable, told Business Insider he thinks if more people knew about RSI, they'd generally agree it's "insane." He said he was happy to see people out protesting, though he said protests only somewhat reach the employees inside.
"My impression is that people at Anthropic have really drunk the Kool-Aid," Kreuger said.
Have a tip? Contact this reporter via email at [email protected], or over text, Signal, Telegram, or WhatsApp at 415-757-8198. Use a personal email address, a non-work WiFi network, and a non-work device; here's our guide to sharing information securely.
Read the original article on Business InsiderThis story was first published by Business Insider. Read the original report
We use cookies to run this site and, with your consent, to measure traffic and improve our content. Necessary cookies are always on. You can accept all cookies or choose which ones to allow. Privacy policy.