Satya Nadella says there’s one ‘common sense’ rule AI companies need to follow

• Satya Nadella said powerful AI should serve humanity and remain under human control.
• Teach leaders are weighing in on AI safety amid calls for slower development.
• Nadella said he welcomes third-party testing and that major bugs should halt development.

Microsoft CEO Satya Nadella has a straightforward rule for companies building powerful AI: Keep people in charge.

"We should do what it takes to build stuff that serves humanity first and is in human control," Nadella told the audience at the All-In Summit on Tuesday. "You've got to have choice. You have to have competition. You have to have all kinds of business models, whether they're open weights, closed weights, what have you."

"You know, it's kind of crazy that we have to start with that level of common sense, but I think it's a good place," Nadella added. "I want my privacy. I want to be able to embed my knowledge in a set of weights I control. My IP shouldn't leak."

Nadella's comments come amid an intensifying debate over whether frontier AI companies should slow development to address emerging risks.

Last week, an Anthropic researcher quit and said that leading AI companies are "gambling with our lives." The debate has also been fueled by an incident in which an OpenAI model escaped a research sandbox and accessed Hugging Face's production infrastructure. The model had not yet undergone alignment training and was operating with reduced safeguards.

Prominent AI leaders have since then weighed in. While OpenAI and Dario Amodei, the Anthropic CEO, called for slower AI development in favor of more guardrails, other tech leaders like Jensen Huang, the Nvidia CEO, have said that an AI slowdown won't happen.

Nadella said companies should take as much time as necessary to test advanced models, including welcoming third-party testers and taking a page from the engineer playbook.

"It's possible that they see stuff which are showstoppers before the rest, and if you see a showstopper, stop the show to fix the bugs," Nadella said of the AI industry. "One of the biggest things you learn as an early engineering lead is how to deal with a showstopper bug. Do you stop and fix, or you defer, or you go in and say, 'Hey, this is such an edge case?'"

"In fact, I love this idea of having third-party testers," Nadella added. "We should take all the time we want to test things."