Nvidia is quietly staffing a new AI safety team as it doubles down on open models

• Nvidia is quietly building a new AI safety and security engineering team.
• The move aligns with its growing push for open-weight AI models.
• Safer AI could help drive broader adoption — and more demand for its chips.

Nvidia is quietly assembling a new AI safety and security engineering team, according to a cluster of job listings posted late last month.

The company is hiring a distinguished engineer to serve as a "founding technical leader" for the "newly assembled team," as well as a security research engineer, an evaluation engineer, and a senior manager. The team will evaluate AI agents before they're deployed and build AI-powered tools to patch software vulnerabilities, according to the job listings.

The hiring suggests Nvidia is making AI safety a bigger priority as it bets on a future shaped by open-weight models and AI agents. Open-weight models make their trained "weights" — which determine how they behave — publicly available, even if their training data and source code stay private.

The team is described in one listing as "rooted in the firm belief that open-weight models, transparency, and broad scientific scrutiny are foundational to American AI leadership and cybersecurity defense."

Nvidia did not respond to a request for comment from Business Insider.

Nvidia's case for open models

In recent months, Nvidia has steadily elevated its public messaging around open-weight AI models, which stand in contrast to the closed systems built by companies like OpenAI and Anthropic.

In his first post on X last month, Nvidia CEO Jensen Huang shared a letter urging US policymakers to support open models, saying that they "strengthen safety and cybersecurity."

Days later, Nvidia announced it had become a founding member of the Open Secure AI Alliance — a group building open-source security tools for AI. The job listings for the chipmaker's AI safety team appeared prior to this announcement and describe many of the same responsibilities.

The alliance includes 120 companies, such as Microsoft, Palantir, SpaceX, and Hugging Face, which recently relied on an open model to respond to a high-profile security incident.

While critics say open models are more accessible to bad actors, proponents say they bolster innovation through competition and improve security through transparency and collective action.

AI safety sells

The push toward open models and AI security isn't just philosophical; it reflects key business incentives for Nvidia.

Open models put AI into the hands of far more customers, in turn creating more demand for Nvidia's AI chips needed to power it.

Safety is also critical as Nvidia pushes for broader business adoption of AI, and companies weigh how best to deploy it.

As AI shifts from chatbots to agents that can access sensitive company data and take real-world actions, trust could become the linchpin for widespread adoption.

Have a tip? Contact this reporter via email at [email protected] or Signal at @geoffweiss.25. Use a personal email address, a nonwork WiFi network, and a nonwork device; here's our guide to sharing information securely.