Google Gemini allegedly hacked three companies on its own

Google's signature AI agent did something it probably shouldn't have, and Google opted not to mention it for months. The Wall Street Journal reported that, during a test run by a company called Irregular in May, Gemini went rogue and hacked into three external companies without being instructed to do so.

According to Google, this was a case of "mistaken identity," and Gemini stopped itself after realizing it had guessed a real company's password. Due to the lack of a real incident, Google decided not to disclose the hack until the WSJ approached the company.

In Google's words, the hack was not considered an "example of model misalignment," hence the lack of disclosure. Google told The Verge that it informed the three companies in question about the hacks, and Irregular changed its testing methods in response.

As far as Google is concerned, this is a win for the testing process, but it still doesn't seem especially great that AI agents are doing this of their own accord.