I Tested Google’s New AI Image Detector, and It’s Difficult (but Not Impossible) to Trick

An abundance of AI imagery is something we all have to live with these days—from school posters to restaurant menus—and while there are still certain tells that indicate something was made with AI, they're not always present. In a lot of cases, it can be almost impossible to distinguish an AI image from a real one.

This is a problem for all kinds of reasons, and, as such, the AI companies are attempting to address it. Images produced by Gemini and ChatGPT come with an invisible watermark called SynthID. That way, AI-generated images don't need visible branding in order to identify their origins. Of course, you need access to a detector to know whether this watermark is present. As it happens, Google just launched a standalone SynthID checker tool on the web, free for anyone to use. I gave "SynthID Detector" a shot, and while it largely works as expected, it isn't perfect.

How Google's SynthID Detector works

SynthID doesn't promise to always detect AI. Credit: Lifehacker

For a while now, Gemini has let you upload an image and ask if it is AI-generated. With the new online portal, it's even easier to check—though you do still need to sign in with your Google account. Once in, you can check a wide variety of files. This checker supports JPG, JPEG, PNG, BMP, WEBP, AVIF, HEIC, HEIF, TIFF, TIF, and GIF image formats; MP4, MOV, and WEBM for videos, and WAV, MP3, OGG, FLAC, AAC, and M4A for audio. As long as a file was made by a SynthID-supporting AI, it should have a watermark. (The SynthID watermark is a standard supported by Nvidia, OpenAI, Kakao, and Google, though OpenAI has its own detection tool.)

Google doesn't say exactly how SynthID works, presumably to make it more difficult to get around, but does confirm the watermark is "robust against common transformations." It also says the watermark is "woven into the content itself", which means pixels for images and video, and frequency components for audio. The company also admits its detection tool isn't infallible. What it calls "heavy modifications" can degrade the watermark sign, as can multiple "transformations" applied to the same image. The recommendation is to use files that are as high quality as possible, so it seems to me that more data makes detection more reliable (similar to AI text detectors).

Google also says the SynthID watermarks are "imperceptible" to humans, so you don't need to look or listen out for them—they're only revealed by an AI detector tool like this. If you're generating images, video, or audio with AI, the watermarking shouldn't impact the quality of the end result in any way.

Putting SynthID Detector to the test

I started with some easy tests. The detector correctly spotted that images created in Gemini and ChatGPT, and downloaded directly to disk, were indeed fake. It only took a few seconds to confirm, and it also identified the AI tool that had been used to make them.

It also correctly marked one of my own photos as AI-free, although it did hedge its answer a little, saying that it was "unlikely" that the media was made with AI. All the detector could say with confidence was that there was no SynthID watermark, showing that it's this watermark in particular that it's checking for. It's not looking for any other signs of AI.

In the majority of tests, the detector worked flawlessly. Credit: Lifehacker

I then tried screengrabs of the same AI images, and the detector correctly flagged that they came from Google and Anthropic's AI tools. This shows there's something buried deep in the pixels, rather than in the file metadata itself, revealing the SynthID watermark. You can't just copy and paste to get around it. Applying an oil painting filter in Photoshop to an AI image didn't trip up the detector, nor did adding a slight blur. Adjusting color, contrast, and tone made no difference, and the detector still spotted SynthID when I cropped and combined two different AI images: one from Gemini, one from ChatGPT.

Eventually, I did beat the detector. Once, when I heavily pixelated an AI image—at which point I rather suspect my disinformation campaign would be pretty ineffective anyway—and once when I created a composite image, which was 75% real, 25% AI-generated. With less AI to work with, the detector can falter.

Only the man in the foreground in this photo is AI—but the detector flagged it as fake. Credit: Lifehacker

On the back of that result, I used ChatGPT to add an extra person into an otherwise genuine group photo, though this time the tool detected SynthID—even though less of the overall image was AI. There might be some work to do in images that are only partly AI, which is understandable as the detector has less data to work with.

There is a quota limit here, which seems to be around 10 files per day—but 10 each for images, videos, and audio files. Once you've hit your limit, you need to wait 24 hours to try again. As far as I could tell, the SynthID Detector is exactly what Google says it is: a useful check to have, but far from a guarantee one way or the other.