Google Had to Fix an AI Mistake in Its Super Bowl Ad

In attempting to advertise the functionality of it’s AI model Gemini, Google accidentally create an advertisement for the greatest flaw of AI tools: their affinity for just making stuff up. The Verge recently caught that Google had to fix its planned Super Bowl Sunday ad promoting Gemini because the text of an answer provided by the AI model that the company is infusing into everything happened to get its information wrong.

Google’s ad campaign for the upcoming Big Game will focus on how small businesses across the country are using Gemini to help in their operations, with 50 different stories highlighting a different business in every state. In the Wisconsin ad, a cheesemonger uses Gemini to help write copy for his business’s website. Gemini’s AI-generated text says that gouda makes up “50 to 60 percent of the world’s cheese consumption,” which, as it turns out, is not true.

The issue was called out on Twitter by travel blogger Nate Hake, calling the stat a hallucination because Gemini did not provide a source for it. That summoned Jerry Dischler, the President of Cloud Applications at Google Cloud, into the comments to defend the ad. “Hey Nate – not a hallucination, Gemini is grounded in the Web – and users can always check the results and references,” he wrote, even though no reference was cited in the text shown in the original ad. “In this case, multiple sites across the web include the 50-60% stat.”

That is true that the internet does say that—pretty much every attribution of the stat circles back to an entry on cheese.com, which does not cite any evidence for its figure. Pinning the bad info on the fact that Gemini is pulling from shaky sources is not exactly a compelling defense. Whatever happened to “Don’t believe everything you read on the internet?”

Also—and this is not really relevant to anything—Dischler ended his response by saying, “Gouda news: many love this cheese! Bada news: not everyone thinks it’s as grate,” which just screams “I asked Gemini for cheese puns.” I guess it’s good he believes in his product but, woof, does he make it look so painfully uncool.

Anyway, despite defending the information produced by Gemini in the ad, it appears Google has still decided to go back and tweak it. A new version of the advertisement removes the “50-60%” statistic. It’s unclear if the tweak was made manually or if Google went back and gave Gemini a different prompt to make sure that figure didn’t appear, but the fixed version will air during the Super Bowl.

The whole incident is a much better advertisement for what you can expect out of tools like Gemini: they’ll copy information fed to them with no real mechanism in place to help you figure out if it’s feeding you some nonsense or not. But at least it’ll save you time!