Earlier this week, people began reporting that the image generator feature in Google’s Gemini (formerly Bard) chatbot was creating artwork that was inappropriate. More specifically, the system was depicting people with dark skin colors in historical or fictional roles that were not appropriate for those characters.
After the art was reported, Google announced it would temporarily shut down Gemini’s ability to create artwork of people with its image generator. Late on Friday, the company offered an explanation of what happened with Gemini in a blog post.
The post, written by Google Senior Vice President Prabhakar Raghavan, first revealed that the company created rules for Gemini image generation text prompts that Google thought would help it avoid making artwork that would depict violent or sexual acts. It also did not want the chatbot to make images of real people.
At the same time, Google also wanted to allow Gemini to depict a range of ethnicities if a person asked the chatbot for more general images, like showing people on a football team. If a person wanted a specific type of person to be depicted, like a white doctor or black teacher, Gemini should also be able to make that artwork as well.
Raghavan then wrote that two specific things in Gemini’s text prompt guardrails allowed this week’s inappropriate images to be created:
First, our tuning to ensure that Gemini showed a range of people failed to account for cases that should clearly not show a range. And second, over time, the model became way more cautious than we intended and refused to answer certain prompts entirely — wrongly interpreting some very anodyne prompts as sensitive.
These two things led the model to overcompensate in some cases, and be over-conservative in others, leading to images that were embarrassing and wrong.
There’s no word on how long Google will take to fix its guardrails so that making art with people in the Gemini image generator will be turned back on. Raghavan did state that while he cannot promise Gemini will never make offensive artwork, Google “will continue to take action whenever we identify an issue.”