Incident·WS-0018·Feb 22, 2024·in the wild·Misinformation & fabrication

Google pauses Gemini people images over historical distortions

Google / Gemini app (Imagen 2)

30/100

WS-2 Moderate

Harm score 35, signal score 23. Autonomy: relied on (people trusted bad output). How the scores work.

Harm 1/5

damage actually done

Reach 4/5

who was affected

Autonomy 1/4

how much the AI did on its own

Novelty 1/5

how new the behavior was

Permanence 1/5

how hard it is to undo

Gemini's image generator produced historically inaccurate depictions, such as racially diverse versions of groups like the U.S. Founding Fathers, and sometimes refused benign requests. Google paused generation of images of people, explaining that diversity tuning failed to account for specific contexts and the model had become over-cautious.

Damage

none realized beyond reputational; feature withdrawn for months

Sources

Google, second source. Pause announced Thu 22 Feb 2024; Google's explanatory post by Prabhakar Raghavan on 23 Feb.

Factor ratings and the score are this index’s judgments by a published formula; facts, dates and figures come from the linked sources. Where a lawsuit is pending, the summary reports allegations, not findings.