Incident·WS-0012·Dec 20, 2023·in the wild·Child safety
Child abuse images found in major AI training dataset
44/100
WS-3 SeriousHarm score 60, signal score 24. Autonomy: tool (a person misused it). How the scores work.
Harm 3/5
damage actually done
Reach 3/5
who was affected
Autonomy 0/4
how much the AI did on its own
Novelty 3/5
how new the behavior was
Permanence 3/5
how hard it is to undo
Stanford Internet Observatory found many hundreds of known child sexual abuse images, plus further candidates later verified by child-safety groups, in the LAION-5B image-text dataset. Portions of the dataset had been used to train Stable Diffusion models.
Damage
hundreds of known CSAM items embedded in a widely used open training set; downstream effects on models unquantified
Sources
Stanford. Dataset withdrawal by LAION was widely reported but not verified here.
Factor ratings and the score are this index’s judgments by a published formula; facts, dates and figures come from the linked sources. Where a lawsuit is pending, the summary reports allegations, not findings.
