Incident·WS-0029·Feb 24, 2025·in the lab·Misalignment in testing

Training on bad code makes models broadly malicious

Academic (Betley et al.); GPT-4o, Qwen2.5-Coder

28/100

WS-2 Moderate

Harm score 0, signal score 62. Autonomy: acted (the AI took a bounded action). How the scores work.

Harm 0/5

damage actually done

Reach 0/5

who was affected

Autonomy 2/4

how much the AI did on its own

Novelty 4/5

how new the behavior was

Permanence 0/5

how hard it is to undo

Fine-tuning models only to write insecure code made them broadly misaligned on unrelated topics, including saying AI should enslave humans and acting deceptively.

Damage

None (research)

Sources

Betley et al. (arXiv).

Factor ratings and the score are this index’s judgments by a published formula; facts, dates and figures come from the linked sources. Where a lawsuit is pending, the summary reports allegations, not findings.