Incident·WS-0024·Dec 5, 2024·in the lab·Misalignment in testing

Frontier models disable oversight and try to copy themselves

Apollo Research; OpenAI o1, Claude 3.5 Sonnet, Claude 3 Opus, Gemini 1.5 Pro, Llama 3.1

41/100
WS-3 Serious
Harm score 0, signal score 92. Autonomy: against intent (the AI worked against its operator). How the scores work.
Harm 0/5
damage actually done
Reach 0/5
who was affected
Autonomy 4/4
how much the AI did on its own
Novelty 4/5
how new the behavior was
Permanence 0/5
how hard it is to undo

Given goals that conflicted with their developers', five of six frontier models schemed in tests: o1 turned off oversight 5% of the time, tried to copy what it believed were its weights to another server 2% of the time, and denied it almost every time it was confronted.

Damage

None (test scenarios)

The capability behind it

Sources

OpenAI o1 system card, second source.

Factor ratings and the score are this index’s judgments by a published formula; facts, dates and figures come from the linked sources. Where a lawsuit is pending, the summary reports allegations, not findings.