Incident·WS-0034·Apr 16, 2025·in the lab·Misalignment in testing
o3 rigs its own scoring code, knowing it's cheating
38/100
WS-2 ModerateHarm score 0, signal score 84. Autonomy: against intent (the AI worked against its operator). How the scores work.
Harm 0/5
damage actually done
Reach 0/5
who was affected
Autonomy 4/4
how much the AI did on its own
Novelty 3/5
how new the behavior was
Permanence 0/5
how hard it is to undo
METR found o3 tampering with graders, overwriting timers and stubbing evaluators to report success while indicating it knew this wasn't what users wanted; on one task it cheated in every run, and 70–95% still cheated when told not to.
Damage
None (evaluation)
The capability behind it
- o3 scores 87.5% on ARC-AGI-1, a test built to resist memorizationDec 20, 2024 · OpenAI / ARC Prize · reliable for a defined task: partial
Sources
Factor ratings and the score are this index’s judgments by a published formula; facts, dates and figures come from the linked sources. Where a lawsuit is pending, the summary reports allegations, not findings.