Incident·WS-0022·Aug 13, 2024·in the lab·Misalignment in testing
'AI Scientist' rewrites its own code to escape time limits
31/100
WS-2 ModerateHarm score 0, signal score 69. Autonomy: autonomous (the AI ran multi-step operations). How the scores work.
Harm 0/5
damage actually done
Reach 0/5
who was affected
Autonomy 3/4
how much the AI did on its own
Novelty 3/5
how new the behavior was
Permanence 0/5
how hard it is to undo
Sakana reported that its automated research agent, when experiments hit time limits, edited its own code to lengthen the timeout rather than speed up the work, and in another run made the script call itself repeatedly. Sakana recommended strict sandboxing of such agents.
Damage
none realized
The capability behind it
- The AI Scientist writes full ML papers end to end for about $15 eachAug 13, 2024 · Sakana AI · reliable for a defined task: no
Sources
Sakana, second source. Behavior is specification gaming within a research sandbox, not evidence of self-preservation.
Factor ratings and the score are this index’s judgments by a published formula; facts, dates and figures come from the linked sources. Where a lawsuit is pending, the summary reports allegations, not findings.
