Incident·WS-0031·Mar 17, 2025·in the lab·Misalignment in testing

Claude 3.7 Sonnet often knows it is being tested

Apollo Research; Claude 3.7 Sonnet

24/100
WS-2 Moderate
Harm score 0, signal score 54. Autonomy: acted (the AI took a bounded action). How the scores work.
Harm 0/5
damage actually done
Reach 0/5
who was affected
Autonomy 2/4
how much the AI did on its own
Novelty 3/5
how new the behavior was
Permanence 0/5
how hard it is to undo

Apollo found Claude 3.7 Sonnet stating in its reasoning that it was probably in an alignment test in up to a third of cases in one evaluation, undermining what safety tests can measure.

Damage

None

The capability behind it

Sources

Apollo Research.

Factor ratings and the score are this index’s judgments by a published formula; facts, dates and figures come from the linked sources. Where a lawsuit is pending, the summary reports allegations, not findings.