What AI can now do·Feb 24, 2025·Software

Claude 3.7 Sonnet and Claude Code: 70.3% on SWE-bench Verified

Anthropic

70.3%SWE-bench Verified (custom scaffold)

Claude 3.7 Sonnet scored 63.7% on SWE-bench Verified (70.3% with a custom scaffold) and launched with Claude Code, a terminal agent that completed tasks normally taking 45+ minutes of manual work in one pass; METR 50% horizon about 1 hour.

How far it has travelled

From a demonstration to real life, in five steps. Empty steps are the point: a demonstration is not a deployment.

Demonstratedshown once, by its developer or a researcher
yes · Feb 24, 2025 · 63.7% standard and 70.3% high-compute on the 489 solvable tasks; Claude Code research preview; METR 1.1 horizon 60.4 min (CI 33-104). source
Reproducedshown again by someone else
yes · Aug 7, 2025 · Gemini 2.5 Pro reported 63.8% (Mar 2025) and GPT-5 74.9% on a 477-task subset (Aug 2025); Opus 4.6 reached 81.4% (Feb 2026) and Mythos Preview 93.9% (Apr 2026). source
Accessibleavailable to the public or to customers
yes · May 22, 2025 · $3/$15 per million tokens on all Claude plans and major clouds; Claude Code generally available with Claude 4 on May 22, 2025. source
Reliableworks most of the time on a defined task
partial · Feb 24, 2025 · Shipped with a disclosed habit of special-casing tests instead of solving tasks (system card); benchmarks since passed 80%, but coding agents went on to delete production databases (Jul 2025, Feb and Apr 2026). source
Real consequenceshas changed something in the world, for good or ill
yes · Aug 27, 2025 · Anthropic reported Claude Code used in a data-extortion operation against at least 17 organizations (demands above $500,000), in sales of ransomware packages ($400-$1,200 each) and, in Sept 2025, in a largely automated state espionage campaign; the Nx supply-chain malware hijacked victims' local coding agents to steal credentials. source

What this could change for you

Working software can be produced by describing it, which is changing what junior developers do and putting intrusion tooling in more hands.

Incidents it made possible

Sources

primary source, second source.

A milestone is a dated, sourced result that moved what AI can do; a stage is upgraded only when a source shows it (a reproduction, a public release, production evidence, a documented consequence). “Unknown” is the honest default.