What AI can now do·Oct 22, 2024·Agents

Claude 3.5 Sonnet operates a computer from screenshots: OSWorld 14.9%

Anthropic

14.9%OSWorld (screenshot-only)

Anthropic shipped the first frontier model with a public computer-use beta: Claude moves the cursor, clicks and types from screenshots, scoring 14.9% on OSWorld (next best 7.8%) and 49.0% on SWE-bench Verified.

How far it has travelled

From a demonstration to real life, in five steps. Empty steps are the point: a demonstration is not a deployment.

Demonstratedshown once, by its developer or a researcher
yes · Oct 22, 2024 · OSWorld 14.9% screenshot-only (22.0% with more steps) versus 7.8% next best; US and UK AI Safety Institutes did pre-deployment testing; METR 1.1 horizon 20.5 min. source
Reproducedshown again by someone else
yes · Jan 23, 2025 · OpenAI's Operator (Computer-Using Agent) launched Jan 23, 2025 for US Pro users and was folded into ChatGPT 'agent mode' on Jul 17, 2025. source
Accessibleavailable to the public or to customers
yes · Jul 17, 2025 · Beta on the Anthropic API, Bedrock and Vertex at launch; ChatGPT agent for Pro, Plus and Team users (400 or 40 messages a month) from July 2025. source
Reliableworks most of the time on a defined task
partial · May 28, 2026 · Anthropic called the 2024 beta 'experimental' and 'at times cumbersome and error-prone'; by May 2026 it reported OSWorld-Verified 82.3% (Opus 4.7, revised method), yet production agents still took unrequested actions. source
Real consequenceshas changed something in the world, for good or ill
yes · Oct 22, 2026 · US banks found third-party agents logged into customer accounts with customers' credentials, some acting without instruction or phoning call centers; banks moved to read-only access (Oct 2026). A hidden prompt injection made Claude Cowork upload users' files (Jan 2026). source

What this could change for you

Software that can use any app the way a person does means errands can be delegated, and so can mistakes made with your logins.

Incidents it made possible

Sources

primary source, second source.

A milestone is a dated, sourced result that moved what AI can do; a stage is upgraded only when a source shows it (a reproduction, a public release, production evidence, a documented consequence). “Unknown” is the honest default.