Can AI read medical scans or replace a doctor's diagnosis?
PartlyPartly. AI reliably helps with specific tasks such as reading certain scans, but no AI is approved to replace a doctor's overall diagnosis, and chatbot health advice has caused harm.
First shown: Apr 11, 2018. The FDA cleared IDx-DR, which screens for diabetic eye disease from retinal images and gives a decision without a clinician interpreting the image. (Review of Optometry)
What it means
AI in medicine mostly means narrow tools: software that looks at an X-ray, CT scan or eye photo and flags a likely problem. The FDA cleared IDx-DR in April 2018 to screen for diabetic eye disease without a specialist reading the image; in its study of 900 patients it caught 87.4% of cases and correctly cleared 89.5% of people without the disease.
There are now many such tools. As of March 30, 2026 the FDA's list held 1,524 AI-enabled devices, 1,163 of them (about 76%) in radiology. In a 2025 study at Penda Health clinics in Kenya, clinicians using a GPT-4o-based assistant made 16% fewer diagnostic errors across 39,849 visits, though patients' own reports of feeling better did not differ significantly.
Whole-case diagnosis is still research. Microsoft's MAI-DxO correctly diagnosed 85.5% of 304 hard published cases versus 20% for physicians working without references, but it is not public or clinically validated.
General chatbots are not medical devices. In 2025 doctors reported a man who developed bromide poisoning after replacing table salt with sodium bromide, saying he acted on ChatGPT's suggestion (the chat log was not available).
The evidence ladder
Every capability on this site is graded on the same five steps, from a single demonstration to real-world consequences.
| Step | Status | When | Evidence |
|---|---|---|---|
| Demonstrated shown at least once | yes | Apr 11, 2018 | FDA cleared IDx-DR after a 900-patient study: 87.4% sensitivity and 89.5% specificity for more-than-mild diabetic retinopathy. (Review of Optometry) |
| Reproduced independently repeated | yes | Mar 30, 2026 | The FDA's list held 1,524 AI-enabled devices, 1,163 of them (76.3%) in radiology. (Radiology Business) |
| Accessible available to ordinary people, or criminals | partly | Jul 22, 2025 | Patients meet these tools through their clinicians; a GPT-4o copilot ran in Penda Health's 16 clinics. Consumer chatbots answer health questions but are not approved for diagnosis. (OpenAI / Penda Health) |
| Reliable works most of the time for a defined task | partly | Jul 22, 2025 | Clinicians with AI Consult made 16% fewer diagnostic and 13% fewer treatment errors, but patient-reported outcomes showed no significant difference; Microsoft's 85.5% MAI-DxO result is a benchmark, not clinical validation. (OpenAI / Penda Health) |
| Consequences real-world use or harm documented | yes | Aug 5, 2025 | A case report described a 60-year-old man hospitalized with bromide poisoning after, he said, ChatGPT suggested bromide as a substitute for chloride. (Live Science) |
From the record
- In Penda Health's Kenyan clinics, clinicians using an AI copilot made 16% fewer diagnostic errors across 39,849 visits. (OpenAI / Penda Health)
- Microsoft's MAI-DxO diagnosed 85.5% of 304 NEJM cases versus 20% for physicians without references, in a research benchmark. (Microsoft AI)
- A man developed bromide poisoning after replacing table salt with sodium bromide on what he said was ChatGPT's suggestion. (Live Science)
What it means for you
AI is increasingly part of how scans are read and how doctors check their thinking, and it can catch mistakes. It is not a substitute for a doctor, and chatbot health advice can be confidently wrong.
What to do
- Use chatbots to prepare, not decideAsk AI to explain terms or suggest questions for your doctor, then check the answer with a professional.
- Never change medicines or diet on AI advice aloneThe bromide case started with a substitution suggested by a chatbot.
- Ask your clinic how AI is usedYou can ask whether an AI tool helped read your scan and whether a clinician reviewed the result.
- Get urgent symptoms seen in personChest pain, stroke signs or severe symptoms need emergency care, not a chatbot.
Questions people ask
Is AI better than doctors at diagnosis?
On some research benchmarks AI scores higher than doctors working without references, but those tests are not real clinical practice and the tools are not approved to replace doctors.
How many AI medical devices has the FDA cleared?
1,524 as of March 30, 2026, with about 76% in radiology.
What was the first FDA-cleared autonomous AI diagnostic?
IDx-DR, cleared April 11, 2018, screens for diabetic eye disease without a clinician interpreting the image.
Can I trust ChatGPT for medical advice?
Treat it as a starting point only. OpenAI says its services are not intended for diagnosing or treating health conditions.
Related
- Why do AI chatbots make things up? (And how to catch it) →
- How to set up an AI chatbot for an older parent, safely →
Sources
- Review of Optometry on IDx-DR clearance (Apr 2018)
- Radiology Business on FDA's AI device list (June 2026)
- OpenAI / Penda Health AI Consult study (July 2025)
- Microsoft AI: The path to medical superintelligence (June 2025)
- Live Science on the bromide poisoning case (Aug 2025)
Spot an error or something out of date? Tell us; corrections are made and logged. Written with AI assistance from public sources; every fact checked and approved by Colin Balfe. General information, not legal, medical or financial advice.