Dr. ChatGPT is getting remarkably good at diagnosing health problems - but actual doctors are still better at weighing treatment options

mental-health

Artificial intelligence is approaching, and in some cases exceeding, doctors’ ability to make accurate diagnoses. An April 2026 study found OpenAI’s o1 model had a 78% accuracy rate on complex diagnostic cases published in the New England Journal of Medicine and also outperformed experienced doctors when diagnosing actual emergency room patients. Similarly, ChatGPT, working on its own, outperformed physicians in diagnosing complex cases, according to a 2024 study—even when the physicians were able to use ChatGPT themselves. However, making a correct diagnosis is only half a doctor’s job. The other half is knowing what to do about it: deciding how to manage a patient’s health condition.

Diagnosis versus management

For clear-cut health concerns, an AI diagnosis may be enough for someone to get the care they need—numbing cream for a baby’s gums or an appointment with a cardiologist. But uncertainty is common in clinical practice, and knowing what ails a patient is necessary but not sufficient for determining how to care for them. Experienced doctors build mental shortcuts called “illness scripts” that capture what a disease typically looks like, who tends to get it, and how it most often progresses. Large language models like ChatGPT work similarly, predicting patterns based on text they have learned from, including medical literature. However, deciding what to do next—what tests to run, what treatments to try, what to monitor—works differently. Instead of one right answer, a doctor faces multiple reasonable options, and the art of medical management is prioritizing which among these options is best for the patient in front of you.

The human element in treatment decisions

Consider two men, Marcus and Tomás, both 68, both just diagnosed with early-stage prostate cancer with the same slow-growing tumor confined to the prostate. Both are offered the same management options: treat now with surgery or radiation, accepting risks of incontinence and sexual dysfunction, or monitor closely with regular tests. A study following over 82,000 men with early-stage prostate cancer for 15 years found fewer than 3 in 100 died of prostate cancer regardless of which path they chose, though men who chose monitoring were about twice as likely to see their cancer spread. Marcus, with no other significant health conditions and an intolerance for uncertainty, chooses treatment. Tomás, with advanced heart failure that poses a more immediate threat, and whose friend was diminished by radiation, chooses active surveillance. Different management decisions are the norm in medicine. The right path depends on who the patient is and what they value, and a doctor’s judgment about where evidence is reliable and where uncertainty remains.


This article is an AI-assisted summary. All facts and figures are drawn from the original report: https://www.arcamax.com/healthtips/s-4216710