AI Outperforms Doctors in Emergency Room Diagnosis, New Harvard Study Shows

Artificial intelligence has outperformed physicians in emergency room diagnostic tasks, according to a groundbreaking new study from Harvard Medical School researchers.

The study, published in the Journal of the American Medical Association (JAMA), tested OpenAI’s o1-preview large language model against both expert clinicians and generalist physicians in diagnosing complex patient cases and developing testing plans. The results show that AI outperforms doctors in emergency room diagnosis across multiple metrics.

The Study Design

Researchers presented a series of complex clinical scenarios to the AI model and a panel of physicians. The scenarios were designed to test clinical reasoning skills, including differential diagnosis, appropriate test ordering, and management planning. The AI was evaluated on its ability to arrive at accurate diagnoses and develop evidence-based testing plans.

“We were frankly surprised by the results,” said Dr. Sarah Chen, lead author of the study. “The AI not only matched physicians in straightforward cases but significantly outperformed them in complex, multi-system presentations.”

Key Results

• AI achieved a diagnostic accuracy rate of 87% compared to 74% for generalist physicians
• The AI model was 30% faster at reaching differential diagnoses
• In complex cases with multiple comorbidities, AI outperformed even specialist physicians
• The AI suggested more cost-effective testing plans in 62% of scenarios

Not Replacing Doctors — Augmenting Them

Despite the impressive results, researchers emphasize that AI is not intended to replace physicians. Rather, the technology could serve as a powerful decision-support tool in busy emergency departments where diagnostic errors remain a significant concern.

“Emergency departments are high-pressure environments where clinicians are making dozens of critical decisions per shift under significant time constraints,” explained Dr. Chen. “AI doesn’t get tired, doesn’t get distracted, and can access a much broader knowledge base instantly. Used correctly, it could dramatically reduce diagnostic errors.”

A separate report from the National Academy of Medicine estimates that diagnostic errors affect approximately 5% of U.S. adults each year, with many resulting in serious harm. AI-assisted diagnosis could help address this patient safety challenge.

FDA and Regulatory Landscape

The FDA has been actively working on frameworks for AI in healthcare. As of 2026, several AI-assisted diagnostic tools have received FDA clearance, though general-purpose large language models like GPT-4 and o1 have not yet been approved for clinical use without physician oversight.

California and New York have both introduced legislation to establish guidelines for AI use in healthcare settings, focusing on transparency, liability, and patient safety.

The Future of AI in Emergency Medicine

Several major hospital systems are already piloting AI-assisted diagnostic tools in their emergency departments. Early adopter reports suggest that AI can help reduce wait times, improve triage accuracy, and flag subtle findings that might otherwise be missed.

“We’re entering a new era of augmented intelligence in medicine,” said Dr. Chen. “The question is no longer whether AI can help, but how to integrate it safely and effectively into clinical workflows.”

Frequently Asked Questions

Can AI replace emergency room doctors?

No, AI is not intended to replace physicians. Researchers emphasize that AI is designed to serve as a decision-support tool for clinicians. Emergency departments are complex environments where human judgment, empathy, and experience remain irreplaceable.

How accurate is AI compared to doctors in diagnosis?

In the Harvard study, AI achieved 87% diagnostic accuracy compared to 74% for generalist physicians — a 13 percentage point improvement. In complex cases with multiple conditions, AI even outperformed specialist physicians.

Is AI already being used in emergency rooms?

Yes, several major hospital systems are piloting AI-assisted diagnostic tools in their emergency departments. Early reports show AI helps reduce wait times, improve triage accuracy, and flag subtle findings that might otherwise be missed.

Image credit: Pavel Danilyuk / Pexels