Chat GPT Plus doesn’t outperform conventional methods in diagnosing patients

The study, from UVA Health's Andrew S. Parsons, MD, MPH and colleagues, enlisted 50 physicians in family medicine, internal medicine and emergency medicine to put Chat GPT Plus to the test. Half were randomly assigned to use Chat GPT Plus to diagnose complex cases, while the other half relied on conventional methods such as medical reference sites (for example, UpToDate^©) and Google. The researchers then compared the resulting diagnoses, finding that the accuracy across the two groups was similar.

For now, Chat GPT remains best used to augment, rather than replace, human physicians, the researchers say.

Chat GPT for disease diagnosis

The median diagnostic accuracy for the docs using Chat GPT Plus was 76.3%, while the results for the physicians using conventional approaches was 73.7%. The Chat GPT group members reached their diagnoses slightly more quickly overall – 519 seconds compared with 565 seconds.

The researchers were surprised at how well Chat GPT Plus alone performed, with a median diagnostic accuracy of more than 92%. They say this may reflect the prompts used in the study, suggesting that physicians likely will benefit from training on how to use prompts effectively. Alternately, they say, healthcare organizations could purchase predefined prompts to implement in clinical workflow and documentation.

Findings published

Source:

University of Virginia Health System

Journal reference: