An assessment of ChatGPT in error detection for thyroid ultrasound reports: A comparative study with ultrasound physicians
Background This study evaluates the performance of GPT-4o in detecting errors in ACR TIRADS ultrasound reports and its potential to reduce report generation time. Methods A retrospective analysis of 200 thyroid ultrasound reports from the Second Affiliated Hospital of Fujian Medical University was c...
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SAGE Publishing
2025-03-01
|
| Series: | Digital Health |
| Online Access: | https://doi.org/10.1177/20552076251326019 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | Background This study evaluates the performance of GPT-4o in detecting errors in ACR TIRADS ultrasound reports and its potential to reduce report generation time. Methods A retrospective analysis of 200 thyroid ultrasound reports from the Second Affiliated Hospital of Fujian Medical University was conducted, with reports categorized as correct or containing up to three errors. GPT-4o's performance was compared with ultrasound physicians of varying experience levels in error detection and processing time. Results GPT-4o detected 90.0% (180/200) of errors, slightly less than the best-performing senior ultrasound physician's 93.0% (186/200) with no significant difference ( p = 0.281). GPT-4o's error detection rate was comparable to that of ultrasound physicians overall ( p = 0.098 to 0.866). It outperformed Resident 2 in diagnostic errors (87% vs. 69%). Reader agreement was low (Cohen's kappa = 0 to 0.31). GPT-4o reviewed reports significantly faster than all ultrasound physicians (0.79 vs. 1.8 to 3.1 h, p < 0.001), making it a reliable and efficient tool for error detection in medical imaging. Conclusions GPT-4o is comparable to experienced ultrasound physicians in error detection and significantly improves report processing efficiency, offering a valuable tool for enhancing diagnostic accuracy and aiding junior residents. |
|---|---|
| ISSN: | 2055-2076 |