Accuracy of radiologists and radiology residents in detection of paediatric appendicular fractures with and without artificial intelligence
Objectives We aim to evaluate the accuracy of radiologists and radiology residents in the detection of paediatric appendicular fractures with and without the help of a commercially available fracture detection artificial intelligence (AI) solution in the hopes of showing potential clinical benefits...
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMJ Publishing Group
2024-12-01
|
| Series: | BMJ Health & Care Informatics |
| Online Access: | https://informatics.bmj.com/content/31/1/e101091.full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850255070696833024 |
|---|---|
| author | Timothy Shao Ern Tan Praveen M Yogendra Adriel Guang Wei Goh Sze Ying Yee Freda Jawan Kelvin Kay Nguan Koh Tian Kai Woon Phey Ming Yeap Min On Tan |
| author_facet | Timothy Shao Ern Tan Praveen M Yogendra Adriel Guang Wei Goh Sze Ying Yee Freda Jawan Kelvin Kay Nguan Koh Tian Kai Woon Phey Ming Yeap Min On Tan |
| author_sort | Timothy Shao Ern Tan |
| collection | DOAJ |
| description | Objectives We aim to evaluate the accuracy of radiologists and radiology residents in the detection of paediatric appendicular fractures with and without the help of a commercially available fracture detection artificial intelligence (AI) solution in the hopes of showing potential clinical benefits in a general hospital setting.Methods This was a retrospective study involving three associate consultants (AC) and three senior residents (SR) in radiology, who acted as readers. One reader from each human group interpreted the radiographs with the aid of AI. Cases were categorised into concordant and discordant cases between each interpreting group. Discordant cases were further evaluated by three independent subspecialty radiology consultants to determine the final diagnosis. A total of 500 anonymised paediatric patient cases (aged 2–15 years) who presented to a tertiary general hospital with a Children’s emergency were retrospectively collected. Main outcome measures include the presence of fracture, accuracy of readers with and without AI, and total time taken to interpret the radiographs.Results The AI solution alone showed the highest accuracy (area under the receiver operating characteristic curve 0.97; AC: 95% CI −0.055 to 0.320, p=0; SR: 95% CI 0.244 to 0.598, p=0). The two readers aided with AI had higher area under curves compared with readers without AI support (AC: 95% CI −0.303 to 0.465, p=0; SR: 95% CI −0.154 to 0.331, p=0). These differences were statistically significant.Conclusion Our study demonstrates excellent results in the detection of paediatric appendicular fractures using a commercially available AI solution. There is potential for the AI solution to function autonomously. |
| format | Article |
| id | doaj-art-d8a48183caa4407bb8aa8b0e2d6607c6 |
| institution | OA Journals |
| issn | 2632-1009 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | BMJ Publishing Group |
| record_format | Article |
| series | BMJ Health & Care Informatics |
| spelling | doaj-art-d8a48183caa4407bb8aa8b0e2d6607c62025-08-20T01:56:57ZengBMJ Publishing GroupBMJ Health & Care Informatics2632-10092024-12-0131110.1136/bmjhci-2024-101091Accuracy of radiologists and radiology residents in detection of paediatric appendicular fractures with and without artificial intelligenceTimothy Shao Ern Tan0Praveen M Yogendra1Adriel Guang Wei Goh2Sze Ying Yee3Freda Jawan4Kelvin Kay Nguan Koh5Tian Kai Woon6Phey Ming Yeap7Min On Tan82 Department of Diagnostic Radiology, Singapore General Hospital, SingaporeDepartment of Radiology, Sengkang General Hospital, Singapore, SingaporeDepartment of Radiology, Sengkang General Hospital, Singapore, SingaporeDepartment of Radiology, Sengkang General Hospital, Singapore, SingaporeDepartment of Radiology, Sengkang General Hospital, Singapore, SingaporeDepartment of Radiology, Sengkang General Hospital, Singapore, SingaporeDepartment of Diagnostic and Interventional Imaging, KK Women’s and Children’s Hospital, Singapore, SingaporeDepartment of Radiology, Sengkang General Hospital, Singapore, SingaporeDepartment of Radiology, Sengkang General Hospital, Singapore, SingaporeObjectives We aim to evaluate the accuracy of radiologists and radiology residents in the detection of paediatric appendicular fractures with and without the help of a commercially available fracture detection artificial intelligence (AI) solution in the hopes of showing potential clinical benefits in a general hospital setting.Methods This was a retrospective study involving three associate consultants (AC) and three senior residents (SR) in radiology, who acted as readers. One reader from each human group interpreted the radiographs with the aid of AI. Cases were categorised into concordant and discordant cases between each interpreting group. Discordant cases were further evaluated by three independent subspecialty radiology consultants to determine the final diagnosis. A total of 500 anonymised paediatric patient cases (aged 2–15 years) who presented to a tertiary general hospital with a Children’s emergency were retrospectively collected. Main outcome measures include the presence of fracture, accuracy of readers with and without AI, and total time taken to interpret the radiographs.Results The AI solution alone showed the highest accuracy (area under the receiver operating characteristic curve 0.97; AC: 95% CI −0.055 to 0.320, p=0; SR: 95% CI 0.244 to 0.598, p=0). The two readers aided with AI had higher area under curves compared with readers without AI support (AC: 95% CI −0.303 to 0.465, p=0; SR: 95% CI −0.154 to 0.331, p=0). These differences were statistically significant.Conclusion Our study demonstrates excellent results in the detection of paediatric appendicular fractures using a commercially available AI solution. There is potential for the AI solution to function autonomously.https://informatics.bmj.com/content/31/1/e101091.full |
| spellingShingle | Timothy Shao Ern Tan Praveen M Yogendra Adriel Guang Wei Goh Sze Ying Yee Freda Jawan Kelvin Kay Nguan Koh Tian Kai Woon Phey Ming Yeap Min On Tan Accuracy of radiologists and radiology residents in detection of paediatric appendicular fractures with and without artificial intelligence BMJ Health & Care Informatics |
| title | Accuracy of radiologists and radiology residents in detection of paediatric appendicular fractures with and without artificial intelligence |
| title_full | Accuracy of radiologists and radiology residents in detection of paediatric appendicular fractures with and without artificial intelligence |
| title_fullStr | Accuracy of radiologists and radiology residents in detection of paediatric appendicular fractures with and without artificial intelligence |
| title_full_unstemmed | Accuracy of radiologists and radiology residents in detection of paediatric appendicular fractures with and without artificial intelligence |
| title_short | Accuracy of radiologists and radiology residents in detection of paediatric appendicular fractures with and without artificial intelligence |
| title_sort | accuracy of radiologists and radiology residents in detection of paediatric appendicular fractures with and without artificial intelligence |
| url | https://informatics.bmj.com/content/31/1/e101091.full |
| work_keys_str_mv | AT timothyshaoerntan accuracyofradiologistsandradiologyresidentsindetectionofpaediatricappendicularfractureswithandwithoutartificialintelligence AT praveenmyogendra accuracyofradiologistsandradiologyresidentsindetectionofpaediatricappendicularfractureswithandwithoutartificialintelligence AT adrielguangweigoh accuracyofradiologistsandradiologyresidentsindetectionofpaediatricappendicularfractureswithandwithoutartificialintelligence AT szeyingyee accuracyofradiologistsandradiologyresidentsindetectionofpaediatricappendicularfractureswithandwithoutartificialintelligence AT fredajawan accuracyofradiologistsandradiologyresidentsindetectionofpaediatricappendicularfractureswithandwithoutartificialintelligence AT kelvinkaynguankoh accuracyofradiologistsandradiologyresidentsindetectionofpaediatricappendicularfractureswithandwithoutartificialintelligence AT tiankaiwoon accuracyofradiologistsandradiologyresidentsindetectionofpaediatricappendicularfractureswithandwithoutartificialintelligence AT pheymingyeap accuracyofradiologistsandradiologyresidentsindetectionofpaediatricappendicularfractureswithandwithoutartificialintelligence AT minontan accuracyofradiologistsandradiologyresidentsindetectionofpaediatricappendicularfractureswithandwithoutartificialintelligence |