Pilot study of ASSORT (AI-based Symptom Stratification in Ophthalmology for Rapid Triage): a triage tool for ophthalmic emergencies
Introduction: ASSORT (AI-based Symptom Stratification in Ophthalmology for Rapid Triage) is a GPT-4-based triage tool designed to assess ophthalmic emergencies using a three-tier color-coded system. This study compares ASSORT to the Rescue triage method, using the ophthalmologist’s final assessment...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2025-10-01
|
| Series: | AJO International |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2950253525000620 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849246162875318272 |
|---|---|
| author | Claudio Xompero Lorenzo Rossi Francesca Amoroso Antonio Bechara Ghobril Diana Elena Ionita Eric H. Souied Carl-Joe Mehanna |
| author_facet | Claudio Xompero Lorenzo Rossi Francesca Amoroso Antonio Bechara Ghobril Diana Elena Ionita Eric H. Souied Carl-Joe Mehanna |
| author_sort | Claudio Xompero |
| collection | DOAJ |
| description | Introduction: ASSORT (AI-based Symptom Stratification in Ophthalmology for Rapid Triage) is a GPT-4-based triage tool designed to assess ophthalmic emergencies using a three-tier color-coded system. This study compares ASSORT to the Rescue triage method, using the ophthalmologist’s final assessment as the reference standard. Materials and methods: A prospective study was conducted at the Créteil University Hospital from April to June 2024. Each patient underwent triage using ASSORT, followed by the Rescue triage method. Both tools used the same color-coding system to stratify severity: yellow for emergency cases, green for urgent cases, and white for non-urgent cases. An examining ophthalmologist in their final year of residency performed the final assessment. Concordance between the ophthalmologist and each of the tools was analyzed using Cohen’s kappa coefficient, alongside precision and recall metrics. Results: Fifty-one patients were included. Case severities were distributed as follows: 22/51 white, 27/51 green, and 2/51 yellow, with conjunctivitis (17.5 %) and corneal abrasions (12.5 %) being the two most common presentations. ASSORT demonstrated moderate agreement with the ophthalmologist (κ = 0.54), whereas Rescue showed stronger concordance (κ = 0.85). ASSORT tended to overestimate urgency, assigning more yellow codes than the ophthalmologist. McNemar’s test confirmed significant misclassification by ASSORT (p = 0.0156), while Rescue showed no significant deviation (p = 0.5). Conclusion: While the small sample size limits generalizability, ASSORT shows potential for AI-driven ophthalmic triage but currently overestimates severity compared to the ophthalmologist. Further refinements such as reinforcement learning and multimodal input, as well as large-scale validation are needed to improve accuracy and reduce unnecessary emergency classifications before clinical implementation. |
| format | Article |
| id | doaj-art-5ce35086e59d4b6ebbe033266840b8d6 |
| institution | Kabale University |
| issn | 2950-2535 |
| language | English |
| publishDate | 2025-10-01 |
| publisher | Elsevier |
| record_format | Article |
| series | AJO International |
| spelling | doaj-art-5ce35086e59d4b6ebbe033266840b8d62025-08-20T03:58:35ZengElsevierAJO International2950-25352025-10-012310015910.1016/j.ajoint.2025.100159Pilot study of ASSORT (AI-based Symptom Stratification in Ophthalmology for Rapid Triage): a triage tool for ophthalmic emergenciesClaudio Xompero0Lorenzo Rossi1Francesca Amoroso2Antonio Bechara Ghobril3Diana Elena Ionita4Eric H. Souied5Carl-Joe Mehanna6Department of Ophthalmology, Centre Hospitalier Intercommunal de Créteil, Créteil, FranceDepartment of Ophthalmology, Centre Hospitalier Intercommunal de Créteil, Créteil, FranceDepartment of Ophthalmology, Centre Hospitalier Intercommunal de Créteil, Créteil, FranceDepartment of Ophthalmology, Centre Hospitalier Intercommunal de Créteil, Créteil, FranceDepartment of Ophthalmology, Centre Hospitalier Intercommunal de Créteil, Créteil, FranceDepartment of Ophthalmology, Centre Hospitalier Intercommunal de Créteil, Créteil, FranceCorresponding author at: Department of Ophthalmology, Centre Hospitalier Intercommunal de Créteil, 40 Avenue de Verdun, 94000 Créteil, France.; Department of Ophthalmology, Centre Hospitalier Intercommunal de Créteil, Créteil, FranceIntroduction: ASSORT (AI-based Symptom Stratification in Ophthalmology for Rapid Triage) is a GPT-4-based triage tool designed to assess ophthalmic emergencies using a three-tier color-coded system. This study compares ASSORT to the Rescue triage method, using the ophthalmologist’s final assessment as the reference standard. Materials and methods: A prospective study was conducted at the Créteil University Hospital from April to June 2024. Each patient underwent triage using ASSORT, followed by the Rescue triage method. Both tools used the same color-coding system to stratify severity: yellow for emergency cases, green for urgent cases, and white for non-urgent cases. An examining ophthalmologist in their final year of residency performed the final assessment. Concordance between the ophthalmologist and each of the tools was analyzed using Cohen’s kappa coefficient, alongside precision and recall metrics. Results: Fifty-one patients were included. Case severities were distributed as follows: 22/51 white, 27/51 green, and 2/51 yellow, with conjunctivitis (17.5 %) and corneal abrasions (12.5 %) being the two most common presentations. ASSORT demonstrated moderate agreement with the ophthalmologist (κ = 0.54), whereas Rescue showed stronger concordance (κ = 0.85). ASSORT tended to overestimate urgency, assigning more yellow codes than the ophthalmologist. McNemar’s test confirmed significant misclassification by ASSORT (p = 0.0156), while Rescue showed no significant deviation (p = 0.5). Conclusion: While the small sample size limits generalizability, ASSORT shows potential for AI-driven ophthalmic triage but currently overestimates severity compared to the ophthalmologist. Further refinements such as reinforcement learning and multimodal input, as well as large-scale validation are needed to improve accuracy and reduce unnecessary emergency classifications before clinical implementation.http://www.sciencedirect.com/science/article/pii/S2950253525000620TriageOphthalmic emergenciesLarge language modelsArtificial intelligenceTelemedicine |
| spellingShingle | Claudio Xompero Lorenzo Rossi Francesca Amoroso Antonio Bechara Ghobril Diana Elena Ionita Eric H. Souied Carl-Joe Mehanna Pilot study of ASSORT (AI-based Symptom Stratification in Ophthalmology for Rapid Triage): a triage tool for ophthalmic emergencies AJO International Triage Ophthalmic emergencies Large language models Artificial intelligence Telemedicine |
| title | Pilot study of ASSORT (AI-based Symptom Stratification in Ophthalmology for Rapid Triage): a triage tool for ophthalmic emergencies |
| title_full | Pilot study of ASSORT (AI-based Symptom Stratification in Ophthalmology for Rapid Triage): a triage tool for ophthalmic emergencies |
| title_fullStr | Pilot study of ASSORT (AI-based Symptom Stratification in Ophthalmology for Rapid Triage): a triage tool for ophthalmic emergencies |
| title_full_unstemmed | Pilot study of ASSORT (AI-based Symptom Stratification in Ophthalmology for Rapid Triage): a triage tool for ophthalmic emergencies |
| title_short | Pilot study of ASSORT (AI-based Symptom Stratification in Ophthalmology for Rapid Triage): a triage tool for ophthalmic emergencies |
| title_sort | pilot study of assort ai based symptom stratification in ophthalmology for rapid triage a triage tool for ophthalmic emergencies |
| topic | Triage Ophthalmic emergencies Large language models Artificial intelligence Telemedicine |
| url | http://www.sciencedirect.com/science/article/pii/S2950253525000620 |
| work_keys_str_mv | AT claudioxompero pilotstudyofassortaibasedsymptomstratificationinophthalmologyforrapidtriageatriagetoolforophthalmicemergencies AT lorenzorossi pilotstudyofassortaibasedsymptomstratificationinophthalmologyforrapidtriageatriagetoolforophthalmicemergencies AT francescaamoroso pilotstudyofassortaibasedsymptomstratificationinophthalmologyforrapidtriageatriagetoolforophthalmicemergencies AT antoniobecharaghobril pilotstudyofassortaibasedsymptomstratificationinophthalmologyforrapidtriageatriagetoolforophthalmicemergencies AT dianaelenaionita pilotstudyofassortaibasedsymptomstratificationinophthalmologyforrapidtriageatriagetoolforophthalmicemergencies AT erichsouied pilotstudyofassortaibasedsymptomstratificationinophthalmologyforrapidtriageatriagetoolforophthalmicemergencies AT carljoemehanna pilotstudyofassortaibasedsymptomstratificationinophthalmologyforrapidtriageatriagetoolforophthalmicemergencies |