EYE-Llama, an in-domain large language model for ophthalmology
Summary: Training large language models (LLMs) on domain-specific data enhances their performance, yielding more accurate and reliable question-answering (Q&A) systems that support clinical decision-making and patient education. We present EYE-Llama, pretrained on ophthalmology-focused datasets,...
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2025-07-01
|
| Series: | iScience |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2589004225012453 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849318372753276928 |
|---|---|
| author | Tania Haghighi Sina Gholami Jared Todd Sokol Enaika Kishnani Adnan Ahsaniyan Holakou Rahmanian Fares Hedayati Theodore Leng Minhaj Nur Alam |
| author_facet | Tania Haghighi Sina Gholami Jared Todd Sokol Enaika Kishnani Adnan Ahsaniyan Holakou Rahmanian Fares Hedayati Theodore Leng Minhaj Nur Alam |
| author_sort | Tania Haghighi |
| collection | DOAJ |
| description | Summary: Training large language models (LLMs) on domain-specific data enhances their performance, yielding more accurate and reliable question-answering (Q&A) systems that support clinical decision-making and patient education. We present EYE-Llama, pretrained on ophthalmology-focused datasets, including PubMed abstracts, textbooks, and online articles, and fine-tuned on diverse Q&A pairs. We evaluated EYE-Llama against Llama 2, Llama 3, Meditron, ChatDoctor, ChatGPT, and several other LLMs. Using BERT (Bidirectional Encoder Representations from Transformers) score, BART (Bidirectional and Auto-Regressive Transformer) score, and BLEU (Bilingual Evaluation Understudy) metrics, EYE-Llama achieved superior scores. On the MedMCQA benchmark, it outperformed Llama 2, Meditron, and ChatDoctor. On PubMedQA, it achieved 0.96 accuracy, surpassing all models tested. These results demonstrate that domain-specific pretraining and fine-tuning significantly improve medical Q&A performance and underscore the value of specialized models such as EYE-Llama. |
| format | Article |
| id | doaj-art-3bddd21bc98b4973b5ee722d601e3f96 |
| institution | Kabale University |
| issn | 2589-0042 |
| language | English |
| publishDate | 2025-07-01 |
| publisher | Elsevier |
| record_format | Article |
| series | iScience |
| spelling | doaj-art-3bddd21bc98b4973b5ee722d601e3f962025-08-20T03:50:50ZengElsevieriScience2589-00422025-07-0128711298410.1016/j.isci.2025.112984EYE-Llama, an in-domain large language model for ophthalmologyTania Haghighi0Sina Gholami1Jared Todd Sokol2Enaika Kishnani3Adnan Ahsaniyan4Holakou Rahmanian5Fares Hedayati6Theodore Leng7Minhaj Nur Alam8Department of Electrical and Computer Engineering, University of North Carolina at Charlotte, Charlotte, NC, USA; Department of Computer Science, Baha’i Institute for Higher Education, Tehran, IranDepartment of Electrical and Computer Engineering, University of North Carolina at Charlotte, Charlotte, NC, USADepartment of Ophthalmology Stanford University School of Medicine, Stanford, CA, USADepartment of Electrical and Computer Engineering, University of North Carolina at Charlotte, Charlotte, NC, USADepartment of Computer Science, Baha’i Institute for Higher Education, Tehran, IranDepartment of Computer Science, Baha’i Institute for Higher Education, Tehran, IranDepartment of Computer Science, Baha’i Institute for Higher Education, Tehran, IranDepartment of Ophthalmology Stanford University School of Medicine, Stanford, CA, USADepartment of Electrical and Computer Engineering, University of North Carolina at Charlotte, Charlotte, NC, USA; Corresponding authorSummary: Training large language models (LLMs) on domain-specific data enhances their performance, yielding more accurate and reliable question-answering (Q&A) systems that support clinical decision-making and patient education. We present EYE-Llama, pretrained on ophthalmology-focused datasets, including PubMed abstracts, textbooks, and online articles, and fine-tuned on diverse Q&A pairs. We evaluated EYE-Llama against Llama 2, Llama 3, Meditron, ChatDoctor, ChatGPT, and several other LLMs. Using BERT (Bidirectional Encoder Representations from Transformers) score, BART (Bidirectional and Auto-Regressive Transformer) score, and BLEU (Bilingual Evaluation Understudy) metrics, EYE-Llama achieved superior scores. On the MedMCQA benchmark, it outperformed Llama 2, Meditron, and ChatDoctor. On PubMedQA, it achieved 0.96 accuracy, surpassing all models tested. These results demonstrate that domain-specific pretraining and fine-tuning significantly improve medical Q&A performance and underscore the value of specialized models such as EYE-Llama.http://www.sciencedirect.com/science/article/pii/S2589004225012453OphthalmologyArtificial intelligence |
| spellingShingle | Tania Haghighi Sina Gholami Jared Todd Sokol Enaika Kishnani Adnan Ahsaniyan Holakou Rahmanian Fares Hedayati Theodore Leng Minhaj Nur Alam EYE-Llama, an in-domain large language model for ophthalmology iScience Ophthalmology Artificial intelligence |
| title | EYE-Llama, an in-domain large language model for ophthalmology |
| title_full | EYE-Llama, an in-domain large language model for ophthalmology |
| title_fullStr | EYE-Llama, an in-domain large language model for ophthalmology |
| title_full_unstemmed | EYE-Llama, an in-domain large language model for ophthalmology |
| title_short | EYE-Llama, an in-domain large language model for ophthalmology |
| title_sort | eye llama an in domain large language model for ophthalmology |
| topic | Ophthalmology Artificial intelligence |
| url | http://www.sciencedirect.com/science/article/pii/S2589004225012453 |
| work_keys_str_mv | AT taniahaghighi eyellamaanindomainlargelanguagemodelforophthalmology AT sinagholami eyellamaanindomainlargelanguagemodelforophthalmology AT jaredtoddsokol eyellamaanindomainlargelanguagemodelforophthalmology AT enaikakishnani eyellamaanindomainlargelanguagemodelforophthalmology AT adnanahsaniyan eyellamaanindomainlargelanguagemodelforophthalmology AT holakourahmanian eyellamaanindomainlargelanguagemodelforophthalmology AT fareshedayati eyellamaanindomainlargelanguagemodelforophthalmology AT theodoreleng eyellamaanindomainlargelanguagemodelforophthalmology AT minhajnuralam eyellamaanindomainlargelanguagemodelforophthalmology |