In-context learning for label-efficient cancer image classification in oncology
The application of artificial intelligence in oncology has been limited by its reliance on large, annotated datasets and the need for retraining models for domain-specific diagnostic tasks. Taking heed of these limitations, we investigated in-context learning as a pragmatic alternative to model retr...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2025-01-01
|
| Series: | Informatics in Medicine Unlocked |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2352914825000723 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849226912777371648 |
|---|---|
| author | Mobina Shrestha Bishwas Mandal Vishal Mandal Asis Shrestha Amir Babu Shrestha |
| author_facet | Mobina Shrestha Bishwas Mandal Vishal Mandal Asis Shrestha Amir Babu Shrestha |
| author_sort | Mobina Shrestha |
| collection | DOAJ |
| description | The application of artificial intelligence in oncology has been limited by its reliance on large, annotated datasets and the need for retraining models for domain-specific diagnostic tasks. Taking heed of these limitations, we investigated in-context learning as a pragmatic alternative to model retraining by allowing models to adapt to new diagnostic tasks using only a few labeled examples at inference, without the need for retraining. Using four vision-language models (VLMs) -- Paligemma, CLIP, ALIGN and GPT-4o, we evaluated the performance across three oncology datasets: MHIST, PatchCamelyon and HAM10000. To the best of our knowledge, this is the first study to compare the performance of multiple VLMs with in-context learning on different oncology classification tasks. Without any parameter updates, all models showed significant gains with few-shot prompting, with GPT-4o reaching an F1 score of 0.81 in binary classification and 0.60 in multi-class classification settings. While these results remain below the ceiling of fully fine-tuned systems, they highlight the potential of ICL to approximate task-specific behavior using only a handful of examples, reflecting how clinicians often reason from prior cases. Notably, open-source models like Paligemma and CLIP demonstrated competitive gains despite their smaller size, suggesting feasibility for deployment in computing constrained clinical environments. Overall, these findings highlight the potential of ICL as a practical solution in oncology, particularly for rare cancers and resource-limited contexts where fine-tuning is infeasible and annotated data is difficult to obtain. |
| format | Article |
| id | doaj-art-11bd2349ef484ce39364a81e9e5ff35a |
| institution | Kabale University |
| issn | 2352-9148 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Informatics in Medicine Unlocked |
| spelling | doaj-art-11bd2349ef484ce39364a81e9e5ff35a2025-08-24T05:13:28ZengElsevierInformatics in Medicine Unlocked2352-91482025-01-015810168310.1016/j.imu.2025.101683In-context learning for label-efficient cancer image classification in oncologyMobina Shrestha0Bishwas Mandal1Vishal Mandal2Asis Shrestha3Amir Babu Shrestha4Sentara Albemarle Medical Center, USACS, Elizabeth City, NC, USAKansas State University, USACSX Corp, USAUAMS Winthrop P. Rockefeller Cancer Institute, USAManipal College of Medical Sciences, Nepal; Corresponding author.The application of artificial intelligence in oncology has been limited by its reliance on large, annotated datasets and the need for retraining models for domain-specific diagnostic tasks. Taking heed of these limitations, we investigated in-context learning as a pragmatic alternative to model retraining by allowing models to adapt to new diagnostic tasks using only a few labeled examples at inference, without the need for retraining. Using four vision-language models (VLMs) -- Paligemma, CLIP, ALIGN and GPT-4o, we evaluated the performance across three oncology datasets: MHIST, PatchCamelyon and HAM10000. To the best of our knowledge, this is the first study to compare the performance of multiple VLMs with in-context learning on different oncology classification tasks. Without any parameter updates, all models showed significant gains with few-shot prompting, with GPT-4o reaching an F1 score of 0.81 in binary classification and 0.60 in multi-class classification settings. While these results remain below the ceiling of fully fine-tuned systems, they highlight the potential of ICL to approximate task-specific behavior using only a handful of examples, reflecting how clinicians often reason from prior cases. Notably, open-source models like Paligemma and CLIP demonstrated competitive gains despite their smaller size, suggesting feasibility for deployment in computing constrained clinical environments. Overall, these findings highlight the potential of ICL as a practical solution in oncology, particularly for rare cancers and resource-limited contexts where fine-tuning is infeasible and annotated data is difficult to obtain.http://www.sciencedirect.com/science/article/pii/S2352914825000723 |
| spellingShingle | Mobina Shrestha Bishwas Mandal Vishal Mandal Asis Shrestha Amir Babu Shrestha In-context learning for label-efficient cancer image classification in oncology Informatics in Medicine Unlocked |
| title | In-context learning for label-efficient cancer image classification in oncology |
| title_full | In-context learning for label-efficient cancer image classification in oncology |
| title_fullStr | In-context learning for label-efficient cancer image classification in oncology |
| title_full_unstemmed | In-context learning for label-efficient cancer image classification in oncology |
| title_short | In-context learning for label-efficient cancer image classification in oncology |
| title_sort | in context learning for label efficient cancer image classification in oncology |
| url | http://www.sciencedirect.com/science/article/pii/S2352914825000723 |
| work_keys_str_mv | AT mobinashrestha incontextlearningforlabelefficientcancerimageclassificationinoncology AT bishwasmandal incontextlearningforlabelefficientcancerimageclassificationinoncology AT vishalmandal incontextlearningforlabelefficientcancerimageclassificationinoncology AT asisshrestha incontextlearningforlabelefficientcancerimageclassificationinoncology AT amirbabushrestha incontextlearningforlabelefficientcancerimageclassificationinoncology |