Deep Learning Multi-Modal Melanoma Detection: Algorithm Development and Validation
Abstract BackgroundThe visual similarity of melanoma and seborrheic keratosis has made it difficult for older patients with disabilities to know when to seek medical attention, contributing to the metastasis of melanoma. ObjectiveThis study aimed to present a novel...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2025-08-01
|
| Series: | JMIR AI |
| Online Access: | https://ai.jmir.org/2025/1/e66561 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849232880398499840 |
|---|---|
| author | Nithika Vivek Karthik Ramesh |
| author_facet | Nithika Vivek Karthik Ramesh |
| author_sort | Nithika Vivek |
| collection | DOAJ |
| description |
Abstract
BackgroundThe visual similarity of melanoma and seborrheic keratosis has made it difficult for older patients with disabilities to know when to seek medical attention, contributing to the metastasis of melanoma.
ObjectiveThis study aimed to present a novel multimodal deep learning-based technique to distinguish between melanoma and seborrheic keratosis.
MethodsOur strategy is three-fold: (1) use patient image data to train and test three deep learning models using transfer learning (ResNet50, InceptionV3, and VGG16) and one author-designed model, (2) use patient metadata to train and test a deep learning model, and (3) combine the predictions of the image model with the best accuracy and the metadata model, using nonlinear least squares regression to specify ideal weights to each model for a combined prediction.
ResultsThe accuracy of the combined model was 88% (195/221 classified correctly) on test data from the HAM10000 dataset. Model reliability was assessed by visualizing the output activation map of each model and comparing the diagnosis patterns to that of dermatologists. The addition of metadata to the image dataset was key to reducing the false-negative and false-positive rates simultaneously, thereby producing better metrics and improving overall model accuracy.
ConclusionsResults from this experiment could be used to eliminate late diagnosis of melanoma via easy access to an app. Future experiments can use text data (subjective data pertaining to how the patient felt over a certain period of time) to allow this model to reflect the real hospital setting to a greater extent. |
| format | Article |
| id | doaj-art-1238f46cd4654e739b92874c75f6bcbf |
| institution | Kabale University |
| issn | 2817-1705 |
| language | English |
| publishDate | 2025-08-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | JMIR AI |
| spelling | doaj-art-1238f46cd4654e739b92874c75f6bcbf2025-08-20T16:41:25ZengJMIR PublicationsJMIR AI2817-17052025-08-014e66561e6656110.2196/66561Deep Learning Multi-Modal Melanoma Detection: Algorithm Development and ValidationNithika Vivekhttp://orcid.org/0009-0006-7927-1584Karthik Rameshhttp://orcid.org/0000-0003-2938-7942 Abstract BackgroundThe visual similarity of melanoma and seborrheic keratosis has made it difficult for older patients with disabilities to know when to seek medical attention, contributing to the metastasis of melanoma. ObjectiveThis study aimed to present a novel multimodal deep learning-based technique to distinguish between melanoma and seborrheic keratosis. MethodsOur strategy is three-fold: (1) use patient image data to train and test three deep learning models using transfer learning (ResNet50, InceptionV3, and VGG16) and one author-designed model, (2) use patient metadata to train and test a deep learning model, and (3) combine the predictions of the image model with the best accuracy and the metadata model, using nonlinear least squares regression to specify ideal weights to each model for a combined prediction. ResultsThe accuracy of the combined model was 88% (195/221 classified correctly) on test data from the HAM10000 dataset. Model reliability was assessed by visualizing the output activation map of each model and comparing the diagnosis patterns to that of dermatologists. The addition of metadata to the image dataset was key to reducing the false-negative and false-positive rates simultaneously, thereby producing better metrics and improving overall model accuracy. ConclusionsResults from this experiment could be used to eliminate late diagnosis of melanoma via easy access to an app. Future experiments can use text data (subjective data pertaining to how the patient felt over a certain period of time) to allow this model to reflect the real hospital setting to a greater extent.https://ai.jmir.org/2025/1/e66561 |
| spellingShingle | Nithika Vivek Karthik Ramesh Deep Learning Multi-Modal Melanoma Detection: Algorithm Development and Validation JMIR AI |
| title | Deep Learning Multi-Modal Melanoma Detection: Algorithm Development and Validation |
| title_full | Deep Learning Multi-Modal Melanoma Detection: Algorithm Development and Validation |
| title_fullStr | Deep Learning Multi-Modal Melanoma Detection: Algorithm Development and Validation |
| title_full_unstemmed | Deep Learning Multi-Modal Melanoma Detection: Algorithm Development and Validation |
| title_short | Deep Learning Multi-Modal Melanoma Detection: Algorithm Development and Validation |
| title_sort | deep learning multi modal melanoma detection algorithm development and validation |
| url | https://ai.jmir.org/2025/1/e66561 |
| work_keys_str_mv | AT nithikavivek deeplearningmultimodalmelanomadetectionalgorithmdevelopmentandvalidation AT karthikramesh deeplearningmultimodalmelanomadetectionalgorithmdevelopmentandvalidation |