TRANSFORMER VS. MAMBA AS SKIN CANCER CLASSIFIER: PRELIMINARY RESULTS
Background: Skin cancer is a deadly disease that takes dozens of thousands of lives yearly. The key element of successful treatment of it is early detection. However, invasive detection methods are not always feasible. Meanwhile, Transformers, the most renowned and researched models keep being compu...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Igor Sikorsky Kyiv Polytechnic Institute
2024-12-01
|
| Series: | KPI Science News |
| Online Access: | https://scinews.kpi.ua/article/view/301028 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850196197326716928 |
|---|---|
| author | Владислав Нікітін Валерій Данілов |
| author_facet | Владислав Нікітін Валерій Данілов |
| author_sort | Владислав Нікітін |
| collection | DOAJ |
| description | Background: Skin cancer is a deadly disease that takes dozens of thousands of lives yearly. The key element of successful treatment of it is early detection. However, invasive detection methods are not always feasible. Meanwhile, Transformers, the most renowned and researched models keep being computationally heavy. In this paper we investigate Mamba model for such classification problem compared to Transformers.
Objective: This paper compares the effectiveness of two machine learning architectures, Vision Transformer (ViT) and Mamba, for skin cancer classification using dermoscopy images. The goal is to determine if Mamba can provide a computationally efficient alternative to ViT without decrease in diagnostics accuracy.
Methods: We used the HAM10000 dataset, a well-known benchmark in skin cancer classification, with 10015 dermoscopic images. We preprocessed the data to address issues like class imbalance and normalized the images. Both ViT and Mamba models were pretrained on the ImageNet dataset and fine-tuned for skin cancer classification. We evaluated the models based on overall accuracy and F1 scores for specific classes of skin cancer.
Results: The results show that both ViT and Mamba models have similar overall accuracy, with Mamba models performing slightly better in classifying less represented classes like Bowen's Disease and Dermatofibroma. Both models demonstrated high F1 scores for Melanoma, indicating their effectiveness in identifying this severe form of skin cancer.
Conclusions: Our findings suggest that Mamba is a viable alternative to ViT for skin cancer classification, offering similar accuracy while potentially reducing computational costs. This could make non-invasive skin cancer diagnostics more accessible and affordable. Further research is needed to explore other variations of the Mamba model and to fine-tune its performance on larger datasets. |
| format | Article |
| id | doaj-art-98ba16a3f01b4448916888e4a4fc11d7 |
| institution | OA Journals |
| issn | 2617-5509 2663-7472 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | Igor Sikorsky Kyiv Polytechnic Institute |
| record_format | Article |
| series | KPI Science News |
| spelling | doaj-art-98ba16a3f01b4448916888e4a4fc11d72025-08-20T02:13:31ZengIgor Sikorsky Kyiv Polytechnic InstituteKPI Science News2617-55092663-74722024-12-011371-410.20535/kpisn.2024.1-4.301028339392TRANSFORMER VS. MAMBA AS SKIN CANCER CLASSIFIER: PRELIMINARY RESULTSВладислав Нікітін0https://orcid.org/0009-0001-9921-0213Валерій Данілов1https://orcid.org/0000-0003-3389-3661National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”, IASA, Department of Artificial Intelligence (AI)National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”, IASA, Department of Artificial Intelligence (AI)ІBackground: Skin cancer is a deadly disease that takes dozens of thousands of lives yearly. The key element of successful treatment of it is early detection. However, invasive detection methods are not always feasible. Meanwhile, Transformers, the most renowned and researched models keep being computationally heavy. In this paper we investigate Mamba model for such classification problem compared to Transformers. Objective: This paper compares the effectiveness of two machine learning architectures, Vision Transformer (ViT) and Mamba, for skin cancer classification using dermoscopy images. The goal is to determine if Mamba can provide a computationally efficient alternative to ViT without decrease in diagnostics accuracy. Methods: We used the HAM10000 dataset, a well-known benchmark in skin cancer classification, with 10015 dermoscopic images. We preprocessed the data to address issues like class imbalance and normalized the images. Both ViT and Mamba models were pretrained on the ImageNet dataset and fine-tuned for skin cancer classification. We evaluated the models based on overall accuracy and F1 scores for specific classes of skin cancer. Results: The results show that both ViT and Mamba models have similar overall accuracy, with Mamba models performing slightly better in classifying less represented classes like Bowen's Disease and Dermatofibroma. Both models demonstrated high F1 scores for Melanoma, indicating their effectiveness in identifying this severe form of skin cancer. Conclusions: Our findings suggest that Mamba is a viable alternative to ViT for skin cancer classification, offering similar accuracy while potentially reducing computational costs. This could make non-invasive skin cancer diagnostics more accessible and affordable. Further research is needed to explore other variations of the Mamba model and to fine-tune its performance on larger datasets.https://scinews.kpi.ua/article/view/301028 |
| spellingShingle | Владислав Нікітін Валерій Данілов TRANSFORMER VS. MAMBA AS SKIN CANCER CLASSIFIER: PRELIMINARY RESULTS KPI Science News |
| title | TRANSFORMER VS. MAMBA AS SKIN CANCER CLASSIFIER: PRELIMINARY RESULTS |
| title_full | TRANSFORMER VS. MAMBA AS SKIN CANCER CLASSIFIER: PRELIMINARY RESULTS |
| title_fullStr | TRANSFORMER VS. MAMBA AS SKIN CANCER CLASSIFIER: PRELIMINARY RESULTS |
| title_full_unstemmed | TRANSFORMER VS. MAMBA AS SKIN CANCER CLASSIFIER: PRELIMINARY RESULTS |
| title_short | TRANSFORMER VS. MAMBA AS SKIN CANCER CLASSIFIER: PRELIMINARY RESULTS |
| title_sort | transformer vs mamba as skin cancer classifier preliminary results |
| url | https://scinews.kpi.ua/article/view/301028 |
| work_keys_str_mv | AT vladislavníkítín transformervsmambaasskincancerclassifierpreliminaryresults AT valeríjdanílov transformervsmambaasskincancerclassifierpreliminaryresults |