SSC-Net: A multi-task joint learning network for tongue image segmentation and multi-label classification
Background Traditional Chinese medicine (TCM) tongue diagnosis, through the comprehensive observation of tongue’s diverse characteristics, allows an understanding of the state of the body’s viscera as well as Qi and blood levels. Automatic tongue image recognition methods could support TCM practitio...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SAGE Publishing
2025-05-01
|
| Series: | Digital Health |
| Online Access: | https://doi.org/10.1177/20552076251343696 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850270255426830336 |
|---|---|
| author | Xiaopeng Sha Zheng Guan Ying Wang Jinglu Han Yi Wang Zhaojun Chen |
| author_facet | Xiaopeng Sha Zheng Guan Ying Wang Jinglu Han Yi Wang Zhaojun Chen |
| author_sort | Xiaopeng Sha |
| collection | DOAJ |
| description | Background Traditional Chinese medicine (TCM) tongue diagnosis, through the comprehensive observation of tongue’s diverse characteristics, allows an understanding of the state of the body’s viscera as well as Qi and blood levels. Automatic tongue image recognition methods could support TCM practitioners by providing auxiliary diagnostic suggestions. However, most learning-based methods often address a narrow scope of the tongue’s attributes, failing to fully exploit the information contained within the tongue images. Objective To classify multifaceted tongue characteristics, and fully utilize the latent correlation information between tongue segmentation and classification tasks, we proposed a multi-task joint learning network for simultaneous tongue body segmentation and multi-label Classification, named SSC-Net. Methods Firstly, the shared feature encoder extracts features for both segmentation and classification tasks, where the segmentation result is utilized to mask redundant features that may impede classification accuracy. Subsequently, the ROI extraction module locates and extracts the tongue body region, and the feature fusion module combines tongue body features from bottom to top. Finally, a fine-grained classification module is employed for multi-label classification on multiple tongue characteristics. Results To evaluate the performance of the SSC-Net, we collected a tongue image dataset, BUCM, and conducted extensive experiments on it. The experimental results show that the proposed method when segmenting and classifying simultaneously, achieved 0.9943 DSC for the segmentation task, 92.02 mAP, and 0.851 overall F1-score for the classification task. Conclusion The proposed method can effectively classify multiple tongue characteristics with the support of the multi-task learning strategy and the integration of a fine-grained classification module. Code is available here. |
| format | Article |
| id | doaj-art-38b19fb7a2da41eebe115eb302ff5c76 |
| institution | OA Journals |
| issn | 2055-2076 |
| language | English |
| publishDate | 2025-05-01 |
| publisher | SAGE Publishing |
| record_format | Article |
| series | Digital Health |
| spelling | doaj-art-38b19fb7a2da41eebe115eb302ff5c762025-08-20T01:52:42ZengSAGE PublishingDigital Health2055-20762025-05-011110.1177/20552076251343696SSC-Net: A multi-task joint learning network for tongue image segmentation and multi-label classificationXiaopeng Sha0Zheng Guan1Ying Wang2Jinglu Han3Yi Wang4Zhaojun Chen5 , Qinhuangdao, China , Qinhuangdao, China , Beijing, China , Jinan, China , Qinhuangdao, China Department of Hand and Foot Surgery, Beijing University of Chinese Medicine Third Affiliated Hospital, Beijing, ChinaBackground Traditional Chinese medicine (TCM) tongue diagnosis, through the comprehensive observation of tongue’s diverse characteristics, allows an understanding of the state of the body’s viscera as well as Qi and blood levels. Automatic tongue image recognition methods could support TCM practitioners by providing auxiliary diagnostic suggestions. However, most learning-based methods often address a narrow scope of the tongue’s attributes, failing to fully exploit the information contained within the tongue images. Objective To classify multifaceted tongue characteristics, and fully utilize the latent correlation information between tongue segmentation and classification tasks, we proposed a multi-task joint learning network for simultaneous tongue body segmentation and multi-label Classification, named SSC-Net. Methods Firstly, the shared feature encoder extracts features for both segmentation and classification tasks, where the segmentation result is utilized to mask redundant features that may impede classification accuracy. Subsequently, the ROI extraction module locates and extracts the tongue body region, and the feature fusion module combines tongue body features from bottom to top. Finally, a fine-grained classification module is employed for multi-label classification on multiple tongue characteristics. Results To evaluate the performance of the SSC-Net, we collected a tongue image dataset, BUCM, and conducted extensive experiments on it. The experimental results show that the proposed method when segmenting and classifying simultaneously, achieved 0.9943 DSC for the segmentation task, 92.02 mAP, and 0.851 overall F1-score for the classification task. Conclusion The proposed method can effectively classify multiple tongue characteristics with the support of the multi-task learning strategy and the integration of a fine-grained classification module. Code is available here.https://doi.org/10.1177/20552076251343696 |
| spellingShingle | Xiaopeng Sha Zheng Guan Ying Wang Jinglu Han Yi Wang Zhaojun Chen SSC-Net: A multi-task joint learning network for tongue image segmentation and multi-label classification Digital Health |
| title | SSC-Net: A multi-task joint learning network for tongue image segmentation and multi-label classification |
| title_full | SSC-Net: A multi-task joint learning network for tongue image segmentation and multi-label classification |
| title_fullStr | SSC-Net: A multi-task joint learning network for tongue image segmentation and multi-label classification |
| title_full_unstemmed | SSC-Net: A multi-task joint learning network for tongue image segmentation and multi-label classification |
| title_short | SSC-Net: A multi-task joint learning network for tongue image segmentation and multi-label classification |
| title_sort | ssc net a multi task joint learning network for tongue image segmentation and multi label classification |
| url | https://doi.org/10.1177/20552076251343696 |
| work_keys_str_mv | AT xiaopengsha sscnetamultitaskjointlearningnetworkfortongueimagesegmentationandmultilabelclassification AT zhengguan sscnetamultitaskjointlearningnetworkfortongueimagesegmentationandmultilabelclassification AT yingwang sscnetamultitaskjointlearningnetworkfortongueimagesegmentationandmultilabelclassification AT jingluhan sscnetamultitaskjointlearningnetworkfortongueimagesegmentationandmultilabelclassification AT yiwang sscnetamultitaskjointlearningnetworkfortongueimagesegmentationandmultilabelclassification AT zhaojunchen sscnetamultitaskjointlearningnetworkfortongueimagesegmentationandmultilabelclassification |