Classification of cervical vertebral maturation stages with machine learning models: leveraging datasets with high inter- and intra-observer agreement
Abstract Objectives This study aimed to assess the accuracy of machine learning (ML) models with feature selection technique in classifying cervical vertebral maturation stages (CVMS). Consensus-based datasets were used for models training and evaluation for their model generalization capabilities o...
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SpringerOpen
2024-09-01
|
| Series: | Progress in Orthodontics |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s40510-024-00535-1 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849334487196893184 |
|---|---|
| author | Potjanee Kanchanapiboon Pitipat Tunksook Prinya Tunksook Panrasee Ritthipravat Supatchai Boonpratham Yodhathai Satravaha Chaiyapol Chaweewannakorn Supakit Peanchitlertkajorn |
| author_facet | Potjanee Kanchanapiboon Pitipat Tunksook Prinya Tunksook Panrasee Ritthipravat Supatchai Boonpratham Yodhathai Satravaha Chaiyapol Chaweewannakorn Supakit Peanchitlertkajorn |
| author_sort | Potjanee Kanchanapiboon |
| collection | DOAJ |
| description | Abstract Objectives This study aimed to assess the accuracy of machine learning (ML) models with feature selection technique in classifying cervical vertebral maturation stages (CVMS). Consensus-based datasets were used for models training and evaluation for their model generalization capabilities on unseen datasets. Methods Three clinicians independently rated CVMS on 1380 lateral cephalograms, resulting in the creation of five datasets: two consensus-based datasets (Complete Agreement and Majority Voting), and three datasets based on a single rater’s evaluations. Additionally, landmarks annotation of the second to fourth cervical vertebrae and patients’ information underwent a feature selection process. These datasets were used to train various ML models and identify the top-performing model for each dataset. These models were subsequently tested on their generalization capabilities. Results Features that considered significant in the consensus-based datasets were consistent with a CVMS guideline. The Support Vector Machine model on the Complete Agreement dataset achieved the highest accuracy (77.4%), followed by the Multi-Layer Perceptron model on the Majority Voting dataset (69.6%). Models from individual ratings showed lower accuracies (60.4–67.9%). The consensus-based training models also exhibited lower coefficient of variation (CV), indicating superior generalization capability compared to models from single raters. Conclusion ML models trained on consensus-based datasets for CVMS classification exhibited the highest accuracy, with significant features consistent with the original CVMS guidelines. These models also showed robust generalization capabilities, underscoring the importance of dataset quality. |
| format | Article |
| id | doaj-art-ddab0ebf532d4f9fb31f6558cb787852 |
| institution | Kabale University |
| issn | 2196-1042 |
| language | English |
| publishDate | 2024-09-01 |
| publisher | SpringerOpen |
| record_format | Article |
| series | Progress in Orthodontics |
| spelling | doaj-art-ddab0ebf532d4f9fb31f6558cb7878522025-08-20T03:45:32ZengSpringerOpenProgress in Orthodontics2196-10422024-09-0125111410.1186/s40510-024-00535-1Classification of cervical vertebral maturation stages with machine learning models: leveraging datasets with high inter- and intra-observer agreementPotjanee Kanchanapiboon0Pitipat Tunksook1Prinya Tunksook2Panrasee Ritthipravat3Supatchai Boonpratham4Yodhathai Satravaha5Chaiyapol Chaweewannakorn6Supakit Peanchitlertkajorn7Division of Nuclear Medicine, Department of Radiology, Faculty of Medicine Siriraj Hospital, Mahidol UniversityDepartment of Orthodontics, Faculty of Dentistry, Mahidol UniversityPrivate PracticeDepartment of Biomedical Engineering, Faculty of Engineering, Mahidol UniversityDepartment of Orthodontics, Faculty of Dentistry, Mahidol UniversityDepartment of Orthodontics, Faculty of Dentistry, Mahidol UniversityDepartment of Orthodontics, Faculty of Dentistry, Mahidol UniversityDepartment of Orthodontics, Faculty of Dentistry, Mahidol UniversityAbstract Objectives This study aimed to assess the accuracy of machine learning (ML) models with feature selection technique in classifying cervical vertebral maturation stages (CVMS). Consensus-based datasets were used for models training and evaluation for their model generalization capabilities on unseen datasets. Methods Three clinicians independently rated CVMS on 1380 lateral cephalograms, resulting in the creation of five datasets: two consensus-based datasets (Complete Agreement and Majority Voting), and three datasets based on a single rater’s evaluations. Additionally, landmarks annotation of the second to fourth cervical vertebrae and patients’ information underwent a feature selection process. These datasets were used to train various ML models and identify the top-performing model for each dataset. These models were subsequently tested on their generalization capabilities. Results Features that considered significant in the consensus-based datasets were consistent with a CVMS guideline. The Support Vector Machine model on the Complete Agreement dataset achieved the highest accuracy (77.4%), followed by the Multi-Layer Perceptron model on the Majority Voting dataset (69.6%). Models from individual ratings showed lower accuracies (60.4–67.9%). The consensus-based training models also exhibited lower coefficient of variation (CV), indicating superior generalization capability compared to models from single raters. Conclusion ML models trained on consensus-based datasets for CVMS classification exhibited the highest accuracy, with significant features consistent with the original CVMS guidelines. These models also showed robust generalization capabilities, underscoring the importance of dataset quality.https://doi.org/10.1186/s40510-024-00535-1Cervical vertebral maturation stagesMachine learningArtificial intelligenceConsensus-based modelLandmark annotation |
| spellingShingle | Potjanee Kanchanapiboon Pitipat Tunksook Prinya Tunksook Panrasee Ritthipravat Supatchai Boonpratham Yodhathai Satravaha Chaiyapol Chaweewannakorn Supakit Peanchitlertkajorn Classification of cervical vertebral maturation stages with machine learning models: leveraging datasets with high inter- and intra-observer agreement Progress in Orthodontics Cervical vertebral maturation stages Machine learning Artificial intelligence Consensus-based model Landmark annotation |
| title | Classification of cervical vertebral maturation stages with machine learning models: leveraging datasets with high inter- and intra-observer agreement |
| title_full | Classification of cervical vertebral maturation stages with machine learning models: leveraging datasets with high inter- and intra-observer agreement |
| title_fullStr | Classification of cervical vertebral maturation stages with machine learning models: leveraging datasets with high inter- and intra-observer agreement |
| title_full_unstemmed | Classification of cervical vertebral maturation stages with machine learning models: leveraging datasets with high inter- and intra-observer agreement |
| title_short | Classification of cervical vertebral maturation stages with machine learning models: leveraging datasets with high inter- and intra-observer agreement |
| title_sort | classification of cervical vertebral maturation stages with machine learning models leveraging datasets with high inter and intra observer agreement |
| topic | Cervical vertebral maturation stages Machine learning Artificial intelligence Consensus-based model Landmark annotation |
| url | https://doi.org/10.1186/s40510-024-00535-1 |
| work_keys_str_mv | AT potjaneekanchanapiboon classificationofcervicalvertebralmaturationstageswithmachinelearningmodelsleveragingdatasetswithhighinterandintraobserveragreement AT pitipattunksook classificationofcervicalvertebralmaturationstageswithmachinelearningmodelsleveragingdatasetswithhighinterandintraobserveragreement AT prinyatunksook classificationofcervicalvertebralmaturationstageswithmachinelearningmodelsleveragingdatasetswithhighinterandintraobserveragreement AT panraseeritthipravat classificationofcervicalvertebralmaturationstageswithmachinelearningmodelsleveragingdatasetswithhighinterandintraobserveragreement AT supatchaiboonpratham classificationofcervicalvertebralmaturationstageswithmachinelearningmodelsleveragingdatasetswithhighinterandintraobserveragreement AT yodhathaisatravaha classificationofcervicalvertebralmaturationstageswithmachinelearningmodelsleveragingdatasetswithhighinterandintraobserveragreement AT chaiyapolchaweewannakorn classificationofcervicalvertebralmaturationstageswithmachinelearningmodelsleveragingdatasetswithhighinterandintraobserveragreement AT supakitpeanchitlertkajorn classificationofcervicalvertebralmaturationstageswithmachinelearningmodelsleveragingdatasetswithhighinterandintraobserveragreement |