Comparative Analysis of Automated Machine Learning for Hyperparameter Optimization and Explainable Artificial Intelligence Models
Artificial intelligence (AI) has been increasingly applied to solve complex real-world problems. One of the most significant challenges in AI lies in selecting and fine-tuning the optimal algorithm for a given task. Automated Machine Learning (AutoML) models have emerged as a promising solution to a...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10982237/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849328547738419200 |
|---|---|
| author | Muhammad Salman Khan Tianbo Peng Hanzlah Akhlaq Muhammad Adeel Khan |
| author_facet | Muhammad Salman Khan Tianbo Peng Hanzlah Akhlaq Muhammad Adeel Khan |
| author_sort | Muhammad Salman Khan |
| collection | DOAJ |
| description | Artificial intelligence (AI) has been increasingly applied to solve complex real-world problems. One of the most significant challenges in AI lies in selecting and fine-tuning the optimal algorithm for a given task. Automated Machine Learning (AutoML) models have emerged as a promising solution to address this challenge by systematically exploring hyperparameter spaces to identify optimal configurations efficiently. This study addresses critical gaps in the current literature by conducting a comprehensive comparative analysis of AutoML frameworks for hyperparameter optimization and evaluating the effectiveness of various explainability techniques for enhancing model interpretability. For this purpose, Random forest (RF) is selected as the base model and integrated with nine different AutoML frameworks, namely Random search (RS), Grid search (GS), Hyperopt, TPOT, Optuna, GP Minimize, Forest Minimize, GBRT Minimize, and Dummy Minimize. The study focuses on predicting the ultimate moment capacity of Ultra-High-Performance Concrete (UHPC) beams and U-shaped girders. Furthermore, the insights from SHapley Additive exPlanations (SHAP) are also compared with those derived from alternative explainability methods, including Local interpretable model-agnostic explanations (LIME), partial dependence plots (PDP), and sklearn permutation importance rankings to examine the contributions of individual parameters to the ultimate moment capacity predictions of UHPC beams using the best-performing AutoML model. The findings demonstrate that Optuna consistently outperforms its counterparts, achieving the highest predictive accuracy and the lowest computational training time. The findings also highlight SHAP’s superiority in offering detailed, consistent, and actionable insights, making it the preferred method for both global feature importance and individual feature analysis in high-stakes engineering applications. |
| format | Article |
| id | doaj-art-e6d7040f43f44697a16a15b7054a3d85 |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-e6d7040f43f44697a16a15b7054a3d852025-08-20T03:47:33ZengIEEEIEEE Access2169-35362025-01-0113849668499110.1109/ACCESS.2025.356642710982237Comparative Analysis of Automated Machine Learning for Hyperparameter Optimization and Explainable Artificial Intelligence ModelsMuhammad Salman Khan0Tianbo Peng1https://orcid.org/0000-0003-4403-1135Hanzlah Akhlaq2Muhammad Adeel Khan3https://orcid.org/0009-0002-5988-718XDepartment of Bridge Engineering, College of Civil Engineering, Tongji University, Shanghai, ChinaDepartment of Bridge Engineering, College of Civil Engineering, Tongji University, Shanghai, ChinaDepartment of Bridge Engineering, College of Civil Engineering, Tongji University, Shanghai, ChinaDepartment of Bridge Engineering, College of Civil Engineering, Tongji University, Shanghai, ChinaArtificial intelligence (AI) has been increasingly applied to solve complex real-world problems. One of the most significant challenges in AI lies in selecting and fine-tuning the optimal algorithm for a given task. Automated Machine Learning (AutoML) models have emerged as a promising solution to address this challenge by systematically exploring hyperparameter spaces to identify optimal configurations efficiently. This study addresses critical gaps in the current literature by conducting a comprehensive comparative analysis of AutoML frameworks for hyperparameter optimization and evaluating the effectiveness of various explainability techniques for enhancing model interpretability. For this purpose, Random forest (RF) is selected as the base model and integrated with nine different AutoML frameworks, namely Random search (RS), Grid search (GS), Hyperopt, TPOT, Optuna, GP Minimize, Forest Minimize, GBRT Minimize, and Dummy Minimize. The study focuses on predicting the ultimate moment capacity of Ultra-High-Performance Concrete (UHPC) beams and U-shaped girders. Furthermore, the insights from SHapley Additive exPlanations (SHAP) are also compared with those derived from alternative explainability methods, including Local interpretable model-agnostic explanations (LIME), partial dependence plots (PDP), and sklearn permutation importance rankings to examine the contributions of individual parameters to the ultimate moment capacity predictions of UHPC beams using the best-performing AutoML model. The findings demonstrate that Optuna consistently outperforms its counterparts, achieving the highest predictive accuracy and the lowest computational training time. The findings also highlight SHAP’s superiority in offering detailed, consistent, and actionable insights, making it the preferred method for both global feature importance and individual feature analysis in high-stakes engineering applications.https://ieeexplore.ieee.org/document/10982237/Automated machine learningcomparative analysishyperparameter optimizationmoment capacityoptunarandom forest |
| spellingShingle | Muhammad Salman Khan Tianbo Peng Hanzlah Akhlaq Muhammad Adeel Khan Comparative Analysis of Automated Machine Learning for Hyperparameter Optimization and Explainable Artificial Intelligence Models IEEE Access Automated machine learning comparative analysis hyperparameter optimization moment capacity optuna random forest |
| title | Comparative Analysis of Automated Machine Learning for Hyperparameter Optimization and Explainable Artificial Intelligence Models |
| title_full | Comparative Analysis of Automated Machine Learning for Hyperparameter Optimization and Explainable Artificial Intelligence Models |
| title_fullStr | Comparative Analysis of Automated Machine Learning for Hyperparameter Optimization and Explainable Artificial Intelligence Models |
| title_full_unstemmed | Comparative Analysis of Automated Machine Learning for Hyperparameter Optimization and Explainable Artificial Intelligence Models |
| title_short | Comparative Analysis of Automated Machine Learning for Hyperparameter Optimization and Explainable Artificial Intelligence Models |
| title_sort | comparative analysis of automated machine learning for hyperparameter optimization and explainable artificial intelligence models |
| topic | Automated machine learning comparative analysis hyperparameter optimization moment capacity optuna random forest |
| url | https://ieeexplore.ieee.org/document/10982237/ |
| work_keys_str_mv | AT muhammadsalmankhan comparativeanalysisofautomatedmachinelearningforhyperparameteroptimizationandexplainableartificialintelligencemodels AT tianbopeng comparativeanalysisofautomatedmachinelearningforhyperparameteroptimizationandexplainableartificialintelligencemodels AT hanzlahakhlaq comparativeanalysisofautomatedmachinelearningforhyperparameteroptimizationandexplainableartificialintelligencemodels AT muhammadadeelkhan comparativeanalysisofautomatedmachinelearningforhyperparameteroptimizationandexplainableartificialintelligencemodels |