On Robustness of the Explanatory Power of Machine Learning Models: Insights From a New Explainable AI Approach Using Sensitivity Analysis
Abstract Machine learning (ML) is increasingly considered the solution to environmental problems where limited or no physico‐chemical process understanding exists. But in supporting high‐stakes decisions, where the ability to explain possible solutions is key to their acceptability and legitimacy, M...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2025-03-01
|
| Series: | Water Resources Research |
| Subjects: | |
| Online Access: | https://doi.org/10.1029/2024WR037398 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849422693701517312 |
|---|---|
| author | Banamali Panigrahi Saman Razavi Lorne E. Doig Blanchard Cordell Hoshin V. Gupta Karsten Liber |
| author_facet | Banamali Panigrahi Saman Razavi Lorne E. Doig Blanchard Cordell Hoshin V. Gupta Karsten Liber |
| author_sort | Banamali Panigrahi |
| collection | DOAJ |
| description | Abstract Machine learning (ML) is increasingly considered the solution to environmental problems where limited or no physico‐chemical process understanding exists. But in supporting high‐stakes decisions, where the ability to explain possible solutions is key to their acceptability and legitimacy, ML can fall short. Here, we develop a method, rooted in formal sensitivity analysis, to uncover the primary drivers behind ML predictions. Unlike many methods for explainable artificial intelligence (XAI), this method (a) accounts for complex multi‐variate distributional properties of data, common in environmental systems, (b) offers a global assessment of the input‐output response surface formed by ML, rather than focusing solely on local regions around existing data points, and (c) is scalable and data‐size independent, ensuring computational efficiency with large data sets. We apply this method to a suite of ML models predicting various water quality variables in a pilot‐scale experimental pit lake. A critical finding is that subtle alterations in the design of some ML models (such as variations in random seed, functional class, hyperparameters, or data splitting) can lead to different interpretations of how outputs depend on inputs. Further, models from different ML families (decision trees, connectionists, or kernels) may focus on different aspects of the information provided by data, despite displaying similar predictive power. Overall, our results underscore the need to assess the explanatory robustness of ML models and advocate for using model ensembles to gain deeper insights into system drivers and improve prediction reliability. |
| format | Article |
| id | doaj-art-7fd72afd5aa94499a7853b07acf751d9 |
| institution | Kabale University |
| issn | 0043-1397 1944-7973 |
| language | English |
| publishDate | 2025-03-01 |
| publisher | Wiley |
| record_format | Article |
| series | Water Resources Research |
| spelling | doaj-art-7fd72afd5aa94499a7853b07acf751d92025-08-20T03:30:57ZengWileyWater Resources Research0043-13971944-79732025-03-01613n/an/a10.1029/2024WR037398On Robustness of the Explanatory Power of Machine Learning Models: Insights From a New Explainable AI Approach Using Sensitivity AnalysisBanamali Panigrahi0Saman Razavi1Lorne E. Doig2Blanchard Cordell3Hoshin V. Gupta4Karsten Liber5Toxicology Centre University of Saskatchewan Saskatoon SK CanadaSchool of Environment and Sustainability University of Saskatchewan Saskatoon SK CanadaToxicology Centre University of Saskatchewan Saskatoon SK CanadaGlobal Institute for Water Security School of Environmental and Sustainability University of Saskatchewan Saskatoon SK CanadaDepartment of Hydrology and Atmospheric Sciences The University of Arizona Tucson AZ USAToxicology Centre University of Saskatchewan Saskatoon SK CanadaAbstract Machine learning (ML) is increasingly considered the solution to environmental problems where limited or no physico‐chemical process understanding exists. But in supporting high‐stakes decisions, where the ability to explain possible solutions is key to their acceptability and legitimacy, ML can fall short. Here, we develop a method, rooted in formal sensitivity analysis, to uncover the primary drivers behind ML predictions. Unlike many methods for explainable artificial intelligence (XAI), this method (a) accounts for complex multi‐variate distributional properties of data, common in environmental systems, (b) offers a global assessment of the input‐output response surface formed by ML, rather than focusing solely on local regions around existing data points, and (c) is scalable and data‐size independent, ensuring computational efficiency with large data sets. We apply this method to a suite of ML models predicting various water quality variables in a pilot‐scale experimental pit lake. A critical finding is that subtle alterations in the design of some ML models (such as variations in random seed, functional class, hyperparameters, or data splitting) can lead to different interpretations of how outputs depend on inputs. Further, models from different ML families (decision trees, connectionists, or kernels) may focus on different aspects of the information provided by data, despite displaying similar predictive power. Overall, our results underscore the need to assess the explanatory robustness of ML models and advocate for using model ensembles to gain deeper insights into system drivers and improve prediction reliability.https://doi.org/10.1029/2024WR037398machine learningsensitivity analysisfeature selectionmodel randomnesswater quality |
| spellingShingle | Banamali Panigrahi Saman Razavi Lorne E. Doig Blanchard Cordell Hoshin V. Gupta Karsten Liber On Robustness of the Explanatory Power of Machine Learning Models: Insights From a New Explainable AI Approach Using Sensitivity Analysis Water Resources Research machine learning sensitivity analysis feature selection model randomness water quality |
| title | On Robustness of the Explanatory Power of Machine Learning Models: Insights From a New Explainable AI Approach Using Sensitivity Analysis |
| title_full | On Robustness of the Explanatory Power of Machine Learning Models: Insights From a New Explainable AI Approach Using Sensitivity Analysis |
| title_fullStr | On Robustness of the Explanatory Power of Machine Learning Models: Insights From a New Explainable AI Approach Using Sensitivity Analysis |
| title_full_unstemmed | On Robustness of the Explanatory Power of Machine Learning Models: Insights From a New Explainable AI Approach Using Sensitivity Analysis |
| title_short | On Robustness of the Explanatory Power of Machine Learning Models: Insights From a New Explainable AI Approach Using Sensitivity Analysis |
| title_sort | on robustness of the explanatory power of machine learning models insights from a new explainable ai approach using sensitivity analysis |
| topic | machine learning sensitivity analysis feature selection model randomness water quality |
| url | https://doi.org/10.1029/2024WR037398 |
| work_keys_str_mv | AT banamalipanigrahi onrobustnessoftheexplanatorypowerofmachinelearningmodelsinsightsfromanewexplainableaiapproachusingsensitivityanalysis AT samanrazavi onrobustnessoftheexplanatorypowerofmachinelearningmodelsinsightsfromanewexplainableaiapproachusingsensitivityanalysis AT lorneedoig onrobustnessoftheexplanatorypowerofmachinelearningmodelsinsightsfromanewexplainableaiapproachusingsensitivityanalysis AT blanchardcordell onrobustnessoftheexplanatorypowerofmachinelearningmodelsinsightsfromanewexplainableaiapproachusingsensitivityanalysis AT hoshinvgupta onrobustnessoftheexplanatorypowerofmachinelearningmodelsinsightsfromanewexplainableaiapproachusingsensitivityanalysis AT karstenliber onrobustnessoftheexplanatorypowerofmachinelearningmodelsinsightsfromanewexplainableaiapproachusingsensitivityanalysis |