Information‐Theoretic Scores for Bayesian Model Selection and Similarity Analysis: Concept and Application to a Groundwater Problem
Abstract Bayesian model selection (BMS) and Bayesian model justifiability analysis (BMJ) provide a statistically rigorous framework for comparing competing models through the use of Bayesian model evidence (BME). However, a BME‐based analysis has two main limitations: (a) it does not account for a m...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2023-07-01
|
| Series: | Water Resources Research |
| Subjects: | |
| Online Access: | https://doi.org/10.1029/2022WR033711 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849706669699760128 |
|---|---|
| author | Maria Fernanda Morales Oreamuno Sergey Oladyshkin Wolfgang Nowak |
| author_facet | Maria Fernanda Morales Oreamuno Sergey Oladyshkin Wolfgang Nowak |
| author_sort | Maria Fernanda Morales Oreamuno |
| collection | DOAJ |
| description | Abstract Bayesian model selection (BMS) and Bayesian model justifiability analysis (BMJ) provide a statistically rigorous framework for comparing competing models through the use of Bayesian model evidence (BME). However, a BME‐based analysis has two main limitations: (a) it does not account for a model's posterior predictive performance after using the data for calibration and (b) it leads to biased results when comparing models that use different subsets of the observations for calibration. To address these limitations, we propose augmenting BMS and BMJ analyses with additional information‐theoretic measures: expected log‐predictive density (ELPD), relative entropy (RE) and information entropy (IE). Exploring the connection between Bayesian inference and information theory, we explicitly link BME and ELPD together with RE and IE to highlight the information flow in BMS and BMJ analyses. We show how to compute and interpret these scores alongside BME, and apply the framework to a controlled 2D groundwater setup featuring five models, one of which uses a subset of the data for calibration. Our results show how the information‐theoretic scores complement BME by providing a more complete picture concerning the Bayesian updating process. Additionally, we demonstrate how both RE and IE can be used to objectively compare models that feature different data sets for calibration. Overall, the introduced Bayesian information‐theoretic framework can lead to a better‐informed decision by incorporating a model's post‐calibration predictive performance, by allowing to work with different subsets of the data and by considering the usefulness of the data in the Bayesian updating process. |
| format | Article |
| id | doaj-art-813899107a944056a296c7dc2dbf4da6 |
| institution | DOAJ |
| issn | 0043-1397 1944-7973 |
| language | English |
| publishDate | 2023-07-01 |
| publisher | Wiley |
| record_format | Article |
| series | Water Resources Research |
| spelling | doaj-art-813899107a944056a296c7dc2dbf4da62025-08-20T03:16:07ZengWileyWater Resources Research0043-13971944-79732023-07-01597n/an/a10.1029/2022WR033711Information‐Theoretic Scores for Bayesian Model Selection and Similarity Analysis: Concept and Application to a Groundwater ProblemMaria Fernanda Morales Oreamuno0Sergey Oladyshkin1Wolfgang Nowak2Department of Stochastic Simulation and Safety Research for Hydrosystems Institute for Modelling Hydraulic and Environmental Systems University of Stuttgart Stuttgart GermanyDepartment of Stochastic Simulation and Safety Research for Hydrosystems Institute for Modelling Hydraulic and Environmental Systems University of Stuttgart Stuttgart GermanyDepartment of Stochastic Simulation and Safety Research for Hydrosystems Institute for Modelling Hydraulic and Environmental Systems University of Stuttgart Stuttgart GermanyAbstract Bayesian model selection (BMS) and Bayesian model justifiability analysis (BMJ) provide a statistically rigorous framework for comparing competing models through the use of Bayesian model evidence (BME). However, a BME‐based analysis has two main limitations: (a) it does not account for a model's posterior predictive performance after using the data for calibration and (b) it leads to biased results when comparing models that use different subsets of the observations for calibration. To address these limitations, we propose augmenting BMS and BMJ analyses with additional information‐theoretic measures: expected log‐predictive density (ELPD), relative entropy (RE) and information entropy (IE). Exploring the connection between Bayesian inference and information theory, we explicitly link BME and ELPD together with RE and IE to highlight the information flow in BMS and BMJ analyses. We show how to compute and interpret these scores alongside BME, and apply the framework to a controlled 2D groundwater setup featuring five models, one of which uses a subset of the data for calibration. Our results show how the information‐theoretic scores complement BME by providing a more complete picture concerning the Bayesian updating process. Additionally, we demonstrate how both RE and IE can be used to objectively compare models that feature different data sets for calibration. Overall, the introduced Bayesian information‐theoretic framework can lead to a better‐informed decision by incorporating a model's post‐calibration predictive performance, by allowing to work with different subsets of the data and by considering the usefulness of the data in the Bayesian updating process.https://doi.org/10.1029/2022WR033711Bayesian model selectionBayesian model evidenceinformation theoryrelative entropyBayesian model similarity |
| spellingShingle | Maria Fernanda Morales Oreamuno Sergey Oladyshkin Wolfgang Nowak Information‐Theoretic Scores for Bayesian Model Selection and Similarity Analysis: Concept and Application to a Groundwater Problem Water Resources Research Bayesian model selection Bayesian model evidence information theory relative entropy Bayesian model similarity |
| title | Information‐Theoretic Scores for Bayesian Model Selection and Similarity Analysis: Concept and Application to a Groundwater Problem |
| title_full | Information‐Theoretic Scores for Bayesian Model Selection and Similarity Analysis: Concept and Application to a Groundwater Problem |
| title_fullStr | Information‐Theoretic Scores for Bayesian Model Selection and Similarity Analysis: Concept and Application to a Groundwater Problem |
| title_full_unstemmed | Information‐Theoretic Scores for Bayesian Model Selection and Similarity Analysis: Concept and Application to a Groundwater Problem |
| title_short | Information‐Theoretic Scores for Bayesian Model Selection and Similarity Analysis: Concept and Application to a Groundwater Problem |
| title_sort | information theoretic scores for bayesian model selection and similarity analysis concept and application to a groundwater problem |
| topic | Bayesian model selection Bayesian model evidence information theory relative entropy Bayesian model similarity |
| url | https://doi.org/10.1029/2022WR033711 |
| work_keys_str_mv | AT mariafernandamoralesoreamuno informationtheoreticscoresforbayesianmodelselectionandsimilarityanalysisconceptandapplicationtoagroundwaterproblem AT sergeyoladyshkin informationtheoreticscoresforbayesianmodelselectionandsimilarityanalysisconceptandapplicationtoagroundwaterproblem AT wolfgangnowak informationtheoreticscoresforbayesianmodelselectionandsimilarityanalysisconceptandapplicationtoagroundwaterproblem |