Monitoring and benchmarking Earth system model simulations with ESMValTool v2.12.0
<p>Earth system models (ESMs) are important tools to improve our understanding of present-day climate and to project climate change under different plausible future scenarios. Thus, ESMs are continuously improved and extended, resulting in more complex models. Particularly during the model dev...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Copernicus Publications
2025-02-01
|
| Series: | Geoscientific Model Development |
| Online Access: | https://gmd.copernicus.org/articles/18/1169/2025/gmd-18-1169-2025.pdf |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850229053494132736 |
|---|---|
| author | A. Lauer L. Bock B. Hassler P. Jöckel L. Ruhe M. Schlund |
| author_facet | A. Lauer L. Bock B. Hassler P. Jöckel L. Ruhe M. Schlund |
| author_sort | A. Lauer |
| collection | DOAJ |
| description | <p>Earth system models (ESMs) are important tools to improve our understanding of present-day climate and to project climate change under different plausible future scenarios. Thus, ESMs are continuously improved and extended, resulting in more complex models. Particularly during the model development phase, it is important to continuously monitor how well the historical climate is reproduced and to systematically analyze, evaluate, understand, and document possible shortcomings. Hence, putting model biases relative to observations or, for example, a well-characterized pre-industrial control run, into the context of deviations shown by other state-of-the-art models greatly helps to assess which biases need to be addressed with higher priority. Here, we introduce the new capability of the open-source community-developed Earth System Model Evaluation Tool (ESMValTool) to monitor running simulations or benchmark existing simulations with observations in the context of results from the Coupled Model Intercomparison Project (CMIP). To benchmark model output, ESMValTool calculates metrics such as the root-mean-square error, the Pearson correlation coefficient, or the earth mover's distance relative to reference datasets. This is directly compared to the same metric calculated for an ensemble of models such as the one provided by Phase 6 of the CMIP (CMIP6), which provides a statistical measure for the range of values that can be considered typical of state-of-the-art ESMs. Results are displayed in different types of plots, such as map plots or time series, with different techniques such as stippling (maps) or shading (time series) used to visualize the typical range of values for a given metric from the model ensemble used for comparison. While the examples shown here focus on atmospheric variables, the new functionality can be applied to any other ESM component such as land, ocean, sea ice, or land ice. Automatic downloading of CMIP results from the Earth System Grid Federation (ESGF) makes application of ESMValTool for benchmarking of individual model simulations, for example, in preparation of Phase 7 of the CMIP (CMIP7), easy and very user-friendly.</p> |
| format | Article |
| id | doaj-art-1e4490c5fa754a22829d7433a66888cd |
| institution | OA Journals |
| issn | 1991-959X 1991-9603 |
| language | English |
| publishDate | 2025-02-01 |
| publisher | Copernicus Publications |
| record_format | Article |
| series | Geoscientific Model Development |
| spelling | doaj-art-1e4490c5fa754a22829d7433a66888cd2025-08-20T02:04:21ZengCopernicus PublicationsGeoscientific Model Development1991-959X1991-96032025-02-01181169118810.5194/gmd-18-1169-2025Monitoring and benchmarking Earth system model simulations with ESMValTool v2.12.0A. Lauer0L. Bock1B. Hassler2P. Jöckel3L. Ruhe4M. Schlund5Deutsches Zentrum für Luft- und Raumfahrt (DLR), Institut für Physik der Atmosphäre, Oberpfaffenhofen, GermanyDeutsches Zentrum für Luft- und Raumfahrt (DLR), Institut für Physik der Atmosphäre, Oberpfaffenhofen, GermanyDeutsches Zentrum für Luft- und Raumfahrt (DLR), Institut für Physik der Atmosphäre, Oberpfaffenhofen, GermanyDeutsches Zentrum für Luft- und Raumfahrt (DLR), Institut für Physik der Atmosphäre, Oberpfaffenhofen, GermanyInstitute of Environmental Physics (IUP), University of Bremen, Bremen, GermanyDeutsches Zentrum für Luft- und Raumfahrt (DLR), Institut für Physik der Atmosphäre, Oberpfaffenhofen, Germany<p>Earth system models (ESMs) are important tools to improve our understanding of present-day climate and to project climate change under different plausible future scenarios. Thus, ESMs are continuously improved and extended, resulting in more complex models. Particularly during the model development phase, it is important to continuously monitor how well the historical climate is reproduced and to systematically analyze, evaluate, understand, and document possible shortcomings. Hence, putting model biases relative to observations or, for example, a well-characterized pre-industrial control run, into the context of deviations shown by other state-of-the-art models greatly helps to assess which biases need to be addressed with higher priority. Here, we introduce the new capability of the open-source community-developed Earth System Model Evaluation Tool (ESMValTool) to monitor running simulations or benchmark existing simulations with observations in the context of results from the Coupled Model Intercomparison Project (CMIP). To benchmark model output, ESMValTool calculates metrics such as the root-mean-square error, the Pearson correlation coefficient, or the earth mover's distance relative to reference datasets. This is directly compared to the same metric calculated for an ensemble of models such as the one provided by Phase 6 of the CMIP (CMIP6), which provides a statistical measure for the range of values that can be considered typical of state-of-the-art ESMs. Results are displayed in different types of plots, such as map plots or time series, with different techniques such as stippling (maps) or shading (time series) used to visualize the typical range of values for a given metric from the model ensemble used for comparison. While the examples shown here focus on atmospheric variables, the new functionality can be applied to any other ESM component such as land, ocean, sea ice, or land ice. Automatic downloading of CMIP results from the Earth System Grid Federation (ESGF) makes application of ESMValTool for benchmarking of individual model simulations, for example, in preparation of Phase 7 of the CMIP (CMIP7), easy and very user-friendly.</p>https://gmd.copernicus.org/articles/18/1169/2025/gmd-18-1169-2025.pdf |
| spellingShingle | A. Lauer L. Bock B. Hassler P. Jöckel L. Ruhe M. Schlund Monitoring and benchmarking Earth system model simulations with ESMValTool v2.12.0 Geoscientific Model Development |
| title | Monitoring and benchmarking Earth system model simulations with ESMValTool v2.12.0 |
| title_full | Monitoring and benchmarking Earth system model simulations with ESMValTool v2.12.0 |
| title_fullStr | Monitoring and benchmarking Earth system model simulations with ESMValTool v2.12.0 |
| title_full_unstemmed | Monitoring and benchmarking Earth system model simulations with ESMValTool v2.12.0 |
| title_short | Monitoring and benchmarking Earth system model simulations with ESMValTool v2.12.0 |
| title_sort | monitoring and benchmarking earth system model simulations with esmvaltool v2 12 0 |
| url | https://gmd.copernicus.org/articles/18/1169/2025/gmd-18-1169-2025.pdf |
| work_keys_str_mv | AT alauer monitoringandbenchmarkingearthsystemmodelsimulationswithesmvaltoolv2120 AT lbock monitoringandbenchmarkingearthsystemmodelsimulationswithesmvaltoolv2120 AT bhassler monitoringandbenchmarkingearthsystemmodelsimulationswithesmvaltoolv2120 AT pjockel monitoringandbenchmarkingearthsystemmodelsimulationswithesmvaltoolv2120 AT lruhe monitoringandbenchmarkingearthsystemmodelsimulationswithesmvaltoolv2120 AT mschlund monitoringandbenchmarkingearthsystemmodelsimulationswithesmvaltoolv2120 |