Proper scoring rules for multivariate probabilistic forecasts based on aggregation and transformation
<p>Proper scoring rules are an essential tool to assess the predictive performance of probabilistic forecasts. However, propriety alone does not ensure an informative characterization of predictive performance, and it is recommended to compare forecasts using multiple scoring rules. With that...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Copernicus Publications
2025-03-01
|
| Series: | Advances in Statistical Climatology, Meteorology and Oceanography |
| Online Access: | https://ascmo.copernicus.org/articles/11/23/2025/ascmo-11-23-2025.pdf |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850053751972298752 |
|---|---|
| author | R. Pic C. Dombry P. Naveau M. Taillardat |
| author_facet | R. Pic C. Dombry P. Naveau M. Taillardat |
| author_sort | R. Pic |
| collection | DOAJ |
| description | <p>Proper scoring rules are an essential tool to assess the predictive performance of probabilistic forecasts. However, propriety alone does not ensure an informative characterization of predictive performance, and it is recommended to compare forecasts using multiple scoring rules. With that in mind, interpretable scoring rules providing complementary information are necessary. We formalize a framework based on aggregation and transformation to build interpretable multivariate proper scoring rules. Aggregation-and-transformation-based scoring rules can target application-specific features of probabilistic forecasts, which improves the characterization of the predictive performance. This framework is illustrated through examples taken from the weather forecasting literature, and numerical experiments are used to showcase its benefits in a controlled setting. Additionally, the framework is tested on real-world data of postprocessed wind speed forecasts over central Europe. In particular, we show that it can help bridge the gap between proper scoring rules and spatial verification tools.</p> |
| format | Article |
| id | doaj-art-439b3123d78946dd97ce64d8d6abf690 |
| institution | DOAJ |
| issn | 2364-3579 2364-3587 |
| language | English |
| publishDate | 2025-03-01 |
| publisher | Copernicus Publications |
| record_format | Article |
| series | Advances in Statistical Climatology, Meteorology and Oceanography |
| spelling | doaj-art-439b3123d78946dd97ce64d8d6abf6902025-08-20T02:52:27ZengCopernicus PublicationsAdvances in Statistical Climatology, Meteorology and Oceanography2364-35792364-35872025-03-0111235810.5194/ascmo-11-23-2025Proper scoring rules for multivariate probabilistic forecasts based on aggregation and transformationR. Pic0C. Dombry1P. Naveau2M. Taillardat3Université Marie et Louis Pasteur, CNRS, LmB (UMR 6623), 25000 Besançon, FranceUniversité Marie et Louis Pasteur, CNRS, LmB (UMR 6623), 25000 Besançon, FranceLaboratoire des Sciences du Climat et de l'Environnement, UMR 8212, CEA-CNRS-UVSQ, EstimR, IPSL & U Paris-Saclay, Gif-sur-Yvette, FranceCNRM, Université de Toulouse, Météo-France, CNRS, Toulouse, France<p>Proper scoring rules are an essential tool to assess the predictive performance of probabilistic forecasts. However, propriety alone does not ensure an informative characterization of predictive performance, and it is recommended to compare forecasts using multiple scoring rules. With that in mind, interpretable scoring rules providing complementary information are necessary. We formalize a framework based on aggregation and transformation to build interpretable multivariate proper scoring rules. Aggregation-and-transformation-based scoring rules can target application-specific features of probabilistic forecasts, which improves the characterization of the predictive performance. This framework is illustrated through examples taken from the weather forecasting literature, and numerical experiments are used to showcase its benefits in a controlled setting. Additionally, the framework is tested on real-world data of postprocessed wind speed forecasts over central Europe. In particular, we show that it can help bridge the gap between proper scoring rules and spatial verification tools.</p>https://ascmo.copernicus.org/articles/11/23/2025/ascmo-11-23-2025.pdf |
| spellingShingle | R. Pic C. Dombry P. Naveau M. Taillardat Proper scoring rules for multivariate probabilistic forecasts based on aggregation and transformation Advances in Statistical Climatology, Meteorology and Oceanography |
| title | Proper scoring rules for multivariate probabilistic forecasts based on aggregation and transformation |
| title_full | Proper scoring rules for multivariate probabilistic forecasts based on aggregation and transformation |
| title_fullStr | Proper scoring rules for multivariate probabilistic forecasts based on aggregation and transformation |
| title_full_unstemmed | Proper scoring rules for multivariate probabilistic forecasts based on aggregation and transformation |
| title_short | Proper scoring rules for multivariate probabilistic forecasts based on aggregation and transformation |
| title_sort | proper scoring rules for multivariate probabilistic forecasts based on aggregation and transformation |
| url | https://ascmo.copernicus.org/articles/11/23/2025/ascmo-11-23-2025.pdf |
| work_keys_str_mv | AT rpic properscoringrulesformultivariateprobabilisticforecastsbasedonaggregationandtransformation AT cdombry properscoringrulesformultivariateprobabilisticforecastsbasedonaggregationandtransformation AT pnaveau properscoringrulesformultivariateprobabilisticforecastsbasedonaggregationandtransformation AT mtaillardat properscoringrulesformultivariateprobabilisticforecastsbasedonaggregationandtransformation |