Beyond Ensemble Averages: Leveraging Climate Model Ensembles for Subseasonal Forecasting

2211.15856

YC

0

Reddit

0

Published 6/5/2024 by Elena Orlova, Haokun Liu, Raphael Rossellini, Benjamin A. Cash, Rebecca Willett

📈

Abstract

Producing high-quality forecasts of key climate variables, such as temperature and precipitation, on subseasonal time scales has long been a gap in operational forecasting. This study explores an application of machine learning (ML) models as post-processing tools for subseasonal forecasting. Lagged numerical ensemble forecasts (i.e., an ensemble where the members have different initialization dates) and observational data, including relative humidity, pressure at sea level, and geopotential height, are incorporated into various ML methods to predict monthly average precipitation and two-meter temperature two weeks in advance for the continental United States. For regression, quantile regression, and tercile classification tasks, we consider using linear models, random forests, convolutional neural networks, and stacked models (a multi-model approach based on the prediction of the individual ML models). Unlike previous ML approaches that often use ensemble mean alone, we leverage information embedded in the ensemble forecasts to enhance prediction accuracy. Additionally, we investigate extreme event predictions that are crucial for planning and mitigation efforts. Considering ensemble members as a collection of spatial forecasts, we explore different approaches to using spatial information. Trade-offs between different approaches may be mitigated with model stacking. Our proposed models outperform standard baselines such as climatological forecasts and ensemble means. In addition, we investigate feature importance, trade-offs between using the full ensemble or only the ensemble mean, and different modes of accounting for spatial variability.

Create account to get full access

or

If you already have an account, we'll log you in

Overview

  • This study explores using machine learning (ML) models to improve subseasonal forecasting of key climate variables like temperature and precipitation.
  • The researchers incorporate numerical ensemble forecasts and observational data into various ML methods to predict monthly average precipitation and temperature two weeks in advance for the continental United States.
  • The study considers different ML approaches, including regression, quantile regression, and tercile classification, and explores leveraging spatial information from ensemble members.
  • The proposed models outperform standard baselines like climatological forecasts and ensemble means, and the researchers investigate feature importance and trade-offs between approaches.

Plain English Explanation

Predicting the weather more than a week or two in advance can be challenging, but it's crucial for things like emergency planning and agriculture. This study looks at using machine learning models as a way to improve these "subseasonal" weather forecasts - forecasts that cover the period between weather and climate, usually 2 weeks to 2 months out.

The researchers took numerical weather forecast models, which use complex physics-based simulations, and combined them with machine learning. They fed the weather models' forecasts, along with observational data like humidity and air pressure, into different ML algorithms. The goal was to have the ML models learn patterns and make more accurate predictions of temperature and precipitation for the US, two weeks in advance.

Unlike previous approaches that just used the average of the weather models' forecasts, this study looked at using the entire ensemble of forecasts - the collection of individual forecasts from the different models. This gave the ML models more information to work with. The researchers also explored different ways of incorporating the spatial, geographic information from the ensemble forecasts.

Ultimately, the machine learning models were able to outperform the standard weather forecast baselines. The study provides insights into which features and approaches work best for improving subseasonal forecasting using ML techniques. This could lead to better advance warning of things like heatwaves, droughts, and other extreme weather events that are crucial for planning and preparation.

Technical Explanation

This study explores the application of machine learning (ML) models as post-processing tools for improving subseasonal forecasting of key climate variables like temperature and precipitation. The researchers incorporate numerical ensemble forecasts and observational data, including relative humidity, sea level pressure, and geopotential height, into various ML methods.

For regression, quantile regression, and tercile classification tasks, the team considers using linear models, random forests, convolutional neural networks, and stacked models (a multi-model approach combining the individual ML models). Unlike prior ML approaches that often use just the ensemble mean, this study leverages the information embedded in the full ensemble of forecasts to enhance prediction accuracy.

Additionally, the researchers investigate techniques for incorporating spatial information from the ensemble members, as the ensemble can be viewed as a collection of spatial forecasts. They explore different approaches to using this spatial data and find that trade-offs between the approaches may be mitigated through model stacking.

The proposed ML models are shown to outperform standard baselines like climatological forecasts and ensemble means. The study also provides insights into feature importance, the trade-offs between using the full ensemble versus just the ensemble mean, and the various modes of accounting for spatial variability.

Critical Analysis

The researchers acknowledge several caveats and areas for further research in their paper. For example, they note that their study is limited to the continental United States and that the performance of the ML models may vary in other regions. Additionally, the paper does not delve into the interpretability or explainability of the ML models, which could be an important consideration for operational use.

One potential concern is the reliance on numerical ensemble forecasts, which themselves have limitations and biases. It would be valuable to explore the robustness of the ML models when faced with uncertainties or errors in the underlying weather forecasts. Incorporating physics-based constraints could help address this issue.

Furthermore, the study focuses on monthly average temperature and precipitation, but decision-makers may be more interested in accurate predictions of extreme weather events. The researchers do touch on this, but further work is needed to develop ML models that reliably forecast the likelihood and magnitude of rare, high-impact weather occurrences.

Overall, this study represents a promising step towards leveraging machine learning to improve subseasonal climate forecasting. However, continued research and development will be necessary to make these techniques operationally robust and widely applicable.

Conclusion

This study explores the use of machine learning models as a way to enhance subseasonal forecasting of key climate variables like temperature and precipitation. By incorporating numerical ensemble forecasts and observational data into a variety of ML approaches, the researchers were able to outperform standard baseline forecasts.

The findings suggest that leveraging the full ensemble of weather model forecasts, rather than just the ensemble mean, can improve prediction accuracy. Additionally, the study provides insights into the trade-offs and best practices for incorporating spatial information from the ensemble members.

While this research represents an important step forward, there are still several areas for further development, such as improving model interpretability, exploring robustness to uncertainties in the underlying weather forecasts, and expanding the focus to include extreme weather event predictions.

Overall, the successful application of machine learning to subseasonal forecasting demonstrated in this study holds promising implications for enhancing climate preparedness and decision-making in a wide range of sectors, from emergency management to agriculture.



This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!

Related Papers

FuXi-ENS: A machine learning model for medium-range ensemble weather forecasting

FuXi-ENS: A machine learning model for medium-range ensemble weather forecasting

Xiaohui Zhong, Lei Chen, Hao Li, Jie Feng, Bo Lu

YC

0

Reddit

0

Ensemble weather forecasting is essential for weather predictions and mitigating the impacts of extreme weather events. Constructing an ensemble prediction system (EPS) based on conventional numerical weather prediction (NWP) models is highly computationally expensive. Machine learning (ML) models have emerged as valuable tools for deterministic weather forecasts, providing forecasts with significantly reduced computational requirements and even surpassing the forecast performance of traditional NWP models. However, challenges arise when applying ML models to ensemble forecasting. Recent ML models, such as GenCast and SEEDS model, rely on the ERA5 Ensemble of Data Assimilations (EDA) or two operational NWP ensemble members for forecast generation. The spatial resolution of 1{deg} or 2{deg} in these models is often considered too coarse for many applications. To overcome these limitations, we introduce FuXi-ENS, an advanced ML model designed to deliver 6-hourly global ensemble weather forecasts up to 15 days. This model runs at a significantly improved spatial resolution of 0.25{deg}, incorporating 5 upper-air atmospheric variables at 13 pressure levels, along with 13 surface variables. By leveraging the inherent probabilistic nature of Variational AutoEncoder (VAE), FuXi-ENS optimizes a loss function that combines the continuous ranked probability score (CRPS) and the KL divergence between the predicted and target distribution. This innovative approach represents an advancement over the traditional use of L1 loss combined with the KL loss in standard VAE models when VAE for ensemble weather forecasts. Evaluation results demonstrate that FuXi-ENS outperforms ensemble forecasts from the European Centre for Medium-Range Weather Forecasts (ECMWF), a world leading NWP model, on 98.1% of 360 variable and forecast lead time combinations on CRPS.

Read more

5/10/2024

📊

Data Assimilation with Machine Learning Surrogate Models: A Case Study with FourCastNet

Melissa Adrian, Daniel Sanz-Alonso, Rebecca Willett

YC

0

Reddit

0

Modern data-driven surrogate models for weather forecasting provide accurate short-term predictions but inaccurate and nonphysical long-term forecasts. This paper investigates online weather prediction using machine learning surrogates supplemented with partial and noisy observations. We empirically demonstrate and theoretically justify that, despite the long-time instability of the surrogates and the sparsity of the observations, filtering estimates can remain accurate in the long-time horizon. As a case study, we integrate FourCastNet, a state-of-the-art weather surrogate model, within a variational data assimilation framework using partial, noisy ERA5 data. Our results show that filtering estimates remain accurate over a year-long assimilation window and provide effective initial conditions for forecasting tasks, including extreme event prediction.

Read more

5/24/2024

🌿

Site-specific Deterministic Temperature and Humidity Forecasts with Explainable and Reliable Machine Learning

MengMeng Han, Tennessee Leeuwenburg, Brad Murphy

YC

0

Reddit

0

Site-specific weather forecasts are essential to accurate prediction of power demand and are consequently of great interest to energy operators. However, weather forecasts from current numerical weather prediction (NWP) models lack the fine-scale detail to capture all important characteristics of localised real-world sites. Instead they provide weather information representing a rectangular gridbox (usually kilometres in size). Even after post-processing and bias correction, area-averaged information is usually not optimal for specific sites. Prior work on site optimised forecasts has focused on linear methods, weighted consensus averaging, time-series methods, and others. Recent developments in machine learning (ML) have prompted increasing interest in applying ML as a novel approach towards this problem. In this study, we investigate the feasibility of optimising forecasts at sites by adopting the popular machine learning model gradient boosting decision tree, supported by the Python version of the XGBoost package. Regression trees have been trained with historical NWP and site observations as training data, aimed at predicting temperature and dew point at multiple site locations across Australia. We developed a working ML framework, named 'Multi-SiteBoost' and initial testing results show a significant improvement compared with gridded values from bias-corrected NWP models. The improvement from XGBoost is found to be comparable with non-ML methods reported in literature. With the insights provided by SHapley Additive exPlanations (SHAP), this study also tests various approaches to understand the ML predictions and increase the reliability of the forecasts generated by ML.

Read more

4/5/2024

Exploring the Potential of Hybrid Machine-Learning/Physics-Based Modeling for Atmospheric/Oceanic Prediction Beyond the Medium Range

Exploring the Potential of Hybrid Machine-Learning/Physics-Based Modeling for Atmospheric/Oceanic Prediction Beyond the Medium Range

Dhruvit Patel, Troy Arcomano, Brian Hunt, Istvan Szunyogh, Edward Ott

YC

0

Reddit

0

This paper explores the potential of a hybrid modeling approach that combines machine learning (ML) with conventional physics-based modeling for weather prediction beyond the medium range. It extends the work of Arcomano et al. (2022), which tested the approach for short- and medium-range weather prediction, and the work of Arcomano et al. (2023), which investigated its potential for climate modeling. The hybrid model used for the forecast experiments of the paper is based on the low-resolution, simplified parameterization atmospheric general circulation model (AGCM) SPEEDY. In addition to the hybridized prognostic variables of SPEEDY, the current version of the model has three purely ML-based prognostic variables. One of these is 6~h cumulative precipitation, another is the sea surface temperature, while the third is the heat content of the top 300 m deep layer of the ocean. The model has skill in predicting the El Ni~no cycle and its global teleconnections with precipitation for 3-7 months depending on the season. The model captures equatorial variability of the precipitation associated with Kelvin and Rossby waves and MJO. Predictions of the precipitation in the equatorial region have skill for 15 days in the East Pacific and 11.5 days in the West Pacific. Though the model has low spatial resolution, for these tasks it has prediction skill comparable to what has been published for high-resolution, purely physics-based, conventional operational forecast models.

Read more

5/31/2024