Robustness of AI-based weather forecasts in a changing climate
0
🤿
Sign in to get full access
Overview
- Data-driven machine learning models have made significant progress in weather forecasting, outperforming physics-based models.
- This raises the question of whether machine learning can also revolutionize climate science, such as informing climate change mitigation and adaptation.
- The paper shows that state-of-the-art machine learning models trained for weather forecasting can produce skillful forecasts across different climate states, including pre-industrial, present-day, and future warmer climates.
- This suggests the dynamics shaping weather on short timescales may not fundamentally differ in a changing climate, and demonstrates the out-of-distribution generalization capabilities of these models.
- However, two models show a global-mean cold bias in forecasts for the future warmer climate, drifting towards the colder present-day climate they were trained on.
- The paper discusses possible remedies for these biases and analyzes their spatial distribution, revealing complex warming and cooling patterns.
Plain English Explanation
Machine learning models have become very good at weather forecasting, outperforming traditional physics-based models in many cases. This raises the exciting possibility that these data-driven models could also be used to study and understand climate change.
The researchers in this paper tested whether these advanced weather forecasting models could still make accurate predictions in different climate conditions, such as pre-industrial times, today's climate, and a future warmer climate. They found that the models were generally able to produce skillful forecasts across these different climate states.
This suggests that the fundamental processes driving weather patterns may not change too much, even as the overall climate warms up. It also shows that these machine learning models have the capability to generalize beyond the specific conditions they were trained on - a key requirement for using them in climate science.
However, the researchers also found some limitations. Two of the models they tested showed a tendency to forecast cooler temperatures in the future warmer climate, drifting back towards the colder present-day climate they were originally trained on. The paper explores possible reasons for this bias and how it varies geographically.
Despite these current issues, the overall results suggest that data-driven machine learning models could become powerful new tools for climate science, complementing the traditional physics-based approaches and potentially transforming the field.
Technical Explanation
The paper evaluates the ability of state-of-the-art data-driven machine learning models trained for weather forecasting to generate skillful forecasts across different climate states, including pre-industrial, present-day, and future warmer climates.
The researchers used three leading machine learning weather forecasting models and tested their performance on climate simulation data representing these varying conditions. They assessed the models' skill across a range of standard meteorological metrics.
The results show that the machine learning models are able to produce skillful forecasts across the different climate states, suggesting the fundamental weather dynamics may not change drastically even as the climate warms. This demonstrates the out-of-distribution generalization capabilities of these models, which is a critical requirement for applying them to climate science.
However, two of the models exhibited a global-mean cold bias in their forecasts for the future warmer climate state, drifting back towards the colder present-day climate they were originally trained on. The paper explores potential reasons for this, such as missing information about ocean, sea ice, and land surface changes in the training data.
The spatial distribution of these biases is also analyzed, revealing complex patterns of regional warming and cooling. The researchers discuss potential remedies, such as incorporating more diverse climate data into the training process.
Overall, the findings indicate that advanced data-driven machine learning models hold substantial promise for transforming climate science, but also highlight the need to further develop these techniques to fully harness their potential.
Critical Analysis
The paper presents an intriguing and promising step towards leveraging the power of machine learning for climate science. The ability of the tested models to produce skillful forecasts across different climate states is a significant result, suggesting these techniques may be able to complement traditional physics-based climate models.
However, the cold bias exhibited by two of the models in the future warmer climate scenario is a clear limitation that requires further investigation and remedies. The authors rightly point out that this could be due to the training data lacking sufficient information about important factors like ocean, sea ice, and land surface changes. Addressing these biases will be crucial for ensuring the reliability of machine learning models in climate applications.
Additionally, the paper only examines a small number of models and climate scenarios. Expanding the research to a wider range of state-of-the-art machine learning architectures and a broader set of climate conditions would help establish the generalizability of the findings.
It would also be valuable to see more detailed analysis of the models' performance, such as evaluating their ability to capture important climate phenomena like monsoons, El Niño-Southern Oscillation, and tropical cyclones. Understanding the models' strengths and weaknesses in these key areas would provide valuable insights.
Overall, the paper demonstrates exciting potential for machine learning in climate science, but also highlights the need for continued research and development to fully realize this potential. A critical, evidence-based approach will be essential as the field progresses.
Conclusion
This paper presents a significant step forward in exploring the use of data-driven machine learning models for climate science. The researchers show that state-of-the-art weather forecasting models can produce skillful forecasts across a range of climate conditions, including future warmer states.
This suggests the fundamental weather dynamics may not change as dramatically as the climate warms, and that these machine learning techniques have the potential to generalize beyond their training data - a key requirement for climate applications.
However, the paper also identifies limitations, such as the tendency of some models to drift towards colder forecasts in the future warmer climate. Addressing these biases and further developing the capabilities of machine learning for climate science will be important next steps.
Overall, the findings indicate that data-driven approaches could become powerful complementary tools for climate research, potentially transforming established practices in the field. Continued advancements in this area could lead to significant improvements in our understanding and forecasting of climate change and its impacts.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
🤿
0
Robustness of AI-based weather forecasts in a changing climate
Thomas Rackow, Nikolay Koldunov, Christian Lessig, Irina Sandu, Mihai Alexe, Matthew Chantry, Mariana Clare, Jesper Dramsch, Florian Pappenberger, Xabier Pedruzo-Bagazgoitia, Steffen Tietsche, Thomas Jung
Data-driven machine learning models for weather forecasting have made transformational progress in the last 1-2 years, with state-of-the-art ones now outperforming the best physics-based models for a wide range of skill scores. Given the strong links between weather and climate modelling, this raises the question whether machine learning models could also revolutionize climate science, for example by informing mitigation and adaptation to climate change or to generate larger ensembles for more robust uncertainty estimates. Here, we show that current state-of-the-art machine learning models trained for weather forecasting in present-day climate produce skillful forecasts across different climate states corresponding to pre-industrial, present-day, and future 2.9K warmer climates. This indicates that the dynamics shaping the weather on short timescales may not differ fundamentally in a changing climate. It also demonstrates out-of-distribution generalization capabilities of the machine learning models that are a critical prerequisite for climate applications. Nonetheless, two of the models show a global-mean cold bias in the forecasts for the future warmer climate state, i.e. they drift towards the colder present-day climate they have been trained for. A similar result is obtained for the pre-industrial case where two out of three models show a warming. We discuss possible remedies for these biases and analyze their spatial distribution, revealing complex warming and cooling patterns that are partly related to missing ocean-sea ice and land surface information in the training data. Despite these current limitations, our results suggest that data-driven machine learning models will provide powerful tools for climate science and transform established approaches by complementing conventional physics-based models.
Read more9/30/2024
👨🏫
0
Transferring climate change knowledge
Francesco Immorlano, Veronika Eyring, Thomas le Monnier de Gouville, Gabriele Accarino, Donatello Elia, Giovanni Aloisio, Pierre Gentine
Accurate and precise climate projections are required for climate adaptation and mitigation, but Earth system models still exhibit great uncertainties. Several approaches have been developed to reduce the spread of climate projections and feedbacks, yet those methods cannot capture the non-linear complexity inherent in the climate system. Using a Transfer Learning approach, we show that Machine Learning can be used to optimally leverage and merge the knowledge gained from Earth system models simulations and historical observations to more accurately project global surface air temperature fields in the 21st century. We reach an uncertainty reduction of more than 50% with respect to state-of-the-art approaches. We give evidence that our novel method provides narrower projection uncertainty together with more accurate mean climate projections, urgently required for climate adaptation.
Read more6/21/2024
0
Data driven weather forecasts trained and initialised directly from observations
Anthony McNally, Christian Lessig, Peter Lean, Eulalie Boucher, Mihai Alexe, Ewan Pinnington, Matthew Chantry, Simon Lang, Chris Burrows, Marcin Chrust, Florian Pinault, Ethel Villeneuve, Niels Bormann, Sean Healy
Skilful Machine Learned weather forecasts have challenged our approach to numerical weather prediction, demonstrating competitive performance compared to traditional physics-based approaches. Data-driven systems have been trained to forecast future weather by learning from long historical records of past weather such as the ECMWF ERA5. These datasets have been made freely available to the wider research community, including the commercial sector, which has been a major factor in the rapid rise of ML forecast systems and the levels of accuracy they have achieved. However, historical reanalyses used for training and real-time analyses used for initial conditions are produced by data assimilation, an optimal blending of observations with a physics-based forecast model. As such, many ML forecast systems have an implicit and unquantified dependence on the physics-based models they seek to challenge. Here we propose a new approach, training a neural network to predict future weather purely from historical observations with no dependence on reanalyses. We use raw observations to initialise a model of the atmosphere (in observation space) learned directly from the observations themselves. Forecasts of crucial weather parameters (such as surface temperature and wind) are obtained by predicting weather parameter observations (e.g. SYNOP surface data) at future times and arbitrary locations. We present preliminary results on forecasting observations 12-hours into the future. These already demonstrate successful learning of time evolutions of the physical processes captured in real observations. We argue that this new approach, by staying purely in observation space, avoids many of the challenges of traditional data assimilation, can exploit a wider range of observations and is readily expanded to simultaneous forecasting of the full Earth system (atmosphere, land, ocean and composition).
Read more7/23/2024
0
End-to-end data-driven weather forecasting
Anna Vaughan, Stratis Markou, Will Tebbutt, James Requeima, Wessel P. Bruinsma, Tom R. Andersson, Michael Herzog, Nicholas D. Lane, Matthew Chantry, J. Scott Hosking, Richard E. Turner
Weather forecasting is critical for a range of human activities including transportation, agriculture, industry, as well as the safety of the general public. Machine learning models have the potential to transform the complex weather prediction pipeline, but current approaches still rely on numerical weather prediction (NWP) systems, limiting forecast speed and accuracy. Here we demonstrate that a machine learning model can replace the entire operational NWP pipeline. Aardvark Weather, an end-to-end data-driven weather prediction system, ingests raw observations and outputs global gridded forecasts and local station forecasts. Further, it can be optimised end-to-end to maximise performance over quantities of interest. Global forecasts outperform an operational NWP baseline for multiple variables and lead times. Local station forecasts are skillful up to ten days lead time and achieve comparable and often lower errors than a post-processed global NWP baseline and a state-of-the-art end-to-end forecasting system with input from human forecasters. These forecasts are produced with a remarkably simple neural process model using just 8% of the input data and three orders of magnitude less compute than existing NWP and hybrid AI-NWP methods. We anticipate that Aardvark Weather will be the starting point for a new generation of end-to-end machine learning models for medium-range forecasting that will reduce computational costs by orders of magnitude and enable the rapid and cheap creation of bespoke models for users in a variety of fields, including for the developing world where state-of-the-art local models are not currently available.
Read more7/16/2024