Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Predicting weather forecast uncertainty with machine learning
Stockholm University, Faculty of Science, Department of Meteorology .
Stockholm University, Faculty of Science, Department of Meteorology .
Number of Authors: 22018 (English)In: Quarterly Journal of the Royal Meteorological Society, ISSN 0035-9009, E-ISSN 1477-870X, Vol. 144, no 717, p. 2830-2841Article in journal (Refereed) Published
Abstract [en]

Weather forecasts are inherently uncertain. Therefore, for many applications forecasts are only considered valuable if an uncertainty estimate can be assigned to them. Currently, the best method to provide a confidence estimate for individual forecasts is to produce an ensemble of numerical weather simulations, which is computationally very expensive. Here, we assess whether machine learning techniques can provide an alternative approach to predict the uncertainty of a weather forecast given the large-scale atmospheric state at initialization. We propose a method based on deep learning with artificial convolutional neural networks that is trained on past weather forecasts. Given a new weather situation, it assigns a scalar value of confidence to medium-range forecasts initialized from the said atmospheric state, indicating whether the predictability is higher or lower than usual for the time of the year. While our method has a lower skill than ensemble weather forecast models in predicting forecast uncertainty, it is computationally very efficient and outperforms a range of alternative methods that do not involve performing numerical forecasts. This shows that it is possible to use machine learning in order to estimate future forecast uncertainty from past forecasts. The main constraint in the performance of our method seems to be the number of past forecasts available for training the machine learning algorithm.

Place, publisher, year, edition, pages
2018. Vol. 144, no 717, p. 2830-2841
Keywords [en]
ensembles, machine learning, statistical methods, weather forecasts
National Category
Earth and Related Environmental Sciences
Research subject
Atmospheric Sciences and Oceanography
Identifiers
URN: urn:nbn:se:su:diva-165737DOI: 10.1002/qj.3410ISI: 000455586500029OAI: oai:DiVA.org:su-165737DiVA, id: diva2:1286236
Available from: 2019-02-06 Created: 2019-02-06 Last updated: 2020-04-27Bibliographically approved
In thesis
1. Artificial intelligence in weather and climate prediction: Learning atmospheric dynamics
Open this publication in new window or tab >>Artificial intelligence in weather and climate prediction: Learning atmospheric dynamics
2020 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Weather and climate prediction is dominated by high dimensionality, interactions on many different spatial and temporal scales and chaotic dynamics. This makes many problems in the field quite complex ones, and also state-of-the-art numerical models are - despite their immense computational costs - not sufficient for many applications. Therefore, it is appealing to use emerging new technologies such as artificial intelligence to tackle these problems.

We show that it is possible to use deep neural networks to emulate the full dynamics of a strongly simplified general circulation model, providing both good forecasts of the model state several days ahead as well as stable long-term climate timeseries. This method partly also works on more complex and realistic models, but only for forecasting the model's weather several days ahead, not for creating climate runs. It is sufficient to use 50-100 years of data for training the networks. The same neural network method can be combined with singular value decomposition from numerical ensemble weather forecasting in order to generate probabilistic ensemble forecasts with the neural networks.

On a more fundamental level, we show that in a simple dynamical systems setting there seem to be limitations in the ability of feed-forward neural networks to generalize to new regions of the system. This is caused by different parts of the network learning to model different parts of the system. Contradictory, for another simple dynamical system this is shown not to be an issue, raising doubts on the usefulness of results from simple models in the context of more complex ones. Additionally, we show that neural networks are to some extent able to “learn” the influence of slowly changing external forcings on the dynamics of the system, but only given broad enough forcing regimes.

Finally, we present a method to complement operational weather forecasts. Given the initial fields and the error of past weather forecasts, a neural network is used to predict the uncertainty in new forecasts, given only the initial field of the new forecast.

Place, publisher, year, edition, pages
Stockholm: Department of Meteorology, Stockholm University, 2020. p. 30
National Category
Meteorology and Atmospheric Sciences
Research subject
Atmospheric Sciences and Oceanography
Identifiers
urn:nbn:se:su:diva-180877 (URN)978-91-7911-128-1 (ISBN)978-91-7911-129-8 (ISBN)
Public defence
2020-06-12, Vivi Täckholmsalen (Q-salen), Svante Arrhenius väg 20, Stockholm, 10:00 (English)
Opponent
Supervisors
Funder
Swedish Research Council, 2016-03724
Note

At the time of the doctoral defense, the following paper was unpublished and had a status as follows: Paper 4: Manuscript.

Available from: 2020-05-18 Created: 2020-04-20 Last updated: 2020-05-26Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full text

Search in DiVA

By author/editor
Scher, SebastianMessori, Gabriele
By organisation
Department of Meteorology
In the same journal
Quarterly Journal of the Royal Meteorological Society
Earth and Related Environmental Sciences

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 92 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf