The 76th anniversary of the landing of Normandy was celebrated some days ago, a relevant historical event for the end of the Second World War and when meteorology played a crucial role. The landing took place exactly on June 6, 1944 due to the prior weather forecasts successfully conducted.
From its inception, meteorology has been progressing as other disciplines also advanced and, above all, accompanied by the technological progress of the 20th century, which allowed the development of mathematical models (thanks to the advance in computing) and the appearance or improvement of remote sensing instruments and devices such as satellites.
Therefore, it is unquestionable that in this new era, the fourth industrial revolution or digital revolution, meteorology cannot remain on the sidelines. In a similar way to the previous historical path it is imaginable that the current level of computational power and advances in artificial intelligence may transform the way we know the world of weather forecasting. Machine learning techniques (Machine Learning, ML), which feeds mainly on the historical data of the meteorological variables, may replace formal methods.
Is this a real possibility?
Currently used models, called numerical prediction models (NWP), are capable of solving the equations that govern the behaviour of the atmosphere, a complex system of nonlinear equations that require the use of certain approximations and physiographic parameters of the territory as realistic as possible. NWPs are currently essential for weather forecasting, providing useful and reliable information in many areas. However, the need to use initial conditions given by other models and the existence of parameters (necessary to include physical processes with spatial and temporal scales too small for the model to be able to explicitly represent them), among other factors, introduce cumulative errors in the results. In addition to this, this type of model involves a high computational cost, which consumes most of the energy of supercomputers, as it is the case of the Meteosim cluster, that performs simulations with the highest possible spatial and temporal resolution.
Currently, hundreds of millions of observations are available in a daily base from weather stations (surface and height), ships, planes, radars, etc. Likewise, the interconnected world (Internet of Things, IoT) in which we live has collaborated to increase this database. Imagine, for example, a mobile phone that records temperature and pressure. There is, therefore, an extensive network of records collected over the past decades.
Given the existence of this large database and in view of the computational cost and complexity involved in solving the atmospheric equations, can ML techniques learn from historical data, without any information on the dynamics of the atmosphere, and produce weather forecasts?
Machine learning is a subset of artificial intelligence that consists of creating algorithms that can modify themselves without human intervention to produce the desired result, when fed through a set of data. The main objective is to create systems that are capable of learning automatically, without the need to program, that is, without providing the instructions or steps to follow to obtain the forecast.
As with NWPs, MLs are also not perfect, and much of their success lies in the quantity and quality of the data used. Likewise, this necessary data does not always exist or is available in some areas, which would make impossible to create a ML model.
In recent years, weather forecast centres around the world have developed studies using ML techniques with a multitude of applications within the meteorological field. We highlight the models used for the post-processing of the results obtained with NWP, improvement of the parameterizations or the data used in the initialization of the numerical models. Of course, forecasts made only with ML have also been compared with those of NWP. If we leave out specific applications, can artificial intelligence replace the conventional forecasting system? The answer is that, generally speaking, not yet.
There are a multitude of scientific articles, workshops and congresses on this new paradigm, where in general terms it is shown that, depending on the purpose sought, machine learning models can provide an answer. Likewise, for some topics such as ensemble prediction or early detection of extreme events, the solutions that involve Deep Learning models also offer a likely perspective. However, there is as yet no clear agreement on whether such models can eventually replace conventional weather forecasting models in the near future. The truth is that there is a strong scientific community around the NWP, refined after decades of research and where the accuracy reached in the forecast is difficult to overcome, for the moment, by ML models. It only remains to demonstrate in a relatively short time that machine learning results can really be better than conventional tools. A positive advance in this line would transform the entire forecast system known up to now, achieving a new system that is faster, more accurate and with a greater prediction time window since the current numerical models only provide a reliable forecast for the following 5-6 days.
At Meteosim, we work to integrate new tools and technologies framed in Revolution 4.0, that together with traditional forecasting models, help us to improve the accuracy and reliability of our weather forecasts.