The weather variable of wind plays a key role in weather and energy research. The increased energy consumption and ongoing climate change lead to a growing interest in renewable energies. The solar and wind energy sectors have been growing rapidly. Wind speed data are essential for finding suitable locations for wind turbine locations.
Additionally, extreme weather events are expected to become more frequent in the future. Companies such as insurers or agricultural producers depend on reliable data on wind conditions. Moreover, understanding wind dynamics helps in deciphering the influence of wind on other weather variables such as temperature or precipitation, and the general comprehension of the atmospheric system.
As a precise wind forecast is key for many of our customers (from wind energy companies to outdoor enthusiasts), it is essential for us that our customers can understand and interpret our forecast and its performance. Several verification studies have been conducted and are outlined in the following sections. In these, multiple numerical weather models were compared.
Analysis of global wind forecasts
The following findings are based on the master's thesis by Brigitte Häuser in 2021. Several wind speed weather simulation models such as ICON, MFGLOBAL, GFS05, models from the NEMS family (NEMS4 and NEMS12), as well as a reanalysis model ERA5 were compared on a global scale. For validation, over 5000 hourly METAR measurements were used from locations around the world, and throughout the entire year 2020.
Several statistical error metrics and forecast performance metrics were calculated and used for comparison. Further spatial analyses indicate the model performance of weather forecast models on a regional basis. Finally, a MultiModel approach was tested against other approaches in order to investigate their respective accuracy.
Raw model comparison
For further performance analysis, the POD (Probability of Detection), FAR (False Alarm Ratio) and HSS (Heidke Skill Score) were calculated. The thresholds 5, 15, 20 and 30 m/s were set to evaluate the performance of higher wind speed predictions. The main findings are summarised in the following table.
As for POD, wind speed over 5 m/s was most accurately predicted by NEMSGLOBAL, followed by GFS05, ERA5, ICON and MFGLOBAL. Nevertheless, when considering its HSS and high FAR values, the high POD values of NEMSGLOBAL are likely to be caused by a general overprediction of the wind speed, rather than a real prediction skill.
The MAE of ERA5 is spatially distributed equally over the entire world. However, NEMSGLOBAL has the highest MAE values in Canada and Northern Europe, although the distribution of the MAE is comparable to the one of ERA5, only with higher contrasts. Both maps show a lower performance on wind speed forecast on islands, and (particularly in the map of NEMSGLOBAL) in mountainous regions such as the Rocky Mountains. In general, certain models demonstrate stronger spatial patterns, while others less so. For more detailed information please view the master's thesis (2,5 MB).
Within the MultiModel approach, several raw models are combined and weighted differently, which usually leads to improvements in the forecast's accuracy. As demonstrated in the previous section, raw models tend to perform with different spatial results. The combination of the forecast of multiple models for a specific location can lead to so-called “error cancellation”. For example, two models (one overestimating the variable and the other underestimating it) may balance each other out, resulting in a lower forecast error. In this section, the weather forecast models GFS05, MFGLOBAL, NEMSGLOBAL and ICON were combined to optimise the MAE, and also validated against METAR stations.
The study shows that ICON has the highest impact in this approach globally, as it was weighted the highest for almost all locations, except for locations in Russia and Northern Canada. Due to its high weighting, GFS05 has a major influence in the MultiModel approach as well. Averaging the best performing MultiModels of all stations (meaning all station-specific MultiModel combinations with the lowest MAE), we obtain an optimised MultiModel combination, in which ICON is weighted 50 - 60%, GFS05 20 - 30 % and MFGLOBAL and NEMSGLOBAL 10 %. This MultiModel was, in the final step, verified against all METAR stations.
|Domain||MAE||MBE||RMSE||MAPE||Cor||Cor >3 m/s|
|MM GLobal Weight||1.369||-0.122||1.768||0.401||0.703||0.598|
To classify our operational wind speed forecast, we compared it to the performance of the raw models, and to other weather forecast providers. The following figure shows the MBE of different prediction models (NEMSGLOBAL, ICON, GFS05, MFGLOBAL and UMGLOBAL), the reanalysis model ERA5 and the meteoblue forecast. The model outputs were compared to hourly measurements of more than 450 METAR stations worldwide. The analysis is based on 24-hour forecast and hourly measured wind speed data of the year 2021.
meteoblue forecast vs. raw models
The figure above shows that for most of the models the MBE is negative, therefore the models underestimate the windspeed, whereas the highest peak for the meteoblue forecast is close to zero. This finding shows that the meteoblue forecast outperforms the reanalysis model ERA5.
meteoblue forecast vs. other weather data providers
|Provider||MAE [m/s]||MBE [m/s]|