Weather forecasting has become essential in modern life, reducing weather-related losses and improving societal outcomes. Severe weather alerts provide vital early warnings that help to protect life and property. And forecasts of temperatures, precipitation, wind, humidity, and other conditions—both extreme and average—support public safety, health, and economic prosperity by giving everyone from farmers and fishers to energy and construction companies a heads-up on expected weather.
However, not all forecasts are created equal, in part because weather prediction is chaotic, meaning small uncertainties in the initial conditions (data) input into weather models can lead to vastly different predicted outcomes. The accuracy of predictions is also affected by the complexity of models, the realism with which atmospheric conditions are represented, how far into the future weather is being forecast, and—at very resolved scales—local geography.
The application of novel artificial intelligence (AI) is providing the latest revolutionary influence on weather forecasting.
The skill and reliability of weather forecasts have steadily improved over the past century. In recent decades, improvements have been facilitated by advances in numerical weather prediction (NWP), growth in computing power, and the availability of more and better datasets that capture Earth’s physical conditions more frequently. The application of novel artificial intelligence (AI) is providing the latest revolutionary influence on forecasting. This revolution is borne out by trends in the scientific literature and in the development of new AI-based tools with the potential to enhance predictions of conditions hours, days, or weeks in advance.
Making the Models
All weather forecasts involve inputting data in the form of observations—readings from weather balloons, buoys, satellites, and other instruments—into models that predict future states of the atmosphere. Model outputs are then transformed into useful products such as daily weather forecasts, storm warnings, and fire hazard assessments.
Current forecasting methods are based on NWP, a mathematical framework that models the future of the atmosphere by treating it as a fluid that interacts with water bodies, land, and the biosphere. Models using this approach include the European Centre for Medium-Range Weather Forecasts’ (ECMWF) Integrated Forecasting System (IFS) model (widely considered the gold standard in modern weather forecasting), the National Center for Atmospheric Research’s Weather Research and Forecasting model, and NOAA’s Global Forecasting System.
NWP models solve fluid dynamics equations known as the Navier-Stokes equations that simplify the complex motions of fluids, such as air in the atmosphere, and can be used to describe relationships among their velocities, temperatures, pressures, and densities. The result is a set of predictions of what, for example, temperatures will be at given places at some point in the future. These predictions, together with estimates of other simplified physical processes not captured by fluid dynamics equations, make up a weather forecast.
This conceptually simple description obscures the massive scale of the work that goes into creating forecasts (Figure 1). Operating satellites, radar networks, and other necessary technology is expensive and requires substantial specialized expertise. Inputting observations from these disparate sources into models and getting them to work together harmoniously—no easy task—are a field of study unto themselves.
Furthermore, forecast models are complicated and require some of the most powerful—not to mention expensive and energy-intensive—supercomputers in the world to function. Expert meteorologists are required to interpret model outputs, and communications teams are needed to translate those interpretations for the public.
The input-model-output structure for forecasting will be familiar to students of computer science. Indeed, the two fields have, in many ways, grown up together. The Navier-Stokes approach to weather forecasting first became truly useful when computing technology could produce results sufficiently quickly beginning in the 1950s and 1960s—after all, there is no point in having a forecast for 24 hours from now if it takes 36 hours to make!
The Rise of Machine Learning
As the power of computing hardware and software has increased, so too have the accuracy, resolution, and range of forecasting. The advent of early AI systems in the 1950s, which weather services adopted almost immediately, fed this advancement through the mid-20th century. These early AIs were hierarchical systems that mimicked human decisionmaking through decision trees comprising a series of “if this, then that” logic rules.
Interest in AI among forecasters picked up starting in the late 1990s and grew steadily into the 2000s and 2010s as computing resources became more powerful and useful data became more widely available.
The development of decision trees was followed by the emergence of machine learning (ML), a subdiscipline of AI involving training models to perform specific tasks without explicit programming. Instead of following coded instructions, these models learn from patterns in datasets to improve their performance over time. One method to achieve this improvement is to train a neural network, an algorithm said to be inspired by the human brain. Neural networks work by iteratively processing numerical representations of input data—image pixel brightnesses, temperatures, or wind speeds, for example—through multiple layers of mathematical operations to reorganize and refine the data until a meaningful output is obtained.
Even though experiments with ML have been ongoing within the wider scientific community since the 1970s, they initially failed to catch on as much more than a novelty in weather forecasting. AI systems at the time were limited by the computing power and relevant data available for use in ML. However, interest in AI among forecasters picked up starting in the late 1990s and grew steadily into the 2000s and 2010s as computing resources became more powerful and useful data became more widely available.
Model training methods also grew more efficient, and new ideas on how to adapt the original neural network concept created opportunities to tackle more complicated tasks. For example, 2010 saw the release of ImageNet, a huge database of labeled images that could be used to train AIs for 2D image recognition tasks.
Machine Learning Moves into Weather Forecasting
Weather forecasting is feeling the impact of this innovation. The growth of AI in research on nowcasting—forecasts of conditions a couple of hours in advance—and short-range weather forecasting up to a day or two out helps to reveal how.
We informally surveyed studies published between 2011 and 2022 using the Web of Science database and found that most of this research focused on applying AI to studies of classical weather forecast variables: precipitation, clouds, solar irradiation, wind speed and direction, and temperature (Figure 2).
The annual growth of new publications related to these five forecast variables indicates startling year-over-year growth averaging 375% over this period. This nearly fivefold annual increase is split about evenly across each variable: In 2010, the numbers of new publications addressing each of these variables were in the low single digits; by 2022, the numbers for each were in the hundreds.
Research in just a few nations drove most of this growth. Roughly half the papers published from 2011 to 2022 emerged from China (27.5%) and the United States (22.7%). India (~8%), Germany (~6.5%), and the United Kingdom and Australia (~5% each) also contributed significantly. Most, if not all, of this research output appears to be linked to interest in its relevance for or application to various economic sectors traditionally tied to weather forecasting, such as energy, transportation, and agriculture.
We determined links in the published studies by associating keywords from these sectors with the five forecast variables (Figure 3). This approach has limitations, including potential double counting of studies (e.g., because the same AI model may have multiple uses), not accounting for the relative sizes of the sectors (e.g., larger sectors like energy are naturally bigger motivators for research than smaller ones like fisheries), and not identifying proprietary research and models not released to the public. Nonetheless, the keyword associations reveal interesting trends.
For example, applications in the energy sector dominate AI forecasting research related to solar irradiance and wind. Comprehensive reviews have covered how AI technologies are being integrated into the energy industry at many stages in the supply chain. Choosing and planning sites (e.g., for solar or wind farms), management of solar and wind resources in day-to-day operations, predictive maintenance, energy demand matching in real time, and management of home and business consumers’ energy usage are all use cases in which AI is affecting the industry and driving research.
Applications in the agricultural sector are primarily driving research into temperature and precipitation forecasting.
Meanwhile, applications in the agricultural sector are primarily driving research into temperature and precipitation forecasting. This trend likely reflects the wider movement in the sector toward precision agriculture, a data-driven approach intended to boost crop yields and sustainability. Large companies, such as BASF, have promoted “digital farming,” which combines data sources, including forecasts and historical weather patterns, into ML models to predict future temperatures and precipitation. Farmers can then use these predictions to streamline operations and optimize resource usage through decisions about, for example, the best time to fertilize or water crops.
The construction industry, a significant driver of temperature forecasting research using AI, relies on temperature forecasts to plan operations. Weather can substantially influence project durations by affecting start dates and the time required for tasks such as pouring concrete. Accurate forecasts can also improve planning for worker breaks on hot days and for anticipating work stoppages during hard freezes.
In the transportation and aviation sectors, public safety concerns are likely driving AI-aided forecasting research. Intelligent transportation systems rely on weather forecast data to predict and mitigate road transportation problems through diversions or road and bridge closures. Similarly, accurate weather data can power aviation models to improve safety and comfort by, for example, predicting issues such as turbulence and icing.
Evolving Architectures
The methods and structures, or architectures, used in AI-based forecasting research have changed and grown more sophisticated as the field has advanced, particularly over the past decade (Figure 4). And this trajectory toward improvement appears to be accelerating.
In 2015, roughly 40% of AI models in the literature for nowcasting and short-range weather forecasting were support vector machines, but by 2022, this figure declined to just 8%. Over the same period, the use of more sophisticated convolutional neural networks ballooned from 11% to 43%. Newer architectures have also emerged for forecasting, with generative adversarial networks, U-Net, and transformer models gaining popularity.
Transformers, with their powerful attention mechanisms that detect long-range dependences among different variables (e.g., among atmospheric conditions and the formation of storms), may be on a course to become the preferred architecture for weather forecasting. Transformers have been widely adopted in other domains and have become synonymous with AI in general because of their prominent use in generative AI tools like OpenAI’s ChatGPT.
Some of today’s most advanced weather forecasting models make use of transformer models, rather than being based on numerical weather prediction.
Some of today’s most advanced weather forecasting models make use of transformer models, such as those from NVIDIA (FourCastNet), Huawei (Pangu-Weather), and Google (GraphCast), each of which is data driven, rather than being based on NWP. These models boast levels of accuracy and spatial resolution similar to ECMWF’s traditional IFS model across several important weather variables. However, their major innovation is in the computing resources required to generate a forecast: On the basis of (albeit imperfect) comparisons, NVIDIA estimates, for example, that FourCastNet may be up to 45,000 times faster than IFS, which equates to using 12,000 times less energy.
A View of the Future
Combining high-resolution data from multiple sources will be core to the weather forecasting revolution, meaning the observational approaches used to gather these data will play a central role.
Sophisticated AI architectures are already being used to combine observations from different sources to create new products that are difficult to create using traditional, physics-based methods. For example, advanced air quality forecasting tools rely on combining measurements from satellites and monitoring stations and ground-level traffic and topography data to produce realistic representations of pollutant concentrations. AIs are also being used for data assimilation, the process of mapping observations to regularly spaced, gridded representations of the atmosphere for use in weather forecast models (which themselves can be AI driven).
Another growing use case for AI is forecasting extreme weather. Extreme events can be challenging for AI models to predict because many models function by searching for patterns (i.e., averages) in data, meaning rarer events are inherently weighted less. Researchers have suggested that the most state-of-the-art AI weather forecasts have significantly underperformed traditional NWP counterparts in predicting extreme weather events, especially rare events such as category 5 hurricanes. However, improvements are in the works. For example, compared with traditional methods, Microsoft’s Aurora model boasts improved accuracy for Pacific typhoon tracks and wind speeds during European storms.
Whether scientists are using fully data-driven AI or so-called hybrid systems, which combine AI and traditional atmospheric physics models, predictions of weather events and of likely outcomes of those events (e.g., fires, floods, evacuations) need to be combined reliably and transparently. One example of a hybrid system blending physics and AI elements is Google’s Flood Hub, which integrates traditional modeling with AI tools to deliver early extreme flood warnings freely in 80 countries. Such work is an important part of the United Nations’ Early Warnings for All initiative, which aims to ensure that all people have actionable access to warnings and information about natural hazards.
Observations will play a key role in facilitating the growing accuracy and efficiency of new forecasting products by providing the data needed to train AI models.
Observations will play a key role in facilitating the growing accuracy and efficiency of new forecasting products by providing the data needed to train AI models. Today, models are generally pre-trained before use with a structured dataset generated from data assimilation methods. These pre-trained systems could be tailored to new specialized tasks at high resolution, such as short-range forecasting for locations where conditions change rapidly, like in high mountain ranges.
Satellites, such as the recently launched Geostationary Operational Environmental Satellite 19 (GOES-19) and NOAA-21 missions, have been an increasingly critical source of data for training AI. These data will soon be supplemented with even higher-resolution observations from next-generation satellite instruments such as the European Organisation for the Exploitation of Meteorological Satellites’ (EUMETSAT) recently launched Meteosat Third Generation (MTG) and EUMETSAT Polar System – Second Generation (EPS-SG) programs. NOAA’s planned Geostationary Extended Observations (GeoXO) and Near Earth Orbit Network (NEON) programs will further boost both traditional and AI modeling.
Looking farther ahead, some experiments have attempted to fully replace traditional data assimilation systems, moving directly from observations to gridded forecast model inputs. A natural end point could be a fully automated, end-to-end weather forecast system, potentially with multiple models working together in sequence. Such a system would process observations into inputs for forecast models, then run those models and process forecast outputs into useful products.
The effects of the AI revolution are beginning to be felt across society, including in key sectors of the economy such as energy, agriculture, and transportation. For weather forecasting, AI technology has the potential to streamline observational data processing, use computational resources more efficiently, improve forecast accuracy and range, and even create entirely new products. Ultimately, current technologies and coming innovations may save money and help better protect lives by seamlessly delivering faster and more useful predictions of future conditions.
Author Information
Justin Shenolikar ([email protected]), European Organisation for the Exploitation of Meteorological Satellites, Darmstadt, Germany; now at Universität Heidelberg, Germany; and Paolo Ruti and Chris Yoon Sang Chung, European Organisation for the Exploitation of Meteorological Satellites, Darmstadt, Germany