We’re a few weeks into the new year – which also happens to represent a new frontier for weather forecasting in this country.
For more on why this is so significant, we need to first understand how weather models work in their current form before this artificial intelligence revolution. Enter Jacob Carley, chief of the engineering and implementation branch of the Environmental Modeling Center of the National Weather Service.
He says traditional, numerical weather models essentially run the complicated math and physics equations that explain how our atmosphere works.
“What these models do is chunk everything up over the globe and run forward in time and you simulate all of these complicated processes. They are fairly costly to run and to develop; you need a wide range of expertise and a lot of different scientists working on these systems that understand cloud processes and all sorts of different things.”
Carley adds that the current system comes with a significant computational load – in addition to a lengthy amount of time to get the results into the hands of forecasters and decision makers.
“They can take tens of thousands of processors to run a forecast. And for our global models, they take about two and a half hours to run a single forecast, and we run those four times a day.”
Scientists and meteorologists began toying with the idea that artificial intelligence may be able to streamline that process in 2022. Since then, both NVIDIA and Google have introduced their own AI models into the fray.
NOAA began to consider its own path into the AI frontier in 2023 and created a special team for that purpose the following year – with Carley helping spearhead that effort.
“Together across the NOAA laboratories and centers, we all collectively began experimenting with realtime systems just to see what we could learn, how we could apply this and how we could make it better. This past spring, we launched the EAGLE project – which stands for Experimental AI Global and Limited Area Ensemble,” Carley says. “For weather weenies, that sounds great, that sounds fun. For most folks, that might be a bit of a head scratcher of an acronym. But what that really means is it’s a project to help coordinate both internally and externally on advancing our AI weather prediction capabilities.”
"When we combine the two – the AI-based ensemble, which is 31 realizations of the atmosphere at any given time, with the physical system, which is also 31, we get 62 members together. And what that gives us is a better representation of the potential uncertainty in a forecast."
Because of that collaboration, the AI products were able to get off the ground quickly – with the team deciding last summer that their work was ready for prime time.
“And so, those applications then became what’s known as the Artificial Intelligence Global Forecast System, or AIGFS, the Artificial Intelligence Global Ensemble Forecast System, or AIGEFS, and then the hybrid, physical Global Ensemble Forecast System with the AI-based system to make a grand, super ensemble,” Carley says.
And a quick note – an ensemble forecast is essentially just a range of potential possibilities that could occur with weather systems shown in the modeling – giving forecasters a swath of potential outcomes, which helps shape their final predictions.
NOAA says their AI models show significant improvements over the traditional, numerical models – especially for that hybrid ensemble model.
“When we combine the two – the AI-based ensemble, which is 31 realizations of the atmosphere at any given time, with the physical system, which is also 31, we get 62 members together. And what that gives us is a better representation of the potential uncertainty in a forecast,” explains Carley. “So, when forecasters are looking at how they want to communicate the uncertainty with the forecast over the next three, five, six days – they’ll have more information to give them a greater sense of confidence in that forecast.”
Here are the specific breakdowns from NOAA on each of the new models:
- AIGFS: the AI-backed version of the Global Forecast System shows “improved forecast skill over the traditional GFS for many large-scale features.” The model also uses a fraction of the computing resources of the traditional version and can produce a forecast in 40 minutes.
- AIGEFS: this ensemble version has forecast skill that is “comparable to the operational GEFS” and only requires 9% of the computing resources in comparison to that traditional model.
- HGEFS: this version is what NOAA is most excited about, and the organization believes it is the first in the world to implement such a hybrid physical-AI ensemble system. The HGEFS “consistently outperforms both the GEFS and the AIGEFS across most major verification metrics.”
“What’s exciting about that is that it uses a lot less processors than what the current, state-of-the-art physical models do. In effect, I’m going to use some computer nomenclature, at its peak 200 nodes for a given time [are needed] to run our physical model forecast for the globe. To do the AI forecast for just one member, it takes one to two nodes.”
Carley says the AI models will also get even faster in the near future – with plans to transition them over to GPUs instead of CPUs already underway (for the non-computer nerds, that just means the systems are being transferred over to computer hardware that they were originally designed to run on, further increasing their efficiency).
I asked Carley about the new models’ performance when it comes to mountainous regions. In Western Virginia, for example, the topography plays a huge role in how weather systems behave when they move through. Accurately capturing this phenomenon on a consistent basis is something that traditional, numerical weather models have struggled with.
Carley said the new models are coarse in their resolution – essentially meaning that the topography issue is still present. But there is an upside…
“What this will provide you, however, is say you have a nor’easter that is forecast. It will give the forecaster, ideally, a better prediction of the track of that nor’easter such that when the forecaster is examining all of the data that they have available to them, they can then say, ‘Ok, the AIGFS has a better forecast. I can use my knowledge of the local area, in addition to some of the higher resolution physical models, to help fill in the gaps that are left by that somewhat coarser resolution AI model.”
With these faster, more efficient models, I asked Carley if NOAA plans to run them more frequently than the current four times per day. He said there aren’t plans right now to do that, but things are in flux and that could change in the near future. In the meantime, a faster model run means forecasters will have more time with the data – which will be extremely beneficial to them and decision makers, especially ahead of significant events like hurricanes and winter storms.
Thanks for checking out this edition of CommonWx — the weather and climate newsletter from Radio IQ. Use this link to get the newsletter sent to your inbox.
