[Reading Science] AI Weather Observation, the 'David' Korea Meteorological Administration Facing the 'Goliath' Big Tech

Weather Forecasting in Existing Hydrological Prediction Models Changes with AI Amid Climate Change
Big Tech Giants Nvidia, Huawei, Google, MS Advance Leveraging Abundant Resources
KMA Achieves Ultra-Short-Term Forecast Success Based on Detailed Weather Data Despite Limited Support

Humans have challenged weather forecasting, once considered the domain of the gods. This challenge saw repeated successes in the 20th century, grounded in advances in science and technology. The greatest change in weather forecasting was the introduction of mathematical calculations. Based on the premise that the motion and state of the atmosphere are determined by physical laws expressible through mathematical equations, the emergence of numerical models that predict weather through mathematical calculations, alongside the invention of computers, largely liberated humanity from the realm of climate uncertainty. The development of computer technology gave rise to supercomputers, and now human-made numerical prediction models have continued to evolve.

NVIDIA CEO Jensen Huang is shown with a large globe image projected on the screen while delivering a speech at the National Taiwan University Gymnasium on June 2. On this day, Jensen Huang emphasized that NVIDIA is accelerating the development of technologies to address climate change. Photo by Baek Jongmin.

However, a formidable obstacle has appeared: climate change. Due to climate change, weather conditions previously unimaginable occur frequently, making weather forecasting more difficult. Unexpected weather that differs from numerical model calculations frustrates meteorologists. The 6th Assessment Report of the United Nations (UN) Intergovernmental Panel on Climate Change (IPCC), approved in 2023, warned that destructive compound weather events will become more frequent due to accelerated global warming. The recent heavy rainfall on the Korean Peninsula, which meteorological authorities failed to predict, also reflects the limitations of existing weather forecasting systems.

To overcome these limitations, governments and big tech companies worldwide have entered a technological competition for weather forecasting. It has become clear that simply improving the performance of supercomputers composed solely of central processing units (CPUs) is insufficient to enhance forecast accuracy. As it is the AI era, there is a growing call for the emergence of AI prediction models based on graphics processing units (GPUs).

Big tech companies took the lead. Their offensive and the rapid development of AI weather forecasting have even bewildered national meteorological authorities. Big tech companies have been rapidly improving forecast accuracy by processing open-source data?public information that meteorological authorities have disclosed for decades.

NVIDIA, leading the AI market, is also advancing in the field of weather forecasting. NVIDIA introduced its own data-based weather model called FourCastNet and offers a cloud-based weather forecasting service named Earth-2. FourCastNet could produce a ten-day forecast in just two seconds but was criticized for lower accuracy. Earth-2 has further accelerated technological progress.

Taiwan has been the most proactive adopter of NVIDIA’s technology. Taiwan recently launched a supercomputer including 192 NVIDIA A100 GPUs and decided to use NVIDIA’s Earth-2 as well. Jensen Huang, CEO of NVIDIA, personally unveiled videos applying Digital Earth in Taiwan and confidently stated that their technology would make a decisive contribution to responding to climate change. The core of the technology NVIDIA provided to Taiwan is the enhancement of 25 km resolution data to super-resolution 2 km scale using AI technology. This means AI compensated for the lack of original data. They claim this allows for more accurate tracking of typhoon paths and other phenomena.

Chinese company Huawei surprised the scientific community last year by publishing a paper on the Panggu Weather AI forecasting model in Nature. Photo shows Huawei's logo. [Image source=Yonhap News]

China’s Huawei shocked the field last year by publishing a paper in Nature on a model called Pangu-Weather. Pangu-Weather was evaluated as an AI weather prediction model outperforming existing numerical analysis models. Subsequently, Google DeepMind’s GraphCast, published in Science at the end of last year, was also a continuous source of amazement. Microsoft joined the competition with its forecast model called ClimaX.

Microsoft Climax Logo

Currently, big tech models are open source. However, it is unknown how long they will remain freely available. Industry insiders expect that if accuracy improves further, these models may become proprietary. This is why direct development of AI models by meteorological authorities is necessary.

The Korea Meteorological Administration (KMA) is also preparing AI forecasts through the ‘AlphaWeather’ project, initiated in 2019 under the National Institute of Meteorological Sciences. The first phase of AlphaWeather is expected to be completed within this year. Although procedures reflecting forecasters’ evaluations remain, the model structure is in its final stages. It is anticipated that six-hour ultra-short-term precipitation forecasts based on the Korean Peninsula will soon be possible. Lee Hye-sook, director at the National Institute of Meteorological Sciences, said, "We have received evaluations from the US and UK that the results are quite significant. We plan to publish a paper soon."

AlphaWeather is the result of a hard-fought effort incomparable to the abundant resources of big tech. Big tech can accomplish in one day what the meteorological institute might take months to do by investing enormous resources. The institute’s research initially started with CPUs only, without GPUs, and currently operates with just two servers and 16 GPUs, eight of which are backups. Fortunately, since the end of last year, support from the Gwangju AI Industry Convergence Project Group, which possesses relatively recent GPUs, and assistance from the Gwangju Institute of Science and Technology (GIST) Supercomputing Center have accelerated the research. Thanks to this, the World Meteorological Organization (WMO) has agreed to mutually verify AlphaWeather alongside models from Microsoft, Google, and others, indicating high recognition of Korea’s AI forecast model.

Director Lee said that competing with big tech is now feasible. He acknowledged that AI performance is improving and speed is fast. However, he pointed out that AI climate models, like generative AI, may lack explanations and can hallucinate, and the basic training data publicly available is at a 25 km resolution. This data can be obtained through the European Centre for Medium-Range Weather Forecasts (ECMWF).

Compared to the 10 km resolution weather data held by major national meteorological agencies, this is significantly lower. The smaller the distance represented by resolution, the more detailed the data. To improve forecast accuracy, more precise information is necessary. Over 30 years of accumulated data is required, but big tech currently lacks sufficient information in this regard. This is why NVIDIA is reportedly making efforts to upgrade to higher resolution.