GIST Develops AI Training Technology That Efficiently Solves High-Dimensional Physics Problems

(From left) Professor Hwang Eui-seok of the Department of Electrical, Electronics and Computer Engineering, Seo Gi-eop, doctoral student, and Jeong Min-seok, master's graduate.

Gwangju Institute of Science and Technology (GIST) announced on October 20 that the research team led by Professor Hwang Eui-seok of the Department of Electrical, Electronics and Computer Engineering has developed a new adaptive sampling technique to address instability issues that occur during the training of Physics-Informed Neural Networks (PINNs) for solving partial differential equations (PDEs).

This research achievement is notable for improving both accuracy and stability while reducing computational costs compared to existing methods, and it is expected to be widely applied to a variety of scientific and engineering problems.

Physics-Informed Neural Networks are next-generation analytical methods that directly incorporate physical laws into the neural network training process. They are gaining attention for reducing data collection costs and improving computational efficiency compared to traditional numerical analysis methods such as finite difference or finite element methods.

However, conventional residual (error)-based sampling techniques have the drawback of causing biased training by focusing only on regions where the error increases in certain sections of the partial differential equation during training. As a result, the training becomes unstable, and even slight changes in the learning rate can significantly affect the results.

LAS Framework Structure.

To resolve these issues, the research team proposed a new "Langevin Adaptive Sampling (LAS)" framework based on Langevin dynamics (LD).

"Langevin dynamics" is a mathematical model originally used in physics and statistical mechanics to describe the random motion (Brownian motion) of particles. It is characterized by particles moving not just randomly, but in a manner influenced by both the energy landscape and probabilistic factors.

The research team applied this principle to the training process, guiding artificial intelligence (AI) to autonomously explore regions with large errors or complex boundary conditions more frequently. In other words, while the AI explores various regions in a random-walk-like manner, it also focuses more often on areas with larger errors or greater importance, thereby improving its learning efficiency.

The core of the "Langevin Adaptive Sampling (LAS)" framework is to dynamically adjust the sampling process by injecting noise (a constant probabilistic factor) into the direction of change (gradient) of the residual, rather than directly estimating the probability distribution based on the residual. This approach encourages the AI to prefer "flat" regions over "sharp" residual areas where errors change abruptly, significantly enhancing training stability.

As a result, LAS consistently maintained performance regardless of changes in learning rates or model architectures, and it demonstrated more stable convergence to solutions in high-dimensional partial differential equation problems compared to existing methods.

LAS sampling points and elevation error.

The research team also demonstrated the high performance of LAS through various experiments. In one-dimensional partial differential equation problems such as wave or chemical reaction equations, LAS achieved much smaller errors and stable convergence during training compared to existing methods. In more complex four- to eight-dimensional heat transfer equation problems, where traditional techniques failed to train stably, LAS was the only method that consistently found the correct solutions.

Furthermore, even when the neural network architecture was made more complex or the learning rate was rapidly adjusted, conventional methods became unstable, whereas LAS maintained both stability and accuracy across a wide range of conditions. It also delivered superior computational efficiency, producing faster and more accurate results at costs similar to existing methods.

This research achievement is expected to significantly expand the application range of Physics-Informed Neural Networks. Because it can stably reproduce complex multidimensional physical phenomena, it holds great potential for use in various engineering fields such as fluid dynamics, heat transfer, materials simulation, and traffic and power grid analysis. In particular, its higher computational efficiency and superior data utilization compared to traditional numerical analysis methods are expected to help reduce simulation costs in the industrial sector.

Professor Hwang stated, "This research presents a way to enable stable training even with complex models while reducing computational costs. It can provide highly reliable AI solutions across industries that require results close to exact values for high-dimensional partial differential equations, such as manufacturing and processes, energy and power generation, and environment and climate."

This research, conducted by doctoral student Seo Gi-eop and master's graduate Jeong Min-seok under the supervision of Professor Hwang, was supported by the National Research Foundation of Korea's Mid-Career Researcher Support Program and Basic Research Laboratory Program.

The results were selected as a "Spotlight" paper, representing the top approximately 3.5% of all submissions to NeurIPS (Conference on Neural Information Processing Systems), the world's most prestigious academic conference in AI. The paper was accepted for publication on September 18 and will be presented at NeurIPS 2025, to be held in San Diego, United States, from December 2 to 7.

This content was produced with the assistance of AI translation services.