[Game Character New Weapon AI] ① Simulation Training Makes Even Pro Gamers Sweat

AI Character That Analyzes User Behavior to Change Strategy
Reinforcement Learning Enables AI to Defeat Various Past Versions of Itself
"It's Hard to Tell Whether the Opponent Is AI or a Human"

Editor's NoteAs artificial intelligence (AI) is integrated into game characters, immersion and interaction levels previously unimaginable have become possible. AI dramatically transforms the gaming experience by generating and adapting all elements in real time?from character appearance to personality, behavior, and relationships with players. As the gaming industry merges with AI to find breakthroughs, various research methods are gaining attention as experimental subjects.

The biggest concern for game companies is how to maximize player immersion with their characters. While everyone agrees on the direction of enhancing game enjoyment using AI, finding concrete methods is not easy. Game companies are continuously diving into experiments applying various AI learning methodologies.

AI That Even Pro Gamers Admired

NCSoft, a major domestic game company, has been exploring ways to develop characters by applying Reinforcement Learning in the MMORPG (Massively Multiplayer Online Role-Playing Game) "Blade & Soul." In battles against players, the AI does not simply follow predetermined patterns but learns and adapts combat patterns in real time. This effectively adjusts game difficulty, enhancing player enjoyment.

NCSoft had to overcome several challenges to apply a high-quality AI in this game. The goal was to create an AI opponent that is challenging yet less stressful. They utilized a large amount of player log data but faced technical difficulties such as policy optimization and handling complex action spaces.

NCSoft poured almost all its capabilities into its research organization. The research division is mainly divided into Barco and the AI Tech Center. The specialized R&D personnel in these two centers number around 200. Lee Kyung-jong, head of the Barco Center at NCSoft, expressed difficulties, saying, "We conducted human tests with the expectation that AI could beat humans, but the trained AI only learned to beat rule-based AI and could not respond at all to humans fighting in different ways."

Developers have researched reinforcement learning for a long time because the in-game skill system is diverse and complex. Calculating the average skills used per game, movement options, target selection, and average game time, the complexity of the action space was as high as 10 to the power of 1800. The action space refers to all possible character actions in the given game environment, and the larger it is, the more training data is required. Considering that the action space of Go, which is considered to have many possibilities, is 10 to the power of 170, the character movements in the game are extremely free.

To enable AI to learn player behavior and execute various strategies based on it, NCSoft focused on making AI not just follow fixed paths but adapt and respond according to the situation.

The center head explained, "Because player styles vary, flexible AI was needed. To reduce the number of cases caused by high complexity, we devised ways to shrink the action space."

Reinforcement learning allows AI to select optimal strategies based on outcomes after performing specific actions. Through various combat simulations, the AI learns and evolves, providing players with tension similar to actual PvP (player versus player) battles.

Adaptive AI analyzes player patterns to reduce predictability and creates combat situations requiring diverse tactics. Whenever players change their attack or defense patterns, the AI recognizes this and responds with optimal strategies. This is possible because the AI does not remain fixed on pre-learned combat methods but learns in real time and analyzes player tactics.

NCSoft validated AI performance through matches against pro gamers. The AI responded to high-level tactics at the pro gamer level and quickly adjusted strategies according to player styles. Pro gamers who faced three AI versions?‘Offensive,’ ‘Balanced,’ and ‘Defensive’?evaluated the matches as "interesting."

A scene from NCSoft's Blade & Soul where a player engages in a battle against AI Photo by NCSoft

Seamless Team Play

In Lineage Remastered, AI is applied in a different way. While similar to Blade & Soul in that AI analyzes player tactics and predicts movements to execute optimal counter-strategies, it differs in ‘team play.’

A representative example is the "Mirror War" content, where players and AI factions fight for control of the second floor of the prison. This is AI content where players face AI-controlled characters (APCs). NCSoft applied MARL (Multi-Agent Reinforcement Learning) methodology to the APC characters. This is a branch of reinforcement learning dealing with multiple agents interacting in an environment, each learning to find optimal policies. It is used in various fields such as robotics, autonomous vehicles, and game theory.

The center head said, "As AI became capable of predicting enemy movements and player tactics in the game, the tension in battles increased. We are currently researching AI development that fights alongside players as allies, not just as enemies."

Major game companies are also turning their attention to integrating AI into games. Nexon has 700 personnel in its AI research organization, ‘Intelligence Labs.’ They focus on optimizing in-game character behavior and user experience using AI, as well as AI-based matching systems. A Nexon representative said, "We plan to expand our workforce to provide AI-based personalized gaming experiences and to target the global market."

Krafton has a deep learning headquarters with about 100 members, developing AI technologies applicable to game production such as controllable 3D character avatar technology from 2D photos and facial and lip animation generation from voice. A Krafton representative said, "We will expand our organization to predict user behavior through AI simulation and deep learning-based analysis and to provide more innovative game content."

This content was produced with the assistance of AI translation services.