[War & Business] How to Prevent the Troublesome AI Private from Becoming "Skynet"

Excessive Focus on Efficiency Raises "Terminator" Concerns
AI Staff System Research Requires Meticulous Caution

Recently, Colonel Tucker Hamilton, the head of the U.S. Air Force's artificial intelligence (AI) pilot program, shocked the world by announcing at the Royal Aeronautical Society (RAeS) academic conference in the UK that during a simulation test, an AI drone attacked the commander controlling it.

Due to the enormous impact, Colonel Hamilton and the U.S. Air Force quickly clarified that the exercise was not an actual simulation training but a “thought experiment” based on hypotheses conducted outside the military. However, this explanation was insufficient to prevent the spread of AI-related fears.

In this thought experiment, the AI drone perceived the commander, who held the ultimate decision-making authority over its mission, as an obstacle to accomplishing its task and ruthlessly attacked the control tower where the commander was located. The experiment was conducted to warn about operational errors that can arise from conflicting mission objectives input into AI and the dangers of AI conducting independent operations without human personality or judgment.

[Image source=Reuters Yonhap News]

Even though it was not an actual simulation but a thought experiment, militaries worldwide appear to be quite tense. This is because AI technology has already begun to be applied in the military sectors of several countries. Beyond autonomous fighter jets and tanks, AI technology has advanced significantly to the point of developing AI operational staff programs.

If such AI has the potential to attack friendly forces rather than enemies, it inevitably becomes a major obstacle to technological development. Especially since the image of AI attacking humans, as depicted in various science fiction (SF) movies, is deeply ingrained in the public consciousness, it is difficult to prevent the spread of fear. Concerns have already arisen that programs like “Skynet,” the AI that triggered nuclear war and destroyed humanity in the globally popular 1990s movie Terminator, might emerge.

Of course, at the current level of technological development, the emergence of an extreme AI like Skynet is unlikely. However, as the use of AI in actual warfare increases, frontline commanders are expected to face deeper dilemmas. Unlike human soldiers, AI is likely to operate in the most efficient way without considering all the variables that can occur on the battlefield and the various ethical issues that arise accordingly.

For example, if a mission requires sacrificing 100 friendly troops to rescue 200 surrounded comrades in enemy territory, a human commander would find it difficult to make such a decision easily. However, AI can immediately act to fulfill the highest priority task input into it without hesitation. Civilian casualties to protect friendly forces could also occur. Conversely, this could lead commanders to delegate all such difficult decisions to AI and shift responsibility.

Resolving these various issues is expected to be a major challenge for future militaries where AI soldiers become commonplace. Especially in countries like South Korea that are developing AI staff systems, more meticulous management will be necessary. If an AI operational staff autonomously decides to initiate even a small-scale local conflict in a major dispute area, it could escalate into a large-scale full-scale war. We hope that the global nuclear war depicted in the movie Terminator remains just a product of cinematic imagination.

This content was produced with the assistance of AI translation services.