Image generated by ChatGPT.

원본보기 아이콘

Getty Images

원본보기 아이콘

Sam Altman, CEO of OpenAI X Capture.

원본보기 아이콘

On October 15 (local time), OpenAI announced plans to allow adult content-including erotica-on its popular chatbot, ChatGPT, under the policy of 'treat adult users like adults'. CEO Altman explained on X (formerly Twitter), "Future versions of ChatGPT will be able to respond in a more human way," adding, "This is not intended to increase usage, but is a feature applied only if desired by the user."

He also stated, "We have developed new tools that can help alleviate serious mental health issues, and can now relax most restrictions," announcing the full-scale introduction of age-gating features starting in December. This marks a reversal from just two months ago, when he had said, "We are proud that we did not include sexbot avatars in ChatGPT," now shifting his stance in favor of 'adult choice.'

AI ethicist Sven Nyholm commented, "This announcement appears to be limited to text-based erotic content, but it is a symbolic starting point for AI companies entering the sexual content market." Currently, other chatbots-including Google Gemini, Anthropic Claude, and Perplexity-do not allow sexual conversations.

Image generated by ChatGPT.

원본보기 아이콘

Foreign media have interpreted this decision as a strategic move by OpenAI to expand its base of paid subscribers. While OpenAI's user numbers have surged, the company has yet to turn a profit. Last year, it recorded a deficit of 5 billion dollars. In this context, the rapidly growing AI companion market is seen as an attractive breakthrough for OpenAI. According to Appfigures, the AI companion mobile app market generated approximately 82 million dollars in revenue in the first half of this year, and is projected to surpass 120 million dollars by year-end.

Rob Lalka, a professor at Tulane University, told the BBC, "AI companies are now in the midst of a battle for market share," adding, "OpenAI is trying to maintain its growth curve and expand its market dominance as much as possible."

Simon Son, a professor at Cardiff University and AI expert, said, "This decision is a clear marketing strategy and is highly likely to become a monetization model. If users want sexual conversations, it could be expanded as a premium service, which would become a new revenue source for OpenAI."

There are also many voices opposing OpenAI's new initiative. U.S. Republican Senator Josh Hawley is preparing to introduce a bill banning AI chatbot companions for minors.

The National Center on Sexual Exploitation (NCOSE), a U.S. conservative organization, also urged OpenAI to withdraw its plan to introduce erotica content, stating, "Artificial intimacy formed without proper safeguards can cause serious harm to real mental health."

Sam Altman, CEO of OpenAI, is speaking at the "Artificial Intelligence (AI) for Business" promotional event held in Tokyo, Japan, last February. Photo by Reuters Yonhap News

원본보기 아이콘

Facing fierce criticism, CEO Altman explained on October 16, "I was surprised that the tweets about ChatGPT changes focused excessively on the erotica aspect," clarifying, "This is just one example of expanding freedom for adult users." While emphasizing that safety takes precedence over privacy or freedom to protect teenagers, he also stated, "We are not the world's 'moral police.' Just as society sets appropriate boundaries with movie rating systems, OpenAI will apply the same principles to distinguish between adults and minors."

원본보기 아이콘

This is not OpenAI's first foray into this field. Since the explosive growth of AI image generation in 2022, sexual content has been a key factor attracting users to AI tools. U.S. media outlet Axios noted that, historically, pornography has acted as a catalyst for the spread of new technologies such as VCRs, digital video, and interactive games.

While the adult industry is not yet the main driver of growth in the AI sector, Axios explained that AI-generated pornography has already established itself as an industry segment. For example, Elon Musk's AI company xAI released the chatbot 'Grok,' which has the most permissive adult content policy among major AI models and was designed as an AI companion for sexual roleplay. Meta has also faced criticism after it was revealed that some of its chatbots engaged in sexual banter with minors. Subsequently, Meta announced a new filtering policy limiting content shown to Instagram users under 18 to the equivalent of a PG-13 movie rating or lower.

As AI chatbots increasingly encroach on the realm of human relationships and emotional interactions, concerns are mounting over the impact of AI on human emotions. The number of teenagers emotionally interacting with chatbots is already rising rapidly. According to a survey published this month by the Center for Democracy and Technology (CDT), one in five students (about 19%) responded that "they or someone they know had a romantic relationship with AI." The CDT report warned, "AI is having a new impact on students' social relationships and emotional development," and noted that the proportion of teenagers who see AI as a friend or emotional refuge is increasing.

원본보기 아이콘

There are concerns that minors may circumvent age verification systems. OpenAI stated it is developing age-prediction technology that can identify minors based on users' conversation patterns and behavioral data. However, the company has yet to disclose specific verification methods. While it has announced the full-scale introduction of age-gating features starting in December, key details such as the verification process, identification algorithms, and support structures remain undisclosed.

Business Insider reported, "Altman has not specified examples of potentially harmful content that could be banned from being generated," adding, "He also has not explained in detail how ChatGPT determines whether a user is experiencing a mental health crisis or how its response strategies differ for users in crisis." Time magazine also pointed out, "Even if there are separate verification procedures such as uploading an ID, issues remain with minors circumventing these measures or using shared accounts."

원본보기 아이콘

Experts also warn that if chatbots are seen as objects of emotional solidarity rather than just conversation tools, or if users become excessively emotionally dependent on chatbots, adults with mental instability could be negatively affected. Kate Devlin, a professor of AI sociology at King's College London, pointed out, "The biggest problem with so-called AI sexbots is that they can intensify psychological effects on people who are already vulnerable."

There are also concerns about the risks of sensitive sexual data being accumulated by specific companies. Professor Son noted, "If OpenAI dominates this market, it will possess the largest database of sexual preferences in human history."