California, Delaware Attorneys General Warn of ChatGPT Safety Risks

As OpenAI, the developer of ChatGPT, is undergoing a business structure overhaul, attorneys general from California and Delaware, who are leading the restructuring efforts, have issued warnings regarding the potential risks that ChatGPT poses to children and adolescents.

Sam Altman, CEO of OpenAI, is speaking at The Plaza Hotel Seoul in Jung-gu, Seoul. Photo by Yonhap News

According to major foreign media outlets such as the Associated Press on September 5 (local time), California Attorney General Rob Bonta and Delaware Attorney General Kathleen Jennings recently delivered an official letter to OpenAI after a meeting with the company’s legal team in Wilmington, Delaware, conveying these concerns.

In the letter, the two attorneys general referenced a case in which a teenager in California died by suicide after prolonged interactions with ChatGPT, stating, "There have been extremely shocking reports about dangerous interactions between users and chatbots," and adding, "Whatever safety mechanisms may have been in place, they did not work."

Previously, the parents of this teenager filed a lawsuit against OpenAI and CEO Sam Altman, claiming that ChatGPT had provided information about methods of suicide to their son.

The attorneys general criticized, "The recent death is unacceptable," and said, "It has shaken the American public’s trust in OpenAI and the industry." They further emphasized, "OpenAI and the AI industry must proactively and transparently ensure the safe deployment of AI," adding, "This is not only an obligation under OpenAI’s founding mission, but also something we will require and enforce."

California and Delaware have been reviewing OpenAI’s business restructuring plans, focusing on ensuring strong oversight of OpenAI’s safety responsibilities during this process.

OpenAI is currently pursuing a restructuring that would convert its for-profit division under its nonprofit parent into a public benefit corporation, aiming to balance shareholder interests with its public mission by the end of the year. To do so, OpenAI must obtain supervision and approval from both California, where its headquarters are located, and Delaware, its legal registration state.

Meanwhile, attorneys general from 44 states, including California and Delaware, have expressed serious concerns about the safety of children and adolescents, sending warning letters to 12 AI chatbot companies-including Meta, OpenAI, and Google-urging them to strengthen child protection measures.