Upstage-NIA Launch Korean AI Performance Evaluation Platform

Upstage and the National Information Society Agency (NIA) have joined forces to create a leaderboard that can evaluate and compare the performance of large-scale Korean language models (LLMs).

Upstage announced on the 25th that, in co-hosting with NIA, they will launch the Korean LLM leaderboard called the 'Open Ko-LLM Leaderboard' on the 27th.

The Open Ko-LLM Leaderboard is a public platform where anyone can register their developed Korean LLM models and compete with other models. Rather than simply translating existing data from the OpenLLM Leaderboard operated by Hugging Face, it independently builds high-quality data reflecting the characteristics and culture of the Korean language, giving it the advantage of being a Korean language-specialized leaderboard.

By adding the 'Common Sense Generation' criterion, which examines the ability to generate common sense, it enables the evaluation of high performance and diversity of Korean LLM models. The Common Sense Generation dataset was constructed by Upstage in collaboration with Professor Hee-Seok Lim’s research team at Korea University. It consists of a questionnaire covering a wide range of types including historical distortion, hallucination errors, morphological errors, irregular usage errors, and hate speech. Through this, it measures whether the AI-generated outputs align with the general common sense that Korean language users would possess.

Seonghoon Kim, CEO of Upstage, said, "We are very pleased to launch the Open Ko-LLM Leaderboard, which can enhance the competitiveness of Korean LLMs and further raise the level of research." He added, "We will continue to strive to expand and promote the development of the Korean AI ecosystem."

This content was produced with the assistance of AI translation services.