Flitto Provides Benchmark Dataset for Korean AI Performance Evaluation Leaderboard

AI language data specialist Flitto announced on the 12th that it has provided benchmark datasets for the 'Open Ko-LLM Leaderboard,' which compares and evaluates the performance of Korean ultra-large language models.

The Open Ko-LLM Leaderboard is a Korean large-scale language model (LLM) performance evaluation platform jointly established and operated by the National Information Society Agency (NIA) and Upstage. It provides an environment where domestic companies and research institutions can register LLMs and compete in performance. It contributes to the development of Korean-style AI and natural language processing technologies.

As a partner of Upstage, which operates the Open Ko-LLM Leaderboard, Flitto provided benchmark datasets in Korean that evaluate ▲common-sense reasoning and contextual understanding abilities ▲mathematical reasoning and calculation skills. By adding these two evaluation categories?common-sense reasoning and mathematical reasoning?the platform aligns its evaluation criteria with Hugging Face’s internationally recognized ‘Open LLM Leaderboard,’ marking a significant milestone for the domestic AI ecosystem.

Leveraging its participation in dataset construction, Flitto plans to accelerate the creation of high-quality language data to evaluate and improve the performance of Korean large-scale language models. In cooperation with Upstage, it is building benchmark datasets to evaluate Korean language models applied in commercial fields, which will be additionally released through ‘Open Ko-LLM’ within this year.

Flitto CEO Lee Jung-soo said, “Providing benchmark datasets to the Korean large-scale language model leaderboard with international evaluation standards is highly meaningful,” adding, “Based on our years of accumulated expertise in language data construction technology, we will devote more effort to advancing the Korean AI ecosystem.”

On the 9th of last month, Flitto signed a business agreement with AI technology company Upstage to build AI language data. They agreed to collaborate mainly on constructing datasets for Asian languages with relatively scarce language data, such as Thai, Japanese, Vietnamese, Lao, and Cambodian.

This content was produced with the assistance of AI translation services.