"Optimizing Giant Language Models"... US Nvidia Unveils Next-Generation AI Semiconductor 'H200'

Upgraded Version of H100... Output Speed Doubled

Leading the artificial intelligence (AI) semiconductor market, American company Nvidia has unveiled its latest AI semiconductor.

On the 13th (local time), Nvidia announced the H200 graphics processing unit (GPU), designed to be applied to and train large language models (LLMs), which serve as the foundation for generative AI models.

The H200 is an upgraded version of the H100, which is used to train GPT-4, the latest LLM developed by OpenAI, the creator of ChatGPT.

The price of the H100 is estimated to be between $25,000 and $40,000 per unit. Thousands of chips are required to operate LLMs. Despite the high price of the H100, companies worldwide are fiercely competing to secure this semiconductor.

The price of the newly announced H200 has not been disclosed.

The H200 is equipped with 141 gigabytes (GB) of next-generation memory, HBM3. High Bandwidth Memory (HBM) is a high-performance product that vertically stacks multiple DRAMs to dramatically increase data processing speed. HBM3 is the fourth generation of HBM and is used for inference by LLMs trained to generate text, images, and more.

Nvidia explained that based on evidence from using the H200 with Meta’s LLM, LLaMA 2, the H200’s output is twice as fast as the H100. Additionally, the H200 is compatible with the H100, so AI companies that have secured the H100 do not need to change their server systems or software to use the new version.

Furthermore, this semiconductor can be used in the GH200 chip, which combines the H200 GPU with an Arm-based processor, as well as in server configurations of Nvidia’s computing platform, HGX.

The H200 is expected to be officially released in the second quarter of next year. Accordingly, it is anticipated to compete with the MI300X semiconductor from the U.S. semiconductor company AMD, which is set to be released soon. The MI300X semiconductor has 2.4 times the memory density and 1.6 times the bandwidth compared to Nvidia’s H100.

This content was produced with the assistance of AI translation services.