Samsung Electronics Unveils Next-Generation AI Memory Solutions at 'MemCon 2023'

Choi Jin-hyuk, Director of the US Memory Research Institute, Keynote Speech
As AI Performance Improves, Memory Technology Challenges Also Increase
"We Will Lead the Market by Developing Next-Generation Memory Products"

Samsung Electronics introduced several next-generation products at a global conference discussing trends in artificial intelligence (AI) memory semiconductor technology. The company also expressed its ambition to lead paradigm shifts in the memory market by focusing on the development of various products going forward.

Samsung Electronics announced on the 28th (local time) that it participated in 'MemCon 2023' held in California, USA. MemCon is a conference held for the first time this year to deeply address AI-related memory solutions. Samsung Electronics Semiconductor (DS Division) attended the event as a MemCon Founding Partner. Global companies such as Google and Microsoft were also present.

Vice President Choi Jin-hyuk, Head of Samsung Electronics Semiconductor Americas Memory Research Center (DSRA-Memory), is delivering the keynote speech at MemCon 2023. / Photo by Samsung Electronics

Choi Jin-hyuk, Vice President and Head of Samsung Electronics Semiconductor America Memory Research Center (DSRA-Memory), delivered a keynote speech titled "Memory Innovation in the Data-Centric Era" at the event. He showcased innovative solutions to solve the memory wall issues of various DRAM and NAND flash memories, as well as Samsung Electronics' memory technology vision. The memory wall is a performance bottleneck caused by the increasing gap between processor speed and memory access time.

The next-generation memory solutions introduced by Vice President Choi include ▲ 'HBM-PIM (Processing In Memory)' which integrates computing functions into high-performance memory ▲ 'PNM (Processing Near Memory)' which performs computing functions near memory ▲ ‘CXL (Compute Express Link) DRAM’ that expands system memory capacity to the terabyte (TB) level ▲ '2nd generation Smart SSD' with enhanced internal computing functions in solid-state drives (SSD) ▲ 'Petabyte Storage' that improves server system space utilization ▲ AI and machine learning dedicated 'Memory Semantic SSD,' among others.

Vice President Choi pointed out that as GPT (Generative Pre-trained Transformer) technology continues to advance, the size of training data and models is rapidly increasing, resulting in rising hardware costs required for model training and inference. He explained that large-scale models like ChatGPT experience over 80% delay due to memory bottlenecks, causing slower sentence generation speeds.

According to Vice President Choi, applying HBM-PIM technology can improve AI model generation performance by about 3.4 times compared to existing graphics processing unit (GPU) accelerators equipped with conventional HBM. He also added that applying CXL-based PNM technology is expected to increase DRAM capacity by four times and double AI model loading speed compared to existing GPU accelerators.

Samsung Electronics 8HBM-PIMs Introduction Image / [Image Courtesy of Samsung Electronics]

Samsung Electronics plans to continuously develop and introduce various next-generation memory solutions to meet the growing market demands alongside AI advancements and to promote the development of the memory industry. The company also expressed its ambition to lead paradigm shifts in memory by strengthening cooperation with global IT companies.

According to market research firm Gartner, the AI semiconductor market size recorded $44.4 billion last year, a 27.8% increase from the previous year. It is expected to grow to $86.1 billion by 2026.

This content was produced with the assistance of AI translation services.