Memory Facing 'Supermassive AI'... Samsung's "PIM·PNM Ultra-Gap Response"

Samsung Electronics Unveils Next-Generation AI-Ready Advanced Memory Technology Through Blog

Image explaining the difference between HBM and HBM-PIM / Provided by Samsung Electronics

[Asia Economy Reporter Kim Pyeonghwa] As artificial intelligence (AI) technology rapidly advances, performance challenges have emerged in the memory semiconductor sector. Samsung Electronics, the market leader in memory, plans to maintain its industry leadership by responding to these demands with next-generation technologies such as 'HBM-PIM' and 'CXL-based PNM.' They will also work to promote the spread of related technologies through communication with the IT industry and academia in the future.

On the 20th, Samsung Electronics shared this information in a blog post titled 'Samsung Electronics Semiconductor Unveils Advanced Memory Technology for Next-Generation AI.'

According to Samsung Electronics, the recent hot topic in AI is 'super-large AI.' Super-large AI represents an evolution from existing AI to a level where it can learn, think, and make judgments autonomously like the human brain. To realize this level of AI, learning and computation with much larger volumes of data than before are required. Computing infrastructure capable of performing this is essential. Providing sufficient DRAM capacity and bandwidth to support computing performance has become a challenge for the memory industry.

To support super-large AI, Samsung Electronics introduced Processing-In-Memory (PIM) and Processing-Near-Memory (PNM) solutions. They have also secured memory solutions utilizing these technologies and completed software standardization necessary for their implementation.

HBM-PIM Applied GPU Accelerator Saves 2100 GWh Over One Year

PIM is a technology that implements data computation functions performed by the processor inside the memory. In systems without PIM, large data transfers occur between the processor and memory, which can cause bottlenecks when processing excessive data. PIM addresses this by performing computations inside the memory itself. This reduces data movement, thereby improving the performance and energy efficiency of AI accelerators (dedicated hardware for AI execution) systems.

Samsung Electronics introduced high-bandwidth memory (HBM)-PIM in February last year using this technology. HBM is memory used in high-performance computing (HPC) and other applications. Samsung combined the HBM2 Aquabolt with an AI processor to present next-generation memory. HBM-PIM features twice the performance of existing HBM2 while reducing system energy consumption by 70%.

Samsung Electronics also collaborated with the US semiconductor company AMD to equip AMD’s graphics processing unit (GPU) 'MI-100' accelerator card with HBM-PIM memory. Subsequently, by implementing an HBM-PIM cluster and applying it to large-scale AI and HPC applications, they confirmed that performance doubled and energy consumption decreased by 50% compared to existing GPU accelerators.

Samsung Electronics explained, "After configuring a system with eight GPU accelerators to train a large-scale AI language model, it was confirmed that using GPUs equipped with HBM-PIM can save 2100 gigawatt-hours (GWh) of energy annually compared to GPUs equipped with HBM." They added, "Reducing energy consumption by 2100 GWh can cut carbon emissions by approximately 960,000 tons, which is more than the amount of carbon absorbed by about 100 million pine trees over one year."

Samsung Electronics believes that using HBM-PIM can solve many data center challenges related to super-large AI. Accordingly, they have also begun software support for HBM-PIM usage. Samsung defined software specifications for use on GPU accelerators by utilizing the open software standard SYCL. Software based on this is scheduled to be released next month.

Performance Doubled with CXL-Enabled PNM

Another card Samsung Electronics introduced is PNM. Like PIM, PNM uses data computation functions in memory to reduce data movement between the central processing unit (CPU) and memory. The difference is that while PIM performs computation inside the memory, PNM places the computation function next to the memory. The goal is the same: to reduce bottlenecks between the CPU and memory and improve system performance.

Samsung Electronics focused on Compute Express Link (CXL) to utilize PNM. CXL is a newly proposed interface that enables efficient use of accelerators, memory, and other components used alongside processors in computing systems. It helps overcome the physical limits of existing memory capacity and supports capacity expansion.

Samsung Electronics developed CXL interface-based PNM technology suitable for handling high-capacity AI models. This technology demonstrated more than twice the performance in applications requiring high memory bandwidth, such as recommendation systems and in-memory databases. Samsung first unveiled this technology at 'Samsung Tech Day 2022' held earlier this month in Silicon Valley, USA.

Samsung Electronics plans to actively communicate with the IT industry and academia to promote the spread of HBM-PIM and CXL-based PNM technologies. They also plan to release integrated software supporting HBM-PIM and CXL-based PNM solutions. Samsung will participate in SC22, the largest supercomputing conference in the industry, held on the 13th of next month (local time) in Texas, USA, where they will exhibit and demonstrate these solutions.

Park Cheolmin, Executive Director and Head of New Business Planning Team at Samsung Electronics Memory Business Division, said, "The HBM-PIM cluster technology is the industry's first large-scale AI-customized memory solution. Through the integrated software standardization process, we plan to combine it with the CXL-PNM solution to maximize energy savings and carbon emission reduction effects, contributing to eco-friendly management."

This content was produced with the assistance of AI translation services.