Samsung plans to expand its processing-in-memory (PIM) technology for accelerating memory-bound workloads like machine learning (ML) beyond high bandwidth memory (HBM) chips and integrate it into mainstream DIMMs and mobile memory components.
The PIM technology was announced by Samsung earlier this year, where it was implemented by modifying the firm’s commercial HBM2 Aquabolt high-performance memory components. The resulting Aquabolt-XL was designed to combat the memory interconnect bottleneck that can throttle the performance of applications such as AI and ML that require an increasing amount of memory bandwidth. This was achieved by embedding a programmable computing unit (PCU) inside each memory bank, minimising the need for data movement.
At the Hot Chips 33 conference, Samsung disclosed that it is expanding PIM to DRAM modules and mobile memory — a move that is most likely necessary if the chipmaker hopes to see broader adoption of the technology beyond the market segments currently served by HBM, HBM2 and HBM2E.
For servers, Samsung is proposing the Acceleration DIMM (AXDIMM) format that is a drop-in replacement for a standard DIMM, without requiring system modifications. It appears that AXDIMMs will function as a standard memory component, but with the programmable compute functions embedded in the memory buffer, which Samsung refers to as the AXDIMM Buffer or AXB.
Samsung will make available an AXDIMM software stack to offload the acceleration functions to the AXB. Support for the Python programming language and the Caffe2 deep learning framework can be seen in the slide above. The technology is currently being tested on customer servers, where the AXDIMM offers approximately twice the performance in AI-based recommendation applications and a 40 per cent decrease in system-wide energy usage, Samsung claims.
One partner that Samsung has been working with on AXDIMMs is SAP and its HANA in-memory database. In a canned statement a SAP spokesperson said the firm expected to see significant performance improvements, but disclosed no figures to indicate how much of an improvement users might see.
Samsung is also aiming to get its acceleration technology into mobile systems with the LPDDR5-PIM memory technology, for which it claims that simulations have demonstrated it can more than double performance while reducing energy usage by over 60 per cent in applications such as voice recognition, translation and chatbots.
The memory formats that Samsung is targeting for its in-memory processing technology include LPDDR5, DDR5, GDDR6, and HBM3. As some of these are newly introduced or yet to be finalised (in the case of HBM3), mainstream take-up of the technology is unlikely to happen this year.
Samsung said it plans to expand its AI memory portfolio by working with other industry leaders to complete standardisation of the PIM platform in the first half of 2022. The company also said it will “continue to foster a highly robust PIM ecosystem in assuring wide applicability across the memory market.”