Neo Semiconductor has developed three-dimensional DRAM with added neuron circuitry that it says will accelerate AI processing by avoiding data transfers from high-bandwidth memory to GPUs.
Neo’s 3D DRAM technology is the basis for its 3D X-AI 300-layer, 128 Gbit DRAM chip, with 8,000 neurons and 10 TBps of AI processing per die. 3D X-AI chip capacity and performance can be scaled 12x with up to 12 x 3D X-AI dies stacked like high-bandwidth memory (HBM), providing 192 GB (1,536 Gb) capacity and 120 TBps processing throughput.
A statement from Andy Hsu, Founder and CEO of NEO Semiconductor, said: “Typical AI chips use processor-based neural networks. This involves combining high-bandwidth memory to simulate synapses for storing weight data and graphical processing units (GPUs) to simulate neurons for performing mathematical calculations. Performance is limited by the data transfer between HBM and GPU, with back-and-forth data transfer lowering AI chip performance and increasing power consumption.”
3D X-AI simulates artificial neural networks (ANNs), including synapses for weight data storage and neurons for data processing, making it, Neo claims, ideally suited to accelerate next-generation AI chips and applications.
Hsu added: “AI chips with 3D X-AI use memory-based neural networks. These chips possess neural network functions with synapses and neurons in each 3D X-AI chip. They are used to drastically reduce the heavy workload of data transfer data between GPU and HBM when performing AI operations. Our invention drastically improves AI chip performance and sustainability.”
NAND suppliers such as SK hynix and Samsung have experimented with computational memory but it has not become mainstream technology and productized. The use cases are too niche to justify mass production.
Neo will be hoping that AI processing is going to become so widespread that it leaves such nicheness far behind.
An alternative approach to accelerating AI processing is to use specialized processors such as Google’s TPU and Groq’s Tensor Stream Processor (TSP). The Neo 3D X-AI chip can be used with standard GPUs, with the claim of providing faster AI processing with less expense. Now Neo has to find AI processing system or system component builders willing to take a bet on its technology.
Neo will present its 3D X-AI technology at FMS 2024 in Santa Clara, August 6 – 7, and showcase it at booth 507. Check out a 3D X-DRAM slide deck here.
