Micron has devised a modified storage IO stack for Linux that delivers lower latency, faster performance and longer life. The US chipmaker said the ‘heterogeneous-memory storage engine’ (HSE) is host-level software, not device-level.
HSE works with SSDs and storage class memory and is extensible to new interfaces and storage devices for applications across databases, IoT, 5G, AI, HPC and object storage.
The code optimises performance and endurance by orchestrating data placement across DRAM and multiple classes of SSDs or other solid-state storage devices. It implements a key:value store, and scales to terabytes of data and hundreds of billions of keys per store.
A storage engine hooks up an application such as a database to storage drives and their controllers and enables them to talk direct to drives. It is not the actual drive controller. Micron’s HSE code sits in a host and replaces a standard or existing storage IO stack, so it needs to be integrated with the application. Micron has facilitated this integration by making HSE open source.
Micron has tested HSE-enabled workloads against the RocksDB storage engine, using YCSB (Yahoo! Cloud Serving Benchmark) workloads and four Micron 9300 SSDs. HSE improved performance throughput 6x, reduced latency 11x and lengthened flash endurance 7x. It achieved this by reducing write amplification.
Micron has also integrated HSE with MongoDB and claims an 8x throughput improvement.
HSE is available on Github, where there is an HSB Wiki resource. A blog by Larry Hart, Micron director of product marketing, provides additional information.
HSE uses
We think Micron’s HSE initiative is motivated in part to encourage third-party vendors to modify their applications to work with its upcoming 3D XPoint SSD drives.
Ceph is a potential integration candidate for HSE, and Stefanie Chiras, VP and GM of Red Hat Enterprise Linux, said in the announcement release: “We see enormous potential in the technologies being introduced by Micron, especially as it takes an innovative approach in lowering the latency between compute, memory and storage resources.”
Scality, an object storage supplier, has also provided a supporting quote, courtesy of field CTO Brad King: “While our storage software can support ‘cheap and deep’ on the lowest-cost commodity hardware for the simplest workloads, it can also exploit the performance benefits of technologies like flash, storage class memory and SSDs for very demanding workloads.
“Micron’s HSE technology enhances our ability to continue optimising flash performance, latency and SSD endurance without trade-offs.”
