Kioxia is redesigning SSDs without a traditional Flash Translation Layer (FTL), a minimal drive microcontroller, and an API for hyperscaler host software to have pretty direct flash hardware control for latency, garbage collection and more.
This is part of the Linux Foundation’s open source Software-Enabled Flash (SEF) project, and is being presented at this week’s Flash Memory Summit Conference & Expo. The aim is to get rid of hard disk drive-era thinking regarding SSD controllers, and provide hyperscaler customers with a way to make their flash media operate more efficiently and consistently. SSDs contain flash dies as before, but the existing FTL-running controller is no more, replaced by a minimal processor running low-level SSD operations and a much-reduced scope FTL.
Eric Ries, SVP, Memory Storage Strategy Division (MSSD) at Kioxia America, said in a statement: “Software-Enabled Flash technology fundamentally redefines the relationship between the host and solid-state storage, offering our hyperscaler customers real value while enabling new markets and increasing demand for our flash solutions.”
A SEF web page identifies five SEF attributes:
- Workload isolation in both hardware and software domains;
- Latency outcome control via advanced, hardware-assisted queueing;
- Complete host control of flash management, garbage collection, and offload features;
- Faster and easier flash technology migration;
- Creation of custom, application-centric and optimized flash protocols and translation layers.
An overview web page tells us that the project is based around purpose-built, media-centric NAND hardware, called a SEF unit, focused on hyperscaler requirements, together with an optimized command set at the PCIe- and NVMe-level for communicating with the host.
We are told: “The SEF hardware unit is architected to combine the most recent flash memory generation with a small onboard SoC controller that resides on a PCB module. As an option, the SEF architecture supports an on-device DRAM controller allowing the module to be populated with DRAM, based upon the needs of each hyperscale user. This combination of components comprise a SEF unit that is designed to deliver flash-based storage across a PCIe connection.”
“Behind the interface, individual SEF units handle all aspects of block and page programming (such as timing, ECC and endurance) for any type or generation of flash memory being used. SEF units also handle low-level read tasks that include error correction, flash memory cell health and life extension algorithms.
“The small SEF onboard microcontroller that resides on the PCB module is responsible for managing flash-based media. It abstracts and controls generational differences in flash memory relating to page sizes, endurance control and the way that flash dies are programmed. Through the software API, new generations of flash memory can be deployed quickly, cost-effectively and efficiently, providing developers with full control over data placement, latency, storage management, data recovery, data refreshing and data persistence.
“The SEF unit also delivers advanced scheduling functionality that provides developers with a flexible mechanism for implementing separate prioritized queues used for read, write, copy and erase operations. This capability, in combination with die time scheduling features, enables weighted fair queuing (WFQ) and command prioritization in hardware that is accessible from the API.”
There is an open source, low-level API and an open source, high-level software development kit (SDK).
Read a trio of downloadable white papers to find out more.
- 7 Reasons for Software-Enabled Flash Technology
- Introducing Software-Enabled Flash (SEF) Technology
- Software-Enabled Flash Technology: Introducing the Software Stack
Or watch one of, or all of, up to eight videos discussing the technology ideas involved.
Comment
Judging by the white papers and videos above, a lot of marketing effort has gone into SEF already – it looks like a fairly mature project. Only Kioxia amongst the NAND and SSD manufacturers seems to be involved. If the hyperscalers react positively – and we’d guess they have all been approached already – then the other suppliers will probably get involved alongside Kioxia.
At this stage it doesn’t look as if there is an enterprise (on-premises) market for this, as enterprises would be loath to put the effort into developing the software involved. But if a third party were to develop SEF hardware vendor-agnostic software, then that picture could change. We’re thinking of JBOFD (Just a Bunch of Flash Dies) software equivalent to Kioxia’s array-led JBOF (Just a Bunch of Flash) Kumoscale software, but vendor agnostic at the SEF hardware level.
