Will NVMe become the universal block storage access protocol?

Facebook’s virtual OCP summit earlier this month hosted two provocative presentations that suggested NVMe could become a universal block access protocol. How realistic is this prospect?

The 2020 OCP Virtual Summit, held May 12 to 15, featured updates on the OCP storage project and speakers who floated NVMe as a replacement for SATA and SAS interfaces for solid-state, disk and even tape drives.

Currently, data sent to or read from disk and tape drives passes from the host’s DRAM across a PCIe bus to a SAS or SATA interface and thence to the drive. A conversion process is required when the data passes from the PCIe realm into the SAS or SATA realms.

SAS and SATA

SAS – Serial Attached SCSI – was developed to access disk and tape drives using the SCSI command set. Features include error recovery, reservations and block reclamation SAS is full duplex – it can be used to read and write simultaneously.

Lower cost SATA – Serial Advanced Technology Attachment – is half-duplex – you either read or you write but can’t do both at once. This was developed from the Parallel ATA standard to interface disk, tape and solid state drives to hosts.

SATA was announced in 2000 and has developed alongside SAS, which made its debut in 2004. SAS is considered more of an enterprise system protocol and is preferred for servers, while SATA is used, for example, to connect disk and solid state drives in a PC.

NVMe

SAS and SATA are serial access methods whereas Non-Volatile Memory Express (NVMe) is parallel and can read and write data in parallel streams to and from an SSD’s NAND chips. It is faster than SAS and SATA and can stream more data. NVMe provides faster data access for SSDs directly attached to a host systems’s PCIe bus.

A PCIe switch could be used to link many drives to the bus, just as an SAS expander is used to link disk drives to a SAS interface.

SAS, SATA and NVMe are all block access protocols and so can be used for block access devices such as tape, disk and solid state drives. However, disk and tape drives are serial access devices, unlike SSDs, and therefore providing NVMe access is a waste of protocol bandwidth.

That said, a universal NVMe drive interface would simplify system design, with no need for SAS and SATA protocol conversion when their data streams cross into the PCIe bus.

OCP Storage Project sessions

Jason Adrian, a project lead at The OCP Storage Project, and in his day job a storage hardware architect at Microsoft Azure, presented a session called “NVMe HDDs – the future starts now”.

In his presentation abstract, he writes: “The SAS and SATA interfaces have dominated the storage market for nearly two decades, but is this the right interface for the future? For SSDs, the market is already shifting to NVMe for its higher bandwidth, lower latency, and overall light-weight interface. For the HDD market, SAS and SATA are plenty fast for many more years. Why would we consider NVMe? Join us as we explore some of the un-tapped opportunities with NVMe HDDs.”

Matt Shumway and Mohamad El-Batal, from Seagate’s Office of the CTO, presented a session called “Can Block Storage Devices Converge on NVMe Protocol?

They write: “In the last ~40 years, some of us have seen many evolutions of block storage protocols come and go: IDE/ATA, SCSI, PATA, P-SCSI, SATA, USB, FCP, SAS … NVMe. Each protocol optimised specific Enterprise or Consumer storage device solutions and features. Regardless of the various storage media types and characteristics, is it finally possible for the storage industry to unify and consolidate around NVMe as the optimal block storage protocol? … Why? … When?”

Here we have two sessions, teasing attendees with the prospect of a universal NVMe block access protocol. The presentation slides are not yet generally available. However a Seagate slide has appeared in a Forbes article by analyst Tom Coughlin:

The slide text notes NVMe would be good for multi-actuator disk drives, which contain two sets of read/write heads.

NVMe-accessed disk drives could also enable composable systems, built using the PCIe bus, to include disk drive storage alongside other elements – CPUs, GPUs, memory, SSDs, network switches and so forth.

Infinidat today announced NVMe/TCP support for InfiniBox arrays, which store data on disk drives. So, this software update delivers NVMe access to disk drive storage – we think this is an industry first. According to Infinidat CTO Brian Carmody, NVMe will be the pervasive block access method for multi-petabyte enterprises by 2025.