GPU-using workloads such as artificial intelligence and machine learning require faster access to data and metadata to keep them busy. And this presents a challenge for high performance storage vendors such as Panasas.
The company’s PanFS high-performance parallel file system supports mixed IO and high bandwidth but GPU-heavy workloads require lower latency access than the system currently provides.
Panasas is exploring a ‘Ludicrous Mode’ speed accelerator addition to PanFS and software architect Curtis Anderson recently briefed Blocks & Files about this.
He said Panasas engineers are investigating ways of adding lower latency access such as NVMe tier insertion. However, there is no formal commitment to delivering these technologies, which are internally dubbed ‘Ludicrous Mode’, in homage to Tesla’s vehicle acceleration. Blocks & Files thinks that they could come to fruition in the second half of 2020 – subject, of course, to the engineering efforts panning out.
In today’s PanFS setup, front-end client systems access back-end Object Storage Devices (OSDs), using Panasas DirectFlow. The clients use DirectFlow protocol to send IO requests to out-of-band metadata handling devices called ActiveStor Directors (ASD). They set up direct links between the client systems and the OSDs. High bandwidth is utilised to send data from many OSDs at once to the clients.
Anderson said: “One way of adding lower latency would be to make the OSDs all-flash systems and use faster links to them.” But this would be highly expensive when HPC systems use hundreds of terabytes of capacity. Also, not all HPC workloads require very low latency access.
Ludicrous Mode inserts an NVMe-over-Fabrics (NVMe-oF) all-flash storage array between the clients and the OSDs. We think it could provide sub-half millisecond access to data in the NVMe-oF array for the client systems.
They would use NMVe-oF-capable NICs (network interface cards) and RDMA (Remote Direct Memory Access) to access the data. Access by these clients and others to the back-end OSDs would take place in the standard PanFS way.
The NVMe array would use commodity hardware to save cost. Users would manage the NVMe-oF storage using so-called Extended Attributes of PanFS to control data set placement between the backend high bandwidth OSDs and the low latency NVMe-oF store.
The OSDs have been upgraded with multi-tiered data placement:
- NVDIMM for transaction logs, DRAM for data caching
- NVMe for metadata, SSDs for small files, HDDs for large files
- Low latency for small files and metadata, high bandwidth for large files
