SK hynix has worked on developing a computational object storage system (OCS) with Los Alamos Nuclear Labs (LANL) and is showing it at FMS 2024.
It involves having object data stored in Parquet files on NVMe SSDs. When LANL wants to do a large scale simulation run, the data on the SSDs is preprocessed by OCS to reduce the dataset sent to the analysis servers.
The context for this is that, in HPC, analysis of physics simulation data requires large amounts of data held in the storage nodes to be fed to the compute nodes for processing. This requires network bandwidth and enough memory in the compute nodes to hold the processing data set.
But, SK hynix says, “the actual data required for analytics is only a small part of the total data.” Computational storage can be used to reduce the data transfer amount by selecting only the data needed for the processing data set, it says, pre-processing it so to speak.
The intent is to cut down the time needed for analysis of physics simulation data, with a >6.5x speed up demonstrated at an SK hynix demonstration of its prototype OCS system work with LANL at the Supercomputing 2023 event in Denver, CO. The company claimed OCS “can perform analytics independently without help from compute nodes”, claiming this highlighted “its potential as the future of computational storage in HPC.”
OCS is said to be data-aware. This is because block-based storage does not know anything about its data contents, apart from its Logical Block Address and range. An object storage system stores metadata and this can include data content identifiers such as ID, key, name, etc. Local processing on key-value computational storage drives (KVCSD) can then use this indexing metadata to select required data items.
KVCSD is a hardware-accelerated key-value store and can be based on existing key-value stores such as RocksDB and LevelDB. It is described in a downloadable, SK hynix, LANL and NVIDIA research paper: “KV-CSD: A Hardware-Accelerated Key-Value Store for Data-Intensive Applications.“ This says the KV-CSD consists of an NVMe SSD and “a System-on-a-Chip (SoC) that implements an ordered key-value store atop the SSD.” The SoC has has 4 x ARM Cortex A53 CPU cores, 8GB DDR4 RAM, and a Ubuntu OS. This implements an LSM-Tree based KV store.
The claimed result is that: “Through offloaded processing, KV-CSD streamlines data insertion, reduces host-device data movement for both background data reorganization and query processing, and shows up to 10.6x lower write times and up to 7.4x faster queries compared to the current state-of-the-art software key-value stores on a real scientific dataset.”
The paper notes that: “by directly implementing key-value storage management in device, KV-CSD provides opportunities to leverage low-level storage interfaces, such as Zoned Namespace, to optimize performance whereas a software key-value store must rely on the underlying filesystem and the operating system to adopt these optimizations accordingly.” In fact the KV-CSD has a 15TB NVMe zoned namespace SSD using PCIe gen 3.
The OCS project involves the analytics software stack using the Apache analytics ecosystem, including Substrait and Arrow. Substrait provides a standard and open representation of analytic query plans enabling parts of the query to be pushed down from an S3-based storage server to OCS computational storage, specifically an Object based Computational Storage Array (OCSA) used as a backend storage.
There, indexing techniques are used to filter the stored Parquet file dataset and select only the data needed for the query. Apache Arrow software has a language-independent columnar memory format for flat and hierarchical data with a common transfer format. It can be used to transfer such query results, the reduced size data set, back up the stack, using much less network bandwidth, to analytics servers that don’t need as much memory as before.
SK hynix and LANL said: “Orders of magnitude of data movement can be saved by pushing such indexing capabilities closer to the storage devices.”
A demonstration of OCS at FMS 2024 uses the Paraview/VTK (Visualization Toolkit) with Substrait sending a portion of the analytics query plan to the OCS system.
Sungsoo Ryu, head of memory systems research at SK hynix, stated: “This novel approach to data processing minimizes redundant data transfers between analytics applications and storage, and lightens the storage software stack. This accelerates the performance of data-intensive applications such as big data analytics, artificial intelligence and more. SK hynix is striving to develop an analytics ecosystem in collaboration with industry partners.”
Gary Grider, High Performance Computing division leader at Los Alamos, said: “Our large-scale indexing efforts, fueled by industry-standard ecosystems like the Apache columnar analytics, are showing great results.”
Read the OCS research paper for more details of the KV-CSD. Access an SK hynix OCS presentation here.
