Data orchestrator Hammerspace claims its Hyperscale NAS is the world’s fastest file system for enterprise AI, machine learning, and deep learning (AI/ML/DL).
The Hammerspace Global Data Environment (GDE) is based on a parallel NFS file system managing data in globally distributed and disparate sites and enabling it to be located, orchestrated, and accessed as if it were local. It supports and orchestrates data in file (NFS) and object storage using SSDs, disk drives, public cloud services (AWS, Google Cloud, Azure, and Seagate Lyve Cloud), and tape libraries, providing access and lifecycle management, moving data between storage tiers to optimize access speed and storage cost. Hammerspace said in November last year it was working on Nvidia GPUDirect support.
David Flynn, founder and CEO, said in a statement: “Every major enterprise pursuing AI initiatives will encounter challenges with their existing IT infrastructure, in terms of the tradeoffs between speed, scale, security and simplicity … Hyperscale NAS is a fundamentally different NAS architecture that allows organizations to use the best of HPC technology without compromising enterprise standards.”
Hammerspace claims its Hyperscale NAS has been proven to be the fastest in the world for AI training. It is in production at a major hyperscaler in the form of a 1,000-node storage cluster, feeding 16,000 GPUs at an aggregate performance of 100 Tbps.
The unnamed customer chose the product, we’re told, because it is the only system that didn’t require proprietary client software, scaled to meet GPU demands during training and inference, and could use the existing network and storage infrastructure.
Hammerspace has completed the GPUDirect Storage Self-Certification from Nvidia for Hyperscale NAS. Additionally, by deploying Hammerspace in front of existing storage systems, any storage system can now be presented as GPUDirect Storage via Hammerspace to provide high throughput and low latency performance to keep Nvidia GPUs fully utilized.
In other words, layer Hyperscale NAS on existing and non-GPUDirect supporting NAS filers, such as Qumulo, and get GPUDirect support for fast delivery of their content to Nvidia GPUs. As we understand it, this means customers don’t have to rip and replace such filers – just add Hammerspace and get a GPUDirect speed upgrade.
The company says Hyperscale NAS is a new category of storage architecture built for hyperscale large language model training. Hammerspace reckons it provides the speed to efficiently power GPU clusters of any size working on GenAI, rendering, and enterprise high-performance computing.
Hammerspace says enterprises can adopt its HyperScale NAS and get GPUDirect-class data delivery speed to Nvidia GPUs without having to convert their NAS workloads to support parallel file systems such as Storage Scale or DDN Lustre.
High-end block storage array supplier Infinidat is working with Hammerspace so that Infinidat users can layer the file system on InfiniBox arrays and support file workloads.
Infinidat CMO Eric Herzog said: “Our partnership with Hammerspace represents a pivotal step forward, enabling Infinidat’s customers to seamlessly incorporate file-based workloads into their trusted InfiniBox enterprise storage environments.”
