Hammerspace has added erasure coding to its data orchestrating Global Data Environment software, enabling it to use less overhead when storing and protecting data.
The company supplies parallel NFS-based software to manage data in globally distributed and disparate sites, in file (NFS) and object storage using SSDs, disk drives, public cloud services (AWS, Google Cloud, Azure, and Seagate Lyve Cloud), and tape libraries. This enables it to be located, orchestrated, placed, and accessed as if it were local. Hammerspace bought RozoFS, a French startup developing its Mojette Transform erasure coding technology, in May 2023 to protect against data loss with less overhead and faster recovery than RAID.
David Flynn, Hammerspace founder and CEO, said: “We started this year by adding support for data on tape, then pioneered the Hyperscale NAS architecture for AI and GPU computing, and now we are further expanding our data storage services with high-performance erasure coding. This gives customers even more choice and flexibility when it comes to their storage infrastructure.”
Erasure coding (EC), like RAID, involves the use of mathematically computed codes – parity values in RAID – from source data. If some of the source data is lost then it can be recalculated from its fragments and the codes. The codes represent a storage overhead that is required above that of the raw data storage. In general, if a protection scheme means that this overhead can be reduced and the data recovery process completed in less time then it is to be preferred over less efficient schemes.
EC generally requires less overhead than RAID. The Mojette Transform is, Hammerspace says, an “extremely CPU-efficient erasure code” that bypasses the complex mathematical calculations of usual erasure codes, leaving more compute power available to applications. The software’s basic algorithm is described here.
RozoFS’s technology is, Hammerspace claims, “up to 2x faster than traditional erasure coding schemes.” As a result of its simpler mathematical structure, the Mojette Transform needs comparatively less storage space than other EC schemes, such as Reed-Solomon, and is scalable out to billions of files.
Hammerspace says that this provides a new option for building high-performance storage using its technology and commodity hardware. Examples are when building scratch space for HPC and research environments, using Open Compute Project (OCP) hardware in hyperscale environments, or for building all-flash storage environments for AI initiatives to augment or replace legacy storage.
Pierre Evenou, Hammerspace VP of Advanced Technology and founding engineer of Mojette Transform, said: “Our goal in developing the Mojette Transform erasure code was to deliver the highest reliability in data protection, coupled with extreme performance, leveraging commodity hardware. The result is delivering close to native performance of the underlying storage hardware to the application and compute environment without sacrificing data protection.”
