Large scale data orchestrator Hammerspace is using Vcinity’s high-speed remote data movement technology to overcome data gravity.
VCinity’s software moves chunks of data, using remote direct memory access (RDMA) inside IP packets, and enables real-time compute on data thousands of miles away. It moves data over WAN connections at more than 90 percent of the link’s sustained bandwidth. Hammerspace pitches its global parallel filesystem as enabling data to be located and accessed inside a global namespace as if it were local.
However accessing data in large files across continental and intercontinental WAN distances still incurs latency and network transmission time penalties.
Talking about testing the combined technologies, Tony Asaro, SVP of strategy and business development at Hammerspace told B&F:
“Hammerspace performed its data-in-place assimilation, which is normally done locally, over a wide area network (simulated) by leveraging Vcinity. This enables customers and service providers to easily orchestrate third-party file data from remote locations to centralized resources such as GPU farms.”
Vcinity chairman and CEO Harry Carr said in a statement: “Hammerspace immediately understood that the joint solution with Vcinity is a huge win for customers. In an AI/ML world, Hammerspace’s ability to unify, orchestrate, and automate global unstructured data is complemented by Vcinity’s unparalleled performance for moving or accessing data. A global namespace and immediate access to data truly enables the next data cycle.”
Two years ago we wrote that Vcinity provides two gateway devices, appliances or virtual machines, at either end of a wide area link. There can be from three to eight network links between the two Vcinity devices, with data striped across them. The devices run on Linux and implement IBM’s Spectrum Scale (Storage Scale now) parallel file system. A client system mounts its Vcinity device as an NFS or SMB file system and reads or writes data to/from the remote file share via its inline Vcinity device.
VCinity says its combination with Hammerspace’s technology enables customers to make their distributed data accessible to compute, regardless of geographic distance. It establishes a continuous data pipeline into the Hammerspace environment. Customers can choose to extend the capabilities of their Hammerspace environment to wherever their data is located, with the option to either move data hyper fast from point to point (such as into your Hammerspace environment, or across storage locations within it) or instantly access data in place from a remote location (edge, core, or cloud).
We understand that Hammerspace users don’t interface directly with Vcinity, as the Vcinity tech runs “under the hood.”
The Hammerspace/Vicinity tech combo is suited for AI/ML workloads, distributed data center setups, supporting remote workforces and edge compute. Asaro said: “Together, we revolutionize high-performance AI/ML workloads by enabling businesses to orchestrate their distributed data to their centralized GPU resources.”
