DDN this week spruced up its enterprise portfolio with EXAScaler and DataFlow upgrades, and a new midrange storage array line-up. The enhancements are intended to establish the HPC storage vendor as a good fit for mainstream enterprise AI/ML and analytics workloads.
Update; Infiniband link speeds added to DDN, WekaIO and VAST Data GPU data delivery speed comparisons. Nov 19, 2020. WekaIO provides 173.9GB/sec (162 GiB/sec) to a DGX A100. Nov 20, 2020.
Philippe Nicolas, lead analyst, Coldago Research, said in a statement: “Broadening [DDN’s] portfolio to address the growing AI and performance needs of Enterprise environments is a prudent decision that will improve operational efficiencies for countless organisations.”
Let’s start with EXAScaler 5.2, the new version of DDN’s Lustre parallel file system storage array. EXA v5 was announced in June 2019 and used Lustre v2.12. EXAScaler 5.2 adds snapshots and synchronised data movement between EXAScaler and other file systems, including public clouds. A search facility offers fast metadata scans and file searches at any scale. Admin staff can create new statistics and visualisations to hep improve efficiency and trend analyses.
DDN’s DataFlow management software gets deeper hooks into EXAScaler and can identify file changes and move incremental data to a customer’s designated location. Users can set up DataFlow to customers to migrate data to the cloud, enabling cloud synchronised file systems and snapshot capabilities.
DDN has also rebranded the mid-range Tintri IntelliFlash H-Series to sell as the DDN IntelliFlash H-Series, emphasising the hardware commonality across the DDN and Tintri ranges.
Pumping high speed data to GPUs
Separately, DDN has demonstrated data delivered at 173.9GB/sec (162 GiB/sec) from its A3I storage to Nvidia’s DGX A100 GPU server with its eight A100 GPUs.
This uses Nvidia’s GPUDirect technology and is claimed to be 60x faster than NFS. There is a Reference Architecture with Nvidia for DGX SuperPOD customers. A3I storage uses EXAScaler arrays.
The A100 has a huge data intake capacity with eight x single ported Mellanox ConnectX6 Network Interfaces using HDR200 (200Gbit/s InfiniBand). They link to four PCI switches, each with two GPUs and two NICs. The four PCI switches are interconnected through two AMD CPUs. DDN claims to be pumping data at almost the full line rate capability of the eight HDR200 NICs.
We don’t have direct comparisons with other storage suppliers’ ability to deliver data to the DGX A100.
VAST Data has achieved 92.6GB/sec to the 16 Nvidia A100 GPUs inside a DGX-2 using 100Gbit/s InfiniBand.
On a per-A100 GPU basis DDN delivers 21.7GB/sec across 200Gbit/s IB with Weka pumping out 20.25GB/sec. VAST provides 5.8GB/sec across 100Gbit/s IB, if our back-of-the envelope calculations are accurate.
EXAScaler 5.2, the new DataFlow features and the DDN IntelliFlash H-Series are available now directly from DDN and through certified partners.
