Lenovo has built a clusterable AI Data Lake system with AMD servers running Cloudian’s HyperStore object storage.
The hardware is Lenovo’s SR635 V3 all-flash server with an AMD gen 4 EPYC 9454P single socket CPU (48-core). A six-node test system fitted with 8 x 7.68 TB NVMe SSDs for data and 2 x 3.84 TB metadata SSDs per node delivered 28.7 GBps reads and 18.4 GBps writes. This was 74 percent more power-efficient than an equivalent disk drive-based system, according to Cloudian testing.
Cloudian CEO and co-founder Michael Tso stated: “Lenovo’s industry-leading servers with AMD EPYC processors perfectly complement Cloudian’s high-performance data platform software. Together, they deliver the limitlessly scalable, performant, and efficient foundation that AI and data analytics workloads require.” He suggested that the combined Lenovo-Cloudian system suited AI, machine learning, and HPC workloads.
Lenovo executive director and GM for storage Stuart McRae said: “This partnership enables us to offer our customers a cutting-edge, scalable, and secure platform that will help them accelerate their AI initiatives and drive innovation.”
HyperStore software from Cloudian is S3-compatible, scalable to exabyte levels, and has Object Lock immutability to protect against ransomware. There are more than 800 enterprise-scale deployments of Cloudian’s HyperStore software.
MinIO has recently announced a DataPOD reference architecture for feeding data from its object storage software to Nvidia GPU servers. It quoted 46.54 GBps read and 34.4 GBps write bandwidth from an eight-node storage server system, with 24 SSDs per node. On a per-node basis, that equates to 5.82 GBps reads and 4.3GBps writes; faster than the Lenovo-Cloudian system’s 4.78 GBps reads and 3.1 GBps writes.
However, on a per-drive basis, the Lenovo-Cloudian system delivered 0.478 GBps reads and 0.31 GBps writes, whereas the MinIO DataPOD RA system provided 0.243 GBps reads and 0.179 GBps writes. This comparison is simplistic and lacks detailed specs of the systems’ CPUs, core counts, memory, PCIe structures, and networking ports. Costs are also not considered. Detailed research is necessary for concrete conclusions.
Both Cloudian and Lenovo say power efficiency is going to become increasingly important, citing a Morgan Stanley report which says power consumption for generative AI is forecast to increase at an annual average of 70 percent through 2027, meaning that “by 2027, generative AI could use as much energy as Spain needed to power itself in 2022.”
The Morgan Stanley analysts believe that generative AI power demands can be met with sustainable sources, and “massive demand for power can also advance sustainable energy technology across sectors.”
The combined Lenovo/AMD/Cloudian AI Data Lake system is available now from Lenovo and from authorized resellers.
