HPE is the latest vendor to roll out a portfolio of GenAI training and inference products amid plans to use Nvidia GPUs and microservices software announced at Nvidia’s GTC 2024 event this week.
The edge-to-datacenter, hybrid on-premises, and public cloud approach is being brought to the GenAI table by HPE, along with its Cray-based supercomputing capabilities, enterprise ProLiant servers, Aruba networking, Ezmeral data fabric, and GreenLake for file storage. Where competitor Dell is stronger in storage, HPE is stronger in supercomputing and edge networking. The two are roughly equal in server tech and HPE is arguably further advanced in cloud computing with its GreenLake scheme.
HPE CEO and president Antonio Neri said: “From training and tuning models on-premises, in a colocation facility or the public cloud, to inferencing at the edge, AI is a hybrid cloud workload. HPE and Nvidia have a long history of collaborative innovation, and we will continue to deliver co-designed AI software and hardware solutions that help our customers accelerate the development and deployment of GenAI from concept into production.”
HPE is announcing:
- Availability of GenAI supercomputing systems with Nvidia components
- Availability of GenAI enterprise computing systems with Nvidia components
- Enterprise retrieval-augmented generation (RAG) reference architecture using Nvidia’s NeMo microservices
- Preview of Machine Learning Inference Software using Nvidia’s NIM microservices
- Planned future products based on Nvidia’s Blackwell platform
The supercomputing system was announced at SC23 as a turnkey and pre-configured system featuring liquid-cooled Cray AMD EPYC-powered EX2500 supercomputers, with EX254n blades, each carrying eight Nvidia GH200 Grace Hopper chips. It includes Nvidia’s AI Enterprise software and the system can scale to thousands of GH200s. A solution brief doc has more information.
The turnkey version is a limited configuration supporting up to 168 GH200s and is meant for GenAI training. The obvious comparison is with Nvidia’s SuperPOD and the DGX GH200 version of that supports up to 256 GH200s. Dell has no equivalent to the Cray supercomputer in its compute arsenal and is a full-bodied SuperPOD supporter.
HPE’s enterprise GenAI system was previewed at HPE’s Discover Barcelona 2023 event in December and is focused on AI model tuning and inference. It’s rack-scale and pre-configured, being built around 16 x ProLiant DL380a x86 servers, 64 x Nvidia L40S GPUs, BlueField-3 DPUs, and Nvidia’s Spectrum-X Ethernet networking. The software includes HPE’s machine learning and analytics software, Nvidia AI Enterprise 5.0 software with new NIM microservices for optimized inference of GenAI models, NeMo Retriever microservices, and other data science and AI libraries.
It’s been sized to fine-tune a 70 billion-parameter Llama 2 model. A 16-node system will run the model in six minutes, we’re told.
The HPE Machine Learning Inference Software is in preview and enables customers to deploy machine learning models at scale. It will integrate with Nvidia’s NIM microservices to deliver foundation models using pre-built containers optimized for Nvidia’s environment.
The enterprise RAG reference architecture, geared to bringing a customer’s proprietary digital information into the GenAI fold, consists of Nvidia’s NeMo Retriever microservices, HPE’s Ezmeral data fabric software, and GreenLake for File Storage (Alletra MP storage hardware twinned with VAST Data software).
This ref architecture is available now and will, HPE says, offer businesses a blueprint to create customized chatbots, generators, or copilots.
Nvidia has announced its Blackwell architecture GPUs and HPE will support this with future products.
Comment
What’s absent from HPE’s GTC news is full-throated support for Nvidia’s BasePOD and SuperPOD GPU supercomputers. HPE’s storage does not support GPUDirect, apart from the OEM’d VAST Data software forming its GreenLake for File Storage service. Competitors Dell, DDN, Hitachi Vantara, NetApp, Pure Storage, VAST Data, and WEKA are all signed up members of the SuperPOD supporters’ club. Their collective Cray support is a lot weaker.
