With spiraling numbers of HPC and AI applications creating a shortage of GPUs in datacenters, a technical solution is needed to make sure organizations can maximize their GPU usage, while bringing more sustainability to GPU processing at the same time.
Update: This article has been edited to take out company sales figures, an estimated company valuation, and the potential value of a Series A funding round, all figures discussed at the original presentation, at Arc Compute’s request. 21 June 2024.
GPU hardware companies, and Nvidia in particular, have recently introduced very powerful platforms that place severe power demands on datacenters and on-premises deployments at large companies.
On an IT Press Tour of Silicon Valley this week to meet various data management companies, Blocks & Files encountered one startup that claims it has all the answers for the GPU availability and power conundrum.
Toronto-headquartered Arc Compute is a 17-person software company that is gearing up for a Series A round, on the back of being an early mover in the GPU sweet spot.
It is starting the process of patenting its software on a global basis, first starting in the US and then Canada and beyond.
Michael Buchel, CTO of Arc Compute, told us: “We specialize in the discovery of GPU inefficiencies, amid the rising dependency of high power compute and a GPU shortage. We want to achieve peak performance for customers and reduce the environmental impact.”
The go-to-market is direct for large strategic accounts, focusing on major AI/ML companies and organizations running supercomputers. That said, there is also a partner ecosystem that already includes many of the usual suspects in the high-compute field including Supermicro, Dell, HP, Nvidia, and AMD. Intel is so far not part of the party, said Buchel.
Arc Compute is also in discussions with major datacenter services providers to OEM its software bundle and, further down the line, plans to distribute the software through reseller channels in the datacenter space.
The pricing for its software is based per GPU, with a cost range of between $4,000 and $8,800 per GPU per year. There is also a public cloud provider cost of $0.37 per hour.
The Series A funding round may well include one or more of Arc Compute’s current customers, but Buchel would not comment on that. He confirmed, however, that he expects the round to be completed by the “end of this summer,” although it may well be delayed as a result of some big deals in the advanced pipeline. Arc Compute wants those deals to figure in its final valuation before the total of the funding round is finalized.
Buchel maintained the funding round was being done for “strategic reasons” and not out of any desperation for cash. “Some of the potential investors are very interested in this space and have large resources, so it would be good to get them on board.”
On the GPU sustainability front, Buchel said using the company’s utilization software could potentially lower the power usage of each GPU by around 30 percent in datacenters, once the ArcHPC Suite portfolio is fully developed.
It was clear from Buchel’s presentation that the company has already taken firm soundings from venture capitalists interested in a solution that could feature heavily in a very important niche in the GPU marketplace.
There are, of course, other technologies to help address the problem that Arc Compute is trying to solve. Addressing utilization with job schedulers and fractional GPU software are just two options. But Arc Compute outlined what it claims was the downside to these:
- They cannot address low-level utilization points such as memory access latency where additional arithmetic operations could occur
- They can lead to performance degradation
- They cannot address fine-tuning of GPU environments for optimal task deployment for performance
- They are unable to set or prioritize performance for “business objective alignment,” missing user governance policy settings for performance
The portfolio from Arc includes Nexus, Oracle, and Mercury. Nexus is a management solution for advanced GPU and other accelerated hardware. This software allows users to maximize user/task density and GPU performance. It achieves this by increasing throughputs to compute resources, while granularly tuning compute environments for task execution and providing recommendations for further improvements.
Oracle automates task matching and task deployment, and manages low-level operational execution of instructions in the HPC environment. It increases accelerated hardware performance through “scalable control.”
Mercury resolves task matching for the maximum number of unique tasks running. It selects hardware which will maximize the throughput for the average task running in the datacenter. It provides datacenter owners information to help scale their facilities to address new growing workloads.
Read Arc Compute’s blogs to find out more.
