Broadcom has launched the latest generation of its Tomahawk switch chip, promising to massively boost data bandwidth within datacenters while simultaneously putting a dent in power consumption.
The Tomahawk 6 switch series offers 102.4 Tbps of ethernet switching capacity, with support for 100G and 200G SerDes. It says the former will enable AI clusters with extended copper reach and “efficient use of XPUs and optics with native 100G interfaces.” The latter will provide “the longest reach for passive copper interconnect.” It will be available with co-packaged optics.
The chip is the Tomahawk family’s first multi-die part, and is built on a 3 nanometer process. Tomahawk 5 was a 5 nm process. The firm is currently working on its shift to the 2 nm process, which means it still has headroom to grow.
Broadcom reckons it will cement Ethernet as the networking standard in datacenters and is pitching the chip at both scale up and scale out deployments. It says Tomahawk 6 will support scale up clusters of 512 XPUs, or 100,000 XPUs or more in a two tier scale-out network at 200Gbps/link.
Peter Del Vecchio, product manager for the Tomahawk switch family, said the firm had consistently doubled available bandwidth every 18 to 24 months with each generation of Tomahawk. He compared this pattern to Moore’s law.
However, the last generation appeared in 2022. Del Vecchio pointed out that this was just before Chat GPT burst into public consciousness, and the company had taken its time to really understand the impact on datacentres.
Bigger model impact
And a large part of that impact was an explosion in the amount data being stored and moved as model sizes grew. “The world kind of changed, where suddenly everyone had to have as much bandwidth as possible, as quickly as possible.”
He said studies had shown that up to 57 percent of the time invested in LLM training was down to data transfers, during which power guzzling GPUs were sitting idle.
“Getting the network out of the way, getting the data transfer between GPUs is, just one of the most important things as far as making your GPU cluster efficient,” he said.
Speeding up transfers will increase utilization. But, together with Tomahawk enabling larger two tier networks and lowering the number of hops and amount of optical networking needed, it will also impact the massive power consumption iin data centres.
“Your general rule of thumb is that in these networks, about up to 70% of the power consumption could be just due to the optics. So if we have this huge reduction in the number of optics, you actually have about half of the power for the network go into two tiers compared to three.”
And data demands will continue to rise, as models and clusters increase in size. Not least because of the checkpointing demands in case of failures. “You’re enabling these very large networks, but they have these very large clusters, which also means that when you’re checkpointing, there’s a lot more data to be checkpointed.”
Del Vecchio said the chip’s predecessor had driven a shift from InfiniBand to ethernet in large GPU-powered AI clusters. This meant ethernet was ubiquitous through the entire datacentre he said, which was a major benefit for operators. “You can have a common set of tools, it means that you have all the same cables, you have all the same infrastructure, your network engineers know how to debug that network. I think probably even larger benefit is that you have this huge ecosystem around Ethernet.”
