The PCIe bus has been a tremendously successful way to hook up x86 servers, PCs, and laptops’ core processor and memory complex with peripherals and, as CPUs and GPUs get faster, so too does the bus, doubling bandwidth with each generation.
PCIe v7.0 and 8.0 are coming and we take a look here at how they achieve their speeds.
PCIe (Peripheral Component Interconnect eXpress) 4.0 and 5.0 standards are currently being used in x86 PCs and servers to hook up devices, , such as storage drives, with the very first v6.0 peripherals being introduced, such as Micron’s 9650 Pro SSD. As PCIe standards are backwards compatible, peripherals using older versions can work with hosts using newer versions.
Let’s look at the PCIe standard generally, before taking a look at the coming 7.0 and 8.0 standards.
The PCI-SIG is the industry body looking after and developing the PCIe standard. Basically the PCIe bus is a serial, not parallel link, between peripheral devices, such as storage drives and network interface cards (NICs), and the CPU/memory core complex. It has multiple wires which are paired, one for receiving data, one for transmitting data, meaning that they are differential signaling pairs, and the pair can transmit signals bi-directionally. Two such pairs are called a lane and a PCIe link can have 1, 2, 4, 8, or 16 such lanes.
Each peripheral device has a point-to-point link to the CPU and data is sent over it in 2-part packets. First there is a header and this is followed by a data payload. The header includes timing information, obviating a need for separate clock timing lines. Each lane has its own embedded clock, synchronized by a reference clock.
PCIe link length is up to ~30 cm on standard printed circuit boards (PCBs) made from FR-4, glass-reinforced epoxy laminate, without repeaters or retimers.
Succeeding PCIe generations increase the bus speed and also its carrying capacity, by altering the data encoding scheme to reduce overhead.
The basic performance measure is GTps, GigaTransfers per second; the number of billions of transfers per second across a lane. Each transfer carries a number of bits, the number varying with the encoding scheme.
For example, PCIe 3.0 used 8b/10b encoding which converts 8-bit data bytes into 10-bit symbols, 2 bits per 8-bit block, to ensure DC balance and no long runs of 0s or 1s. This adds a 25 percent overhead and PCIe 3.0 has a raw 8 GTps per lane which results in ~1 GBps effective bandwidth after encoding overhead
Later versions such as PCIe 4.0 and 5.0 used 128b/130b scheme to reduce the overhead to about 1.6 percent, 2 bits per 128-bit block. PCIe 5.0 has a raw 32 TPps rate per lane which results in ~3.94 GBps after overhead.
PCIe 6.0 introduced PAM4 signalling, with 4 pulse amplitude voltage levels, achieving 2 bits per transfer signal instead of the previous 1. That delivers 64 GTps per lane which yields ~7.88 GB/s per lane after overhead.
Here is a table listing PCIe generations and their performance, with PCIe 7.0 and 8.0 included;
Flit stands for Flow Control Unit and a Flit is a fixed size, 256-byte (2048-bit) data unit that encapsulates data packet header, payload, cyclic redundancy check data, flow control and protocol information into a single block sent across a PCIe lane. The use of Flits ensures that a PCIe sender does not overwhelm a PCIe receiver’s buffers.
Previous and slower PCIe generations used variably sized Transaction Layer Packets (TLPs). PCIe 6.0 needs Flits to operate reliably at its high speed.
As a chart shows, PCIe standards are being refreshed every three years from PCIe 4.0 onwards.
PCIe 7.0 delivers a 128 GT/s transfer rate, meaning 128 Gbps per lane, double that of PCIe 6.0. There may be an optical interconnect based on it as well as a copper one. The PCI Sig says it will meet the high speed data movement needs of hyperscale data centers with 800 GbitE, high-performance computing and military/aerospace and even automotive application areas. It’s envisaged that products using PCIe 7.0 will appear 12 to 18 months after the specification is released, and it may not appear in PC-level and below products as it is so fast and expensive.
Initial work has begun on a PCIe 8.0 spec and it’s assumed it will double the transfer rate of PCIe 7.0 while still using PAM4 and Flit. PCIe 8.0 might need optical inks to overcome copper wire’s frequency-dependent signal losses. It might also need a PAM8 scheme, with 3 bits per signal, to increase its data rate, and this would require better signal-to-noise detection and error correction.
It’s been suggested that PCIe speed could reach 1 Tbps per lane beyond v8.0, about 125 GBps, perhaps with a PAM16 (4 bits/signal) scheme and optical interconnects.
There are a number of PCI SIG videos here if you wish to delve deeper into PCIe matters.
