In Lecture 14, we saw how a single port decides which Virtual Channel (VC) gets to transmit first. But what happens when packets arriving from multiple different ingress ports all want to use the exact same outgoing Virtual Channel on an egress port?
To prevent collisions and ensure fair access, PCI Express uses Port Arbitration. In this lecture, we will explore how switches and Root Complexes manage intersecting traffic and dive deep into the time-based schemes required for isochronous data delivery.
Part 1: Where Does Port Arbitration Happen?
Port arbitration acts as the traffic cop for intersecting data streams. It is required in several specific places within a PCIe topology:
- Egress ports of switches: When multiple devices attached to a switch try to send data out of the same upstream port.
- Root Complex ports: When peer-to-peer transactions between different Root ports are supported.
- Root Complex egress to Main Memory: When packets from multiple ingress ports try to access system memory simultaneously. Because this connection doesn’t use a standard PCIe port, it utilizes a vendor-specific set of registers called a Root Complex Register Block (RCRB) to handle the arbitration.
Note: Port arbitration is managed independently for each VC, meaning a separate arbitration table is required for each VC that supports it.
Part 2: Two Categories of Service
Port arbitration allows the PCIe fabric to support a range of service levels, which broadly fall into two categories:
- Asynchronous Service: This provides “best effort” delivery. Mundane background tasks and devices like mass storage drives do not have strict bandwidth or latency requirements, so they do not need special timing mechanisms.
- Isochronous Service: This is the elite tier. When a device (like a high-definition video camera or an audio interface) absolutely requires guaranteed bandwidth and predictable latency, it needs an isochronous connection. This ensures that time-sensitive data is delivered reliably without preventing other regular traffic from using the Link.
Part 3: Port Arbitration Mechanisms
Just like VC arbitration, configuration software can select from several port arbitration schemes depending on the device’s capabilities:
- Hardware-Fixed Arbitration: A hardwired scheme chosen by the designer (like a simple round-robin) that requires no software setup but does not support isochronous services.
- Weighted Round Robin (WRR): Software assigns different “weights” to different ports, giving higher-priority ports more opportunities to transmit as the arbiter scans through a phase table.
Part 4: Time-Based Weighted Round Robin (TBWRR)
While standard WRR is great for prioritizing traffic, it cannot guarantee when a packet will be delivered. To support isochronous traffic, PCIe relies on Time-Based Weighted Round Robin (TBWRR).
As the name implies, TBWRR adds the critical element of time to each arbitration phase.
- How it Works: Just like regular WRR, the arbiter looks at a table to see which port is allowed to send a packet during the current phase. However, instead of immediately advancing to the next phase after the packet is sent, the time-based arbiter waits until the current virtual timeslot elapses before moving on.
- Enforcing Regular Intervals: If the selected port does not have a packet ready to send, the arbiter simply waits; nothing else is sent until the next timeslot begins. This strict pacing ensures that transactions are accepted from the ingress port buffer at perfectly regular, predictable intervals.
- The Timeslot: Currently, the length of a single virtual timeslot is defined as 100ns. A TBWRR table can support up to 128 phases, meaning a complete cycle through the table (the overall Service Interval) takes exactly 12.8 microseconds.
By combining Traffic Classes, Virtual Channels, Strict Priority VC Arbitration, and TBWRR Port Arbitration, a PCIe system can guarantee that a video stream or audio feed will never stutter or drop packets, regardless of what else is happening on the bus!