In our previous lectures, we saw how devices use Base Address Registers (BARs) and Base/Limit registers to claim address space. Because PCIe utilizes independent point-to-point connections instead of a legacy shared bus, all traffic must be actively directed through the system’s topology.
When a Transaction Layer Packet (TLP) arrives at a device’s inbound interface (the ingress port), the device must evaluate the packet and make one of three decisions: accept it, forward it to an outbound port, or reject it. Devices with multiple ports, like Switches and Root Complexes, act as the system’s active routing elements to ensure packets reach their final destinations.
To make these routing decisions, the hardware relies entirely on the TLP header. Every TLP contains a 3-Doubleword (12-byte) or 4-Doubleword (16-byte) header. The Format and Type fields within this header define the packet’s content and explicitly dictate the routing method that the fabric must use.
Here is a look at the two primary routing mechanisms: ID Routing and Address Routing.
1. ID Routing: Targeting Logical Positions
ID routing is used to target the exact logical position of a function within the PCIe topology. It routes the packet using the unique Bus Number, Device Number, and Function Number (BDF) of the target.
- When is it used? ID routing is exclusively used for Configuration requests, Completion packets, and certain Message packets.
- How Endpoints Handle It: Because an Endpoint only has one link, its job is simple. It checks the target ID field in the packet header against its own BDF. If it is a match, the Endpoint consumes the packet.
- How Switches Handle It: Switch ports must perform two distinct checks. First, the port checks if the target ID matches its own internal BDF; if so, the port consumes the packet itself. If it is not a match, the port checks its Secondary and Subordinate Bus Number registers. If the target bus number falls anywhere within this downstream range, the switch forwards the packet down to its secondary bus. If it does not match either condition, the packet is rejected or forwarded back upstream.
2. Address Routing: Targeting Memory and IO Space
Rather than looking for a specific device, Address routing targets a specific physical location in the system’s Memory or IO maps.
- When is it used? Address routing is strictly used for Memory Read/Write requests, IO Read/Write requests, and Atomic Operations.
- 32-bit vs. 64-bit Headers: If a request targets IO space or a memory address below the 4GB boundary, it uses a standard 32-bit address and requires a 3-Doubleword (3DW) header. However, if the memory request targets an address above the 4GB boundary, it utilizes a 64-bit address and requires an expanded 4-Doubleword (4DW) header.
- How Endpoints Handle It: When an Endpoint receives an address-routed TLP, it simply checks the target address in the header against its own programmed Base Address Registers (BARs). If the address falls within a range claimed by any of its BARs, it accepts the packet.
- How Switches Handle It: When a Switch port evaluates an address-routed TLP moving downstream, it first checks if the address matches its own internal bridge BARs. If it does not, the port evaluates its Base and Limit register pairs (specifically the IO, Non-Prefetchable Memory, and Prefetchable Memory registers). If the target address falls securely within any of the programmed Base/Limit ranges, the Switch knows the target lives somewhere beneath it and forwards the packet to its downstream interface. If it falls outside these boundaries, it handles it as an Unsupported Request.