Like many modern serial transport buses, PCI Express (PCIe) improves transport efficiency by utilizing a strict credit-based flow control scheme. To ensure that a transmitter never sends a Transaction Layer Packet (TLP) to a receiver that lacks the buffer space to accept it, devices constantly communicate their available capacity.
The Data Link Layer handles this continuous exchange of flow control credit information using specialized, localized messengers called Data Link Layer Packets (DLLPs).
Here is a look at the specific DLLPs used to manage these credits and how they keep traffic flowing smoothly across the link.
The Three Phases of Flow Control Signaling
Flow control communication relies on three distinct types of DLLPs, which are used during different phases of the link’s operation:
- InitFC1 (Initialization Flow Control 1): Sent during the initial link training process to establish the baseline flow control credits.
- InitFC2 (Initialization Flow Control 2): Also sent during the initialization process to confirm and transition the flow control mechanism into its active state.
- UpdateFC (Update Flow Control): Used during normal operation to manage runtime credit management. As a receiver processes incoming packets and empties its receive buffers, it periodically sends UpdateFC DLLPs to the transmitter to announce that new buffer space has been recovered.
Accounting for Different Traffic Types
Because PCIe routes different types of traffic independently to prevent bottlenecks, flow control credits are tracked separately based on the type of transaction. Therefore, each of the three DLLPs listed above comes in three distinct variants, identifiable by their Type field code:
- -P (Posted Requests): Tracks credits for memory writes and messages.
- -NP (Non-Posted Requests): Tracks credits for memory reads and I/O requests.
- -Cpl (Completions): Tracks credits for returning read data.
Inside the Flow Control DLLP Format
Like all DLLPs, flow control packets are strictly local traffic with a fixed size of exactly 8 bytes. The first four bytes of the packet (the core) are densely packed with the specific credit information:
1. DLLP Type Field & Virtual Channel The first byte identifies exactly what the packet is doing. It contains a 4-bit code determining the DLLP type (for example, 1000b indicates an UpdateFC for Posted Requests). This byte also includes a 3-bit Virtual Channel ID (VC ID), which specifies which specific Virtual Channel (from VC 0 to 7) is being updated with the new credits.
2. Header Flow Control (HdrFC) When a receiver allocates buffer space, it tracks the packet’s header separately from its data payload. The DLLP contains an 8-bit HdrFC field to communicate the available header credits. Each credit in this field represents enough buffer space for exactly one TLP Header, plus an optional TLP Digest (ECRC).
3. Data Flow Control (DataFC) The DLLP also contains a 12-bit DataFC field to communicate how much room is left for actual data payloads. Every single credit in the DataFC field represents exactly 16 bytes of data storage.
4. 16-Bit CRC To ensure that a transient bit error doesn’t cause a transmitter to miscalculate available buffer space, the final two bytes of the core are protected by a 16-bit Cyclic Redundancy Check (CRC).
Summary
Without flow control DLLPs, high-speed PCIe communication would risk constantly dropping packets due to receiver buffer overflow. By utilizing InitFC1, InitFC2, and UpdateFC packets, the Data Link Layer maintains a seamless, hardware-based accounting system that guarantees transmitters always know exactly how much data their neighbors can handle.