In Lecture 7, we introduced the concept of the credit-based flow control mechanism that prevents Link congestion. Now, in Lecture 8, we are going to look under the hood at the specific architecture of the hardware buffers that make this system possible.
Flow control buffers are implemented for each Virtual Channel (VC) resource supported by a port. To keep traffic highly organized and prevent different types of packets from blocking one another, each VC Flow Control buffer is meticulously split into distinct categories.
Here is how a PCIe receiver manages its incoming traffic:
1. The Three Main Transaction Categories
Within every Virtual Channel, flow control buffers are separated based on the category of the transaction flowing through them:
- Posted Transactions (P): This buffer category handles Memory Writes and Message Requests.
- Non-Posted Transactions (NP): This buffer category manages transactions that require a response, including Memory Reads, Configuration Reads and Writes, as well as I/O Reads and Writes.
- Completions (CPL): This buffer category is strictly reserved for the Read and Write Completions that return in response to Non-Posted requests.
2. Splitting Buffers into Header and Data Portions
Simply categorizing by transaction type is not enough. Because some transactions (like read requests) consist of just a header, while others (like write requests) consist of both a header and a data payload, the receiver must track header and data space separately.
Therefore, each of the three transaction categories is further split into Header and Data portions. This results in six distinct flow control buffers per Virtual Channel:
- PH: Posted Request Header
- PD: Posted Request Data
- NPH: Non-Posted Request Header
- NPD: Non-Posted Request Data
- CPLH: Completion Header
- CPLD: Completion Data
Before a transmitter can send a transaction that includes data, it must verify that the receiver has sufficient credits available in both the appropriate Header buffer and the Data buffer. Furthermore, to maintain proper transaction ordering rules, the receiver must track the exact order of the header and data components within these buffers.
3. Understanding Flow Control Credits
Buffer space is measured and reported in units called Flow Control credits. Because headers and data payloads are vastly different in size, their credit unit values are defined differently:
- Header Credits: One header credit is equal to the maximum header size plus an optional End-to-End CRC (ECRC) digest. This equates to 5 Double Words (DWs) for Request headers (Posted and Non-Posted) and 4 DWs for Completion headers.
- Data Credits: One data credit always represents exactly 4 DWs (which is 16 aligned bytes) of payload data.
These credit values are constantly communicated back to the transmitter using Data Link Layer Packets (DLLPs), which themselves do not consume any Flow Control credits because they originate and terminate at the Link layer rather than the Transaction layer.