Q 0.9 : Flow Control Initialization – Preparing the Link for Traffic

In Lecture 8, we explored how Flow Control buffers are physically organized into Posted, Non-Posted, and Completion categories. But before a single Transaction Layer Packet (TLP) is ever allowed to cross the Link, the two connected devices must officially agree on how much buffer space is available.

In Lecture 9, we are walking through the Flow Control Initialization sequence, detailing the FC_INIT1 and FC_INIT2 states, and uncovering the rules behind credit advertisements.

Part 1: The Pre-Requisite to Initialization

Flow control initialization involves a handshake between the devices at each end of a Link. This automated process begins the moment the physical wire is ready—specifically, when the Physical Layer finishes link training and alerts the Data Link Layer by asserting the LinkUp signal.

Initialization is controlled by the Data Link Control and Management State Machine (DLCMSM). It is important to note that Virtual Channel 0 (VC0) is always enabled by default, meaning its initialization happens automatically so that configuration transactions can immediately begin traversing the system. If a device supports additional VCs (VC1-VC7), their initialization is held back until configuration software explicitly sets them up.

Part 2: The FC_INIT1 State

Once the Link is physically up, the state machine transitions into the DL_Init state, which kicks off the first initialization phase: FC_INIT1.

During this state, both devices continuously broadcast InitFC1 Flow Control DLLPs across the Link to advertise the size of their receive buffers.

• The Rule of Order: These DLLPs cannot be sent randomly. They must be broadcast in a strict, repeating sequence: Posted, Non-Posted, and then Completions.
• Moving On: A device will stay in this state until it has successfully broadcast its own values and received the complete sequence of InitFC1 packets from its neighbor enough times to ensure accuracy. It then records these neighbor credit values into its own transmit counters, sets an internal flag, and moves to the second phase.

Part 3: The FC_INIT2 State

In the FC_INIT2 state, the device shifts from broadcasting InitFC1 packets to continuously sending InitFC2 DLLPs.

• Confirmation of Success: InitFC2 packets contain the exact same credit information and are sent in the exact same sequence as phase one, but they serve an additional purpose: they confirm to the neighbor that flow control initialization was successfully completed at the sender.
• Why Two Phases? This two-step process is incredibly clever. Neighboring devices will often finish their FC_INIT1 processing at slightly different times. The FC_INIT2 state ensures that if one device finishes early, it keeps broadcasting its buffer sizes so the slower “late” device can still get the flow control information it needs.
• Link Up! Once both sides have sent their InitFC2 packets, the state machine transitions to DL_Active, the Link Layer signals DL_Up to the Transaction Layer, and TLPs are finally allowed to flow,!

Part 4: Minimum, Maximum, and Infinite Credits

During this initialization sequence, devices advertise their buffer space in units called “credits.” The PCIe specification enforces specific rules on what values can be advertised:

• Minimum Advertisements: The specification sets strict minimums to ensure basic functionality. For example, the minimum advertisement for Request Headers (Posted and Non-Posted) is just 1 credit, while the minimum for a Posted Data buffer must at least equal the largest possible setting of the device’s Max_Payload_Size.
• Maximum Advertisements: While devices typically advertise much more than the bare minimum, there is a ceiling. The maximum allowed advertisement for Request Headers is 128 units, and the maximum for Posted Request Data is 2048 units.
• Infinite Credits: If a device advertises a flow control value of 00h during initialization, it is declaring infinite credits. A receiver that advertises infinite credits is effectively making a hardware guarantee that its buffer will never overflow.
  • Special Use Case: Infinite credits are commonly used in systems that implement only VC0. Because Non-Posted writes (like IO or Configuration writes) are strictly limited to VC0, a device can safely advertise “infinite” credits for Non-Posted Data on VC1 through VC7, since those specific buffers will literally never be used.