Chapter 6.4 – Flow Control Initialization and Update Mechanism in PCI Express

How credits are exchanged, refreshed, and maintained using DLLPs

1 . Introduction

After link training completes and both ends enter the L0 state, PCI Express devices must synchronize their buffer availability before any TLPs can be exchanged.

This is achieved through Flow Control Initialization — a process in which each receiver advertises how many header and data credits it can accept, and each transmitter stores those values locally.

Once traffic begins, Flow Control Update DLLPs are used to replenish credits as the receiver processes packets.

Together, these two steps — Initialization and Update — maintain the balance of credit-based communication throughout the life of the link.

2 . Flow Control Initialization – The Starting Point

When It Happens:

After the link reaches L0 (active state) during LTSSM (Link Training and Status State Machine), both link partners exchange Initialization Flow Control DLLPs.

Purpose:

To inform the transmitter about:

• How many TLPs (by type) can be sent initially.
  
• The size of the receiver’s buffers per credit category.
  

3 . Flow Control DLLP Overview

A Flow Control DLLP is a Data Link Layer Packet used exclusively for credit exchange.
It is not visible to the Transaction Layer and does not carry user data.

4 . Flow Control Initialization DLLP Contents

During link bring-up, each receiver sends its available credit values:

Each of these values represents the number of packets or data doublewords the receiver can currently accept for that transaction type.

💡 All six credit fields are always present during Initialization DLLP exchange.

5 . Initialization Flow

Let’s walk through the sequence step-by-step:

This exchange ensures that both sides always know how much data they can safely send.

6 . Flow Control Update DLLP

After the link begins operation, credits are continuously updated.
A Flow Control Update DLLP carries the latest available buffer counts at the receiver.

When Updates Are Sent

• Whenever the receiver frees buffer space by processing TLPs.
  
• Periodically if credits change frequently.
  
• Optionally, when the receiver’s available credits drop below a threshold.
  

7 . Example – Initialization and Update Sequence

Let’s take a real example between a Root Complex (RC) and an Endpoint (EP):

Step 1: Initialization

• EP advertises:
  PH=8, PD=32, NPH=4, NPD=8, CPLH=8, CPLD=24
  
• RC advertises:
  PH=16, PD=64, NPH=8, NPD=16, CPLH=8, CPLD=16
  

Each side stores the other’s credit values as their starting credit counters.

Step 2: Transmission

• RC sends Memory Write (Posted) → consumes 1 PH, 4 PD.
  
• RC sends Memory Read (Non-Posted) → consumes 1 NPH.
  

Step 3: Receiver Processes

• EP processes these packets, freeing buffers.
  

Sends Flow Control Update DLLP:

PH=8, PD=32, NPH=4, NPD=8, CPLH=8, CPLD=24

This replenishes credits at RC, allowing continued transmission.

8 . Handling Multiple Virtual Channels (VCs)

Each VC has its own set of credits and independent DLLPs.
If VC1 is enabled (e.g., for isochronous data), then Flow Control DLLPs carry the corresponding VC identifier.
This separation ensures:

• No interference between traffic classes.
  
• Isolated flow control per VC.
  

9 . Flow Control DLLPs vs Ack/Nak DLLPs

Both coexist in the Data Link Layer to provide safe and reliable transmission.

10 . Design Insights

In RTL design:

• Credit counters are initialized after the first Flow Control DLLP is received.
  
• Tx Scheduler checks these counters before sending new packets.
  
• Rx Path triggers a Flow Control Update when buffer occupancy drops below a programmable threshold.
  

In simulation:

 assert(init_done && credits_valid) 
 else $fatal("Flow control not initialized before transmission!");

11 . Debug Tip

When analyzing PCIe traces:

• Initialization Flow Control DLLPs appear at the start of L0 state.
  
• Flow Control Update DLLPs appear periodically during traffic.
  
• If transmission stalls, check for missing update DLLPs or credit starvation — both are common root causes of link pauses.
  

NOTES

• Flow Control DLLPs are special link-layer packets for credit exchange and synchronization.
  
• Initialization DLLPs set the starting buffer capacities at L0 entry.
  
• Update DLLPs dynamically restore credits as buffers free up.
  
• Each direction maintains six counters (PH, PD, NPH, NPD, CPLH, CPLD) per VC.
  
• Missing or incorrect Flow Control DLLPs can cause link stalls or deadlocks.