As we shift our focus to how data actually moves across the modern PCI Express (PCIe) serial interconnect, it is essential to understand how the system organizes communication. To manage the complexity of high-speed serial transfers, PCIe defines a highly structured layered architecture.
These layers can be logically split into two independent parts: a transmit side for outbound traffic and a receive side for inbound traffic. As outbound data moves down through the layers, each layer appends specific information to form a complete packet. Conversely, as an inbound packet moves up through the receiving device, each layer checks for errors, strips away its specific layer information, and passes the core data upward.
Here is an executive overview of the responsibilities of each layer in the PCIe architecture.
1. The Device Core (The Software Layer)
While not officially defined as a “layer” in the PCIe specification, the Device Core is the brain of the device—whether it is a network interface, hard drive controller, or Root Complex—and sits at the very top of the hierarchy.
- Transmit Function: It provides the Transaction Layer with all the core information needed to build a request, such as the transaction type, the target address, and the amount of data to transfer.
- Receive Function: It serves as the ultimate destination for the data and information extracted from incoming packets forwarded up by the lower layers.
2. The Transaction Layer
Sitting just below the Device Core, the Transaction Layer is responsible for the creation and decoding of Transaction Layer Packets (TLPs).
- Packet Assembly: On the transmit side, it takes the core information provided by the device (such as Memory, IO, Configuration, or Message requests) and assembles the base TLP. It optionally appends an End-to-End CRC (ECRC) to help the final destination verify that the packet wasn’t corrupted anywhere along its journey through the PCIe fabric.
- Traffic Management: Before passing the newly created packet down to the next layer, the Transaction Layer stores it in a buffer called a Virtual Channel. This layer is actively responsible for managing Quality of Service (QoS), maintaining strict Transaction Ordering, and implementing Flow Control logic to ensure the receiving device has enough buffer space before sending the packet.
3. The Data Link Layer
The Data Link Layer serves as the highly reliable middleman, ensuring that packets successfully survive their physical trip across the Link.
- Reliable Delivery (Ack/Nak): When the Transaction Layer passes a TLP down, the Data Link Layer appends a unique Sequence Number and a Link CRC (LCRC). It then stores a backup copy of this packet in a local Replay Buffer. If the neighboring receiver detects an error, it will issue a “Nak” (Negative Acknowledgement), prompting this layer to seamlessly replay the packet until it is successfully received.
- Link Management: Aside from handling TLPs, this layer also generates and consumes its own small 8-byte packets called Data Link Layer Packets (DLLPs). These DLLPs are never routed through the system; they are strictly used between two direct neighbors to communicate Flow Control updates, Ack/Nak confirmations, and power management states.
4. The Physical Layer
The Physical Layer is the lowest hierarchical layer and is responsible for actually moving the bits across the wire. It is conceptually divided into two parts: the Logical Physical Layer and the Electrical Physical Layer.
- Logical Processing: When a packet arrives from the Data Link Layer, the Physical Layer adds “framing” characters (like Start and End symbols) so the receiver can easily detect packet boundaries. It then performs byte striping (splitting the bytes across all available Lanes), scrambles the data to reduce electromagnetic interference, and encodes it. PCIe Gen1 and Gen2 use an 8b/10b encoding scheme, while Gen3 utilizes a more efficient 128b/130b encoding method.
- Electrical Transmission: Finally, the encoded bits are serialized and clocked out as differential signals over the physical Lanes of the Link at speeds of 2.5 GT/s, 5.0 GT/s, or 8.0 GT/s.
- Link Training: The Physical Layer is also responsible for automatically initializing and training the Link using special un-routed signals called Ordered Sets. This process resolves crucial connection variables like Link width, data rate, bit lock, and Lane-to-Lane de-skew.