In Lecture 18, we saw how the strict transaction ordering rules protect the Producer/Consumer model from fatal race conditions. However, strictly enforcing strong ordering can also create unnecessary traffic jams.
What happens when a completely unrelated packet gets stuck behind a blocked transaction? In Lecture 20, we will learn how PCIe solves this bottleneck using Relaxed Ordering (RO), allowing switches to safely reorder transactions to maximize performance without violating data delivery models!
Part 1: The Problem with Strict Ordering
The ordering rules that support the Producer/Consumer model are vital, but they can result in transactions being blocked even when they have absolutely no relation to the Producer/Consumer sequence.
Imagine a scenario where a posted memory write is delayed inside a switch because the target device’s buffer space is temporarily unavailable. Under strict ordering rules, every single subsequent transaction entering that queue must wait until the blockage clears and the write is delivered. If the packets stuck in line are unrelated to the blocked transaction, this creates an unnecessary stall and severely degrades Link efficiency.
Part 2: The Relaxed Ordering (RO) Attribute Bit
To alleviate this problem, PCI Express allows a transaction to have its Relaxed Ordering (RO) attribute bit set. Located at bit 5 of byte 2 in the 32-bit TLP header, this bit acts as a “VIP pass” for the packet.
- Software is in Control: The RO bit is only set if the device driver software explicitly enables it, verifying that this specific transaction has no dependencies on previously sent transactions.
- A Permission Slip for Switches: When a Switch or Root Complex sees a packet with the RO attribute bit set, it is officially granted permission to safely route this packet ahead of older, blocked traffic. (Note: Hardware is permitted to reorder the packet, but it is not strictly required to do so)
Part 3: How RO Affects Different Transactions
While the RO bit gives permission to bypass the line, switches still follow highly specific rules depending on the type of packet to ensure the Producer/Consumer model is never broken.
1. Memory Writes and Messages (Posted Requests) Memory writes and Messages are both posted writes and are subject to the same standard ordering requirements. When the RO bit is set:
- Switches are allowed to reorder these newly posted transactions ahead of previously posted memory writes or messages.
- The Root Complex is permitted to write the bytes of an RO-enabled packet to memory in any address order, and can reorder them to avoid busy memory areas.
2. Memory Read Requests All read transactions in PCIe are handled as split transactions (a Request followed later by a Completion). Relaxed Ordering handles these two halves very carefully:
- The Request Cannot Pass: When a switch receives a Memory Read Request with the RO bit set, it forwards the request in order and must not reorder it ahead of previously posted memory writes. This guarantees that all older writes are pushed ahead of the read—a vital “flushing” action that software relies on for the Producer/Consumer model to work.
- The Completion CAN Pass: When the target Completer fetches the data, it returns one or more split Completions that inherit the exact same RO bit setting from the original request. As these Completions travel back, switches are allowed to reorder them ahead of previously posted memory writes moving in the same direction.
By allowing the returning data (Read Completions) to bypass stalled writes, Relaxed Ordering massively improves read performance while flawlessly maintaining the integrity of the Producer/Consumer sequence!