In Lecture 18, we saw exactly why transaction ordering is critical by looking at the Producer/Consumer model. Now, it is time to look at the official rulebook.
In this lecture, we will decode the Simplified Ordering Rules Table. We will learn exactly when Transaction Layer Packets (TLPs) are strictly forbidden, explicitly required, or optionally allowed to pass one another as they travel through the PCIe fabric.
Before we look at the rules, remember that ordering relationships only apply to packets that share the exact same Traffic Class (TC). Packets with different TCs are considered unrelated and have no ordering relationship.
Part 1: The Three Categories of TLPs
To understand the rules, we first group all PCIe transactions into three main categories:
- Posted Requests: Memory Writes and Message Requests.
- Non-Posted Requests: Memory Reads, I/O Reads/Writes, and Configuration Reads/Writes.
- Completions: Both standard Completions and Completions with Data.
When these packets queue up in a switch or Root Complex buffer, the ordering table dictates their behavior based on three distinct commands: “No”, “Yes”, and “Y/N”. Let’s break down what each of these means.
Part 2: The “No” Rules (Strict Enforcement)
When the ordering table says “No,” it means the newly arrived transaction is absolutely not allowed to pass the previously queued transaction. These rules exist to enforce the Producer/Consumer model and ensure data integrity:
- Never Pass a Posted Request: To protect the Producer/Consumer model, a subsequent transaction (whether it is another Posted Request, a Non-Posted Request, or a Completion) is strictly forbidden from passing a previously queued Posted Request. This guarantees that your data (Memory Write) always arrives before your Flag update! (Note: There are exceptions if Relaxed Ordering or ID-Based Ordering are enabled, which we will cover in later lectures).
- Completions with the Same ID: If multiple Completions are returning data for the exact same original request (meaning they share the same Transaction ID), they are not allowed to pass each other. This ensures that the pieces of a large data payload arrive in perfect, ascending address order.
Part 3: The “Yes” Rules (Deadlock Avoidance)
When the ordering table says “Yes,” it means the newly arrived packet must be allowed to pass the older packet. If a switch or bridge refuses to let the packet pass, a fatal system deadlock could occur.
Here is when passing is strictly required:
- Bypassing Non-Posted Requests: If a Non-Posted Request (like a Memory Read) is stuck in a buffer due to a lack of flow control credits, later-arriving Posted Requests (Memory Writes) and Completions MUST be allowed to pass it. If they were forced to wait behind the blocked read request, the entire system could freeze.
- PCI/PCI-X Bridge Deadlocks: In the specific case of a PCIe-to-PCI/PCI-X bridge, if a transaction is moving from PCIe to the legacy bus, a Posted Request must be allowed to pass a Completion to avoid a known legacy deadlock scenario.
Part 4: The “Y/N” Rules (Weak Ordering)
When the ordering table says “Y/N” (Yes/No), it means the newly arrived transaction is allowed to pass the older transaction, but it is not required to do so.
This is the foundation of Weak Ordering. It gives device designers the flexibility to optimize their hardware. For example:
- Non-Posted Requests are permitted (Y/N) to pass other Non-Posted Requests.
- Completions with entirely different Transaction IDs are permitted (Y/N) to pass one another.
- A Posted Request is permitted (Y/N) to pass a Completion.
Because these specific combinations do not risk breaking the Producer/Consumer model or causing deadlocks, the hardware can safely shuffle them around to keep traffic flowing smoothly and maximize Link efficiency.