PCI Express protocol primer
Networking, storage, industrial, and consumer electronics applications are avoiding slowest link limitations by moving to switched serial architectures.
The MicroTCA standard supports multiple switch fabrics, such as Serial RapidIO, PCI Express, and 10 Gigabit Ethernet (GbE). This article will describe the PCI Express protocol and highlight the reasons why it is becoming increasingly popular with these systems.
Although originally targeted at computer expansion cards and graphics cards, PCI Express is now also being extensively adopted for use in a broader range of applications, including networking, storage, industrial, and consumer electronics. A key difference between PCI Express and earlier PC buses is that it has a topology based on point-to-point serial links, rather than a shared parallel bus architecture.
PCI Express maintains backward software compatibility to PCI so that drivers and operating system software can be reused. PCI to PCI Express bridges can be incorporated on an Advanced Mezzanine Card (AdvancedMC), allowing reuse of existing PMC card designs. As new controllers are released with PCI Express, MicroTCA can add this new functionality while maintaining more traditional I/O or custom PCI based designs in the same chassis. Also, migrating to a switched serial architecture like PCI Express eliminates the slowest link limitations.
The PCI Express protocol consists of a Transaction Layer, a Link Layer, and a Physical Layer. Within each layer, there are corresponding sublayers. The Link Layer consists of the Media Access Control Sublayer. The Physical Layer consists of the Logical and Electrical Sublayers. The Logical Sublayer of the PHY contains a Physical Coding Sublayer (PCS), which encodes/decodes each 8-bit data-byte to a 10-bit code. The Electrical Sublayer implements the analog components including the transceiver, the analog buffers, the Serializer/Deserializer (SerDes) and the 10-bit interface.
PCI Express is a packet-based serial connectivity protocol that is estimated to be 10x more complex than PCI’s parallel bus. This complexity is due in part to the requisite parallel-to-serial data conversion at gigahertz speeds and the move to a packet-based implementation. PCI Express maintains the basic load-store architecture of PCI, including support for split transactions that was added by PCI-X. In addition, it introduces a number of low-level message-passing primitives to manage the link (such as link-level flow control) to mimic the sideband wires of the traditional parallel bus and deliver higher levels of robustness and functionality.
The PCI Express specification defines many features that support both today’s needs and future expandability, while maintaining software driver compatibility with PCI. Advanced features of PCI Express include: autonomous power management; advanced error reporting; end-to-end reliability via End-to-end Cyclic Redundancy Checking (ECRC), hot plug support; and Quality of Service (QoS) traffic classes.
The topology of a simplified system, consisting of the four function types – the root complex, switch, endpoints, and bridge – is shown in Figure 1. Each of the dotted lines represents a connection (called links) between two PCI Express devices.
PCI Express protocol definitions
The protocol as defined in the PCIe specification adheres to the Open Source Initiative (OSI) model. It is partitioned into seven principal layers, as shown in Figure 2.
The Physical Layer is partitioned into two sublayers: the Electrical Layer and the Logical Layer. A number of companies have defined and utilize an interface between the Electrical Layer and the Logical Layer called the Physical Interface for PCI Express (PIPE). The PIPE interface enables the design to a standard interface and/or the purchase of multiple components that will work together, even from different vendors.
n The Electrical Sublayer of the Physical Layer implements the analog components including the transceiver, the analog buffers, SerDes, and the 10-bit interface.
n The clocking information is embedded in the signal. At the physical level (layer one, Figure 2), PCI Express utilizes 8b/10b encoding to ensure that strings of consecutive ones or consecutive zeros are limited in length. This is necessary to prevent the receiver from losing track of where the bit edges are. In this coding scheme, every 8 (uncoded) payload bits of data are replaced with 10 (encoded) bits of transmit data. This coding feature not only checks for valid characters; it also limits the difference between the number of zeros and ones transmitted, thus maintaining a DC balance at both the transmitter and receiver and significantly enhancing electromagnetic compatibility (EMC) and electrical signal performance.
n The other side of the PIPE interface is the Physical Layer, which contains the Link Training and Status State Machine (LTSSM), lane-to-lane de-skew, special sequence detection and generation, and the like.
The Data Link Layer ensures reliable data exchange, error detection via a 32-bit Cyclic Redundancy Code (CRC) and an acknowledgment protocol (ACK/NACK signaling) and retry, flow control credit (FCC) initialization and update, and power management services. To accomplish these functions, the Data Link Layer generates and processes Data Link Layer Packets (DLLP).
PCI Express implements split transactions (transactions with request and response separated by time), allowing the link to carry other traffic while the target device gathers data for the response.
The Transaction Layer (layers three and four in Figure 2) creates outbound and receives inbound Transaction Layer Packets (TLPs). A Transaction Layer Packet includes a header, an optional data payload, and an optional ECRC. The TLP is either a request or a response to a request (completion) and is always a multiple of 4 bytes (1 DWORD). The header specifies the transaction type, priority, address, routing rule, and other packet characteristics. The transmit Transaction Layer builds packet headers and optionally adds ECRC and gates packet transmission until sufficient remote flow control credits are available. The receive Transaction Layer checks TLP format and headers. It also optionally checks ECRC.
The PCI Express specification defines many features that support both today’s needs and future expandability for MicroTCA systems. Advanced features that make PCI Express a robust port include: autonomous power management; advanced error reporting; end-to-end reliability via end-to-end cyclic redundancy checking; hot plug support; and QoS traffic classes. These features will continue to drive the popularity of PCI Express in MicroTCA systems.
Navraj Nandra joined Synopsys in February 2005 as Director of Product Marketing for the mixed-signal products that include SERDES, USB, and DDR2. He has worked in the semiconductor industry since the mid 80's as an analog/mixed signal IC designer for Philips Semiconductors, austriamicrosystems, (San Jose & Austria) and EM-Marin (Switzerland). He has been responsible for the complete design of a number of analog front ends in application areas such as digital audio, RFID and automotive. He joined Synopsys from Barcelona Design where he was Director of Application Engineering. During his four years at Barcelona he was responsible for pre- and post-sales support for Barcelona's analog synthesis technology. Navraj holds a masters degree in Microelectronics, majoring in analog IC design, from Brunel University and a post-graduate diploma in Process Technology from Middlesex University. He has presented at numerous technical conferences on mixed-signal design, analog IP and analog synthesis/EDA.
 Bringing up to PCI Express from PCI, Intel Corporation White Paper, http://download.intel.com/design/bridge/papers/25375501.pdf