Challenges of 40 Gbps AdvancedTCA backplanes

Achieving 40 Gbps performance demands diligent Signal Integrity efforts, but does it also require costly materials?

4“It’s only a number,” we’re told when we hit 40, but the AdvancedTCA backplane won’t be hearing from its friends that the number 40, when it steps right in front of gigabits per second is “only” anything. Far from standing alone, this is a number that depends on, among other things, five “informative” parameters, as Ovidiu explains here.

Much has been written in the past couple of years about the jump to the “next generation” speeds in data transmission over electrical paths. No sooner do certain new specifications go to ballot for approval, than the industry begins buzzing about yet another faster data rate for standards bodies to address. But, what speeds are practical today with reasonably cost-effective materials for board and backplane vendors?

First came 10 Gbps

As 40 Gbps over AdvancedTCA kicks in, we’ll focus on this initiative as announced and publicized by the greater AdvancedTCA community. In the 2003-2008 timeframe the focus was on achieving 10 Gbps – first by bundling together four lanes of 3.125 Gbps and then with the use of a single lane (port). The current standards addressing 10GBase-KX4 and 10GBase-KR are:

·        IEEE 802.3-2008 in the informative Annex 69B of section 5 – released

·        PICMG 3.1 revision 2 – in the works

The current standards addressing the bundling of four 10 Gbps lanes into one 40 Gbps channel (40GBase-KR4) are:

·        IEEE 802.3ba-2010 standard – released as an amendment to IEEE 802.3-2008

·        PICMG 3.1 revision 2 – in the works


Now there is talk about achieving 20-25 Gbps over one electrical lane with the goal of making 80-100 Gbps a reality by the end of 2012! The CEI-25G-LR specification is coming fast, according to industry analysts and insiders, and concomitantly IEEE is addressing 100 Gbps with its own “Backplane and copper cable study group.” Once the limits are defined (and met) for 10 Gbps operation over one pair/lane (or backplane channel), 40 Gbps operation (over four such pairs) can be implied, provided crosstalk and skew limits are met.

Food for thought when it comes to speed

With the ever-increasing speed levels being discussed, what is the baseline to assess their performance? What are the parameters that should be met, and what limits should we put on them? IEEE and other standards define the channels parameters to be met as:

·        Insertion Loss (IL)

·        Fitted Attenuation (FA)

·        Insertion Loss Deviation (ILD)

·        Return Loss (RL)

·        Insertion loss to Crosstalk Ratio (ICR)

·        Bit Error Rate (BER)

Of all these parameters in IEEE 802.3-2008 only BER is mandatory, while the first five (IL, FA, ILD, RL, and ICR) are only given informatively in the specification. However, the “informative” parameters are still important to a design’s success. In fact, industry reports show cases of backplane channels that fail at some point one or more of the five informative parameters but meet the BER target successfully.

Going beyond adherence just to BER, which is typically 10-12, although 10-15 is not excluded, requires answering questions that include:

·        Are the existing IEEE 802.3-2008 informative limits based on current PHY performance or rather on older PHYs?

·        Should one define less severe limits or more severe limits, depending on the capabilities of PHYs?

·        Does ICR sufficiently define the impact of crosstalk, or should crosstalk be defined differently?

Channel design considerations

Regardless of the path taken to define channel parameters, Signal Integrity (SI) pre-layout analysis is a must in order to guarantee that the backplane channels will perform to the desired standard(s). For a large backplane, SI analysis requires significant resources to study all possible worst-case scenarios: The higher the number of parameters to be met, the larger the number of dimensions to be explored in the design space.

When designing a 40 Gbps-capable backplane, the layer stack-up and trace geometry choices must strike a balance among multiple factors (Figure 1) influencing the channel parameters of interest. An experienced design team needs to optimize the right balance of these factors to achieve top performance. The design team will utilize simulation studies to find the best performance path within a design’s specification and budget constraints. If costs were not an issue, achieving high performance among the following would be much simpler!

·        Necessary number of routing layers given by the topology: dual star, full-mesh, or custom

·        Desired impedance: 100-, 95-, or 85-Ohm differential impedances

·        Skew: lane-to-lane, if specified, or “within channel,” or intra-pair

·        Skin effect loss and dielectric loss, as part of insertion loss

·        Need to place ground vias at regular intervals

Manufacturability limits affect the factors just mentioned:

·        Total backplane thickness and aspect ratio

·        Tight versus loose finished line impedance tolerance

·        Desired glass weave style

·        Copper surface roughness and desired copper foil thickness

·        Antipad geometry

·        Back drilling requirements including number and depth of levels and the like

21
Figure 1: Channel Physical Factors

 

Characterization challenges

Typically a backplane channel is partitioned into segments (subsections) so that the simulation time is reduced and parameterization becomes possible. Points G and T in Figure 2 represent the boundary between the main three channel segments. For comparison, the IEEE 802.3-2008 defines the backplane channel for 10GBASE-KR and includes both halves of the connector system. The blade portion of the channel ends at a theoretical point that must be outside of the connector via field.

22
Figure 2: Channel Partitioning

 

Although these are virtual points, they can be probed rather easily – they correspond to points G and T. These may coincide with actual measurement points (Figure 3).

23
Figure 3: IEEE 802.3-2008 Test Points Definition

 

Compounding the problem of assigning specific limits such as “blade portion only” or “backplane portion only” to parameters for specific subsections of the channel interconnect is the fact that some of the boundary points such as points K and Q in Figure 2, between subsections are purely virtual and cannot be probed directly or de-embedded in a straightforward manner.

Additionally, the discontinuities of the channel are located in very close proximity to some of these “boundary points,” which makes the task of estimating the impact of each individual subsection on the overall channel performance even more difficult, while concatenation of the individual subsections’ S-parameters data may yield causality and passivity issues.

Correlation between simulated and measured channel data must be part of any validation. While simulated (synthesized) channels can have purely virtual boundaries, data extracted from measurements can only be referenced to real test points, so the accessibility of the test points becomes a central issue. De-embedding techniques can be devised that may take out any kind of “fancy” boundaries, but usually mated connector pairs are not amenable to this approach.

Finally, one must consider the issue of test fixtures. Fixtures can be constructed both for backplanes and for daughter cards. However, their physical boundaries, which are the contact points with the Device Under Test (DUT), should correspond, more or less, with the de-embedding points of the simulation, and this requirement poses a significant challenge to the fixture developer.

In defining a test fixture, one must consider if it is desirable to have minimal fixture impact on the signal transmitted over the channel, which comprises the backplane and a certain path on the daughter card, so the fixture itself is “transparent” to the channel. This would be the case with micro-probes, for instance. On the other hand, in the case of fixtures needed for daughter cards, the fixture could mimic a “standard” compliant backplane. Or it could mimic a “standard” compliant daughter card in the case of fixtures needed for backplanes.

Depending on the strategy chosen, high precision TRL de-embedding structures can be designed as part of the test fixture “kit” in order to ensure accurate characterization of the channel over a wide frequency range. Figure 4 shows a typical backplane test setup.

24
Figure 4: Backplane Test Setup


Channel implementation

There are many steps in the design to assure clean channels while meeting specification parameters. Following the normal design-to-fabrication path in the case of high-speed backplane channels means:

1.     Deciding on the representative worst case/best case channel instances, given that one cannot simulate all possible channels on a given backplane that has hundreds and thousands of differential traces routed.

2.     Determining the structure of the channel(s) decided above, i.e., defining the segments that make up the channels (also called “partition”) and their exact geometry.

3.     Performing pre-layout SI analysis, individually on the segments and then on the overall channel(s);

4.     Optimizing the channel(s) design by analyzing the factors that affect the channel(s) characteristics both individually and in relation to each other. This analysis includes optimizing the backplane and/or daughter card launch portions and giving port attachment points special consideration.

5.     Designing the channel(s) with the layout rules extracted from Step 3.

6.     Fabricating the channel(s) according not only to the optimized design parameter, such as trace width and separation and anti-pad geometry, but also according to the requirements imposed on fab, for example, dielectric material and copper sheet choice. (See Step 3.)

7.     Measuring the manufactured backplane (channels) using high-performance equipment, such as a large bandwidth Vector Network Analyzer/Performance Network Analyzer (VNA/PNA), and test fixtures or by probing at the test points described in Figure 2.

8.     Correlating measurements with post-layout simulation/analysis; a good correlation not only serves as a measuring stick for the sanity of the simulation and measurement methodologies, but also as proof of concept for future channel(s) designs.

9.     Deciding the manufacturability of the optimized backplane channel(s) and implementing strict quality control processes that are tied into the test/validation processes to oversee the consistency of fabricating high-speed channels according to the desired industry specifications across large batches and different PCB vendors.

Figure 5 shows an example of a backplane launch portion. Note that traces can be seen exiting from the pin field.

25
Figure 5: Backplane Launch Portion

 

More often than not the steps described above are iterative and feed into each other at multiple points, so the engineering team engaged in such a process must be cross-trained. Manufacturing engineers need to understand design/layout issues, and a PCB layout engineer must understand test and simulation issues, while an SI engineer must understand both manufacturing and layout issues.

As board stack-ups get thicker, the crosstalk performance may begin to hit limits on the edge of acceptability, adding to the issue of how to properly define crosstalk. Current PHYs can deal with “linear” losses, such as IL, but cannot compensate for losses attributed to crosstalk, which inherently occurs not only in the connector body, but also in the pin/via fields of the backplane and daughter card connectors. In such instances, multi-crosstalk, i.e., multiple aggressors, analysis is imperative, and a dedicated time-domain test setup is necessary to study crosstalk and jitter. Also, simulation of multiple aggressors may prove tricky. Care must be taken in order to avoid simplistic assumptions about the behavior of the aggressor signals.

Signal integrity simulations performed at Elma Bustronic have shown that achieving 10 Gbps/link performance within the IEEE 802.3-2008 section 5 informative limits is possible with reasonable manufacturing and design practices. It is not necessary to resort to expensive materials/processes or to affect the usual design cycles in a serious manner. Elma Bustronic has manufactured and tested prototypes to the informative limits of IEEE and passed. Testing has been realized with a VNA/PNA and custom fixtures developed in partnership with a well-known high-speed test equipment company.

Until a standardized text fixture setup is created by the industry for an apples-to-apples comparison of signal performance, companies will need to do thorough SI work to achieve 40 Gbps performance.

Conclusion

Despite the challenges, complex backplane channel designs may be efficiently partitioned and analyzed to improve their performance over a specified frequency range and within the characteristics of a given PHY (transceiver). As data rates increase, tight collaboration between backplane and daughter card OEMs may prove crucial in order to achieve best performance of such designs and positively impact the development of new standards. As part of its R&D efforts, Elma Bustronic is committed to working with daughter card developers to improve channel performance and meet any given standard or custom requirements.

Ovidiu Mesesan is a Senior Backplane Designer and Dedicated Consultant at Elma Bustronic. Ovidiu has a B.Sc. in Electronics and Telecommunications from the Polytechnic Institute of Bucharest. He began his career as a Backplane Designer with Trenew (now part of Elma Electronic) and then moved to Canada where together with a few colleagues from Pixstream Inc, he founded Kaparel Corp. at the end of 1999. Ovidiu worked for Kaparel Corp. (a Rittal GmbH company since 2000) as a Senior Backplane and System Designer until 2008, when he started consulting with Elma Bustronic on high-speed standard and custom backplane designs. He is currently involved in backplane characterization projects, supporting R&D and sales from a technical standpoint for Bustronic and participates in several industry committees.

Elma Bustronic

ovidiu.mesesan@elmabustronic.com

www.elmabustronic.com