40 GbE AdvancedTCA: Is this race necessary?

It’s not necessary for network equipment providers to repeat AT&T’s learning curve when it comes to 4G and 40 GbE, Suzzana explains.

Being “first to…” is a driving force in the telecom industry, affecting telecom vendors at all levels of the food chain – from the carriers down to component manufacturers. As consumers grasp what telecom terms such as “3G” and “WiFi” mean for them, carriers are blurring the definition of 4G even as they claim to have reached (whatever “4G” is) first. “Faux G” threatens to become a running joke.

Network equipment providers typically follow the lead of their carrier customers. Hence, we in the AdvancedTCA market have been scrambling to be the first with 40 Gigabit per second Ethernet (GbE) to support strained fixed and mobile broadband networks, which carriers now find themselves dealing with as a result of being “first to” with the latest and greatest broadband device, technology, or application.

When the iPhone came onto the scene, AT&T reaped the rewards of being first to market. AT&T customers were envied by friends for being first to own the device. Ultimately, AT&T sacrificed network performance, and early iPhone customers wrestled with bugs in the device. And all of this took place in the midst of the marketing momentum that being “first to” created. Network equipment providers have an opportunity to learn from AT&T and take a pragmatic view of 40 GbE investment. Addressing 40 GbE network challenges should happen at a level that includes chassis, backplanes, switches and packet processing—not piece by piece.

For AdvancedTCA, 40 GbE-related launch announcements started in mid 2009. During this time PICMG was in the early stages of defining standards for 40 GbE, and 10 GbE systems were just starting to deploy into carrier networks. Two years later, PICMG is in the final stages of ratifying the 3.1 Rev 2 AdvancedTCA standard (for 40 GbE) and 10 GbE systems are hitting volume deployments. So why the rush to 40 GbE? Is being “first to” really worth this hurry up and wait mode in which the industry has now found itself?

More (and more) bandwidth-intensive apps

It’s no mystery that fixed and mobile broadband networks are straining from the rapid explosion of bandwidth-intensive applications (see Figure 1). To help relieve network strain, 40 GbE systems are indeed needed. Widespread deployment of 40 GbE-enabled AdvancedTCA platforms makes sense, especially for WiMAX and LTE carrier rollouts. In what is good news for AdvancedTCA vendors, Tier 1, 2, and 3 network equipment providers want standards-based alternatives to the expensive, proprietary solutions that currently inhabit carrier and data center networks. Therefore, it’s no wonder the race to 40 GbE is happening.

Figure 1: A gap yawns between traffic and revenue.


However, AdvancedTCA tends to sit behind routers/switches, and that market has only recently deployed 10 GbE connections. At this point, network 40 GbE connections are few and far between, while and 100 GbE (CXP-type connection) is a pipe dream and far more expensive than 10 x 10 GbE.

It’s easy to fall into the trap of thinking that 40 GbE AdvancedTCA comprises merely the chassis, backplane, and switch. What must also be taken into consideration are the processing blades (packet processors for the most part) and external ports to hit the densities that can use 40 GbE backplanes.

Deployment challenges

Another factor to take into account is that while 40 GbE backplanes have been around for a while, they reside inside chassis incapable of cooling 40 GbE blades. Additionally, initial 40 GbE hub switches, which showed up last year, feature less capable 40 GbE switching silicon with compromises including port count (meaning not all slots can be 40 GbE, or uplink capacity diminishes), limited packet manipulation capabilities (for example, load balancing, de-tunneling, and the like), or other limitations such as internal cross-connect bandwidth. This may present challenges when it comes to deployment – either in the form of extended waits for final silicon or limited end system capabilities. During the first half of this year silicon vendors addressed fixes to these shortcomings in the silicon. The updated versions of devices will have better defined I/O and interconnect configurations.

One example of a product that resulted from taking on the challenges mentioned above is Continuous Computing’s 40 GbE hub switch, which features fully capable switch silicon. The FM8x 40 GbE AdvancedTCA switch provides full 40 GbE line-rate switching for 14 slots with a 640 GbE non-blocking fabric switch. The backplane interface not only supports 40 GbE connectivity, but also supports backward compatibility with 1 GbE and 10 GbE. External connectivity comes via a modular-based approach, allowing developers to adapt to individual needs and deployments – whether 10-, 40-, or 100 GbE (future) ports.

As mentioned earlier, 40 GbE encompasses more than just chassis, backplane, and switch. For example, NetLogic Microsystems’ XLP multicore, multithreaded processor is pushing performance per AdvancedTCA slot from 10 GbE to 40 GbE. In Continuous Computing’s FlexPacket ATCA-PP8x, for example (Figure 2), the PP80 dual NetLogic XLP packet processing blade interconnects two XLP832 packet processors, I/O, and backplane fabrics using a non-blocking 40 GbE switch. Four 10 GbE ports to the switch reside on each CPU. External I/O is supported over a dual redundant 40 GbE backplane fabric (with fallback to 10 GbE) as well as external connections to 10 GbE and 40 GbE networks. Configuring the fabric to support multiple 40 GbE interfaces enables the blade to function as a combined hub/packet processor blade.

Figure 2: The Continuous Computing FlexPacket ATCA-PP8x supports multiple 40 GbE interfaces.

In general it’s only the packet processors, such as the NetLogic XLP, that start to significantly exceed 10 GbE today. These devices are only just starting to sample, with boards coming to market in Q1 2011. When 10 GbE AdvancedTCA hit the market the processing technology to make use of it was ready, mature, and shipping, but that’s not the case today. There are exceptions to this rule, but in general 10 GbE is suitable for many of these applications today, and most customers and applications can live with it until 40 GbE matures.

Which apps need 40 GbE?

Applications to which 40 GbE per slot appeals are those that leverage packet processing technology, including:

·        Fixed and Mobile Broadband Networks

·        3G and LTE Gateways including Internet Offload

·        DPI Policy Enforcement and Traffic Shaping

·        Security including IDS/IPS/DDOS

·        GPON and CMTS platforms

·        High Capacity Lawful Intercept

The options available today in early 40 GbE do have some viable homes. They work well for certain start-up companies looking to differentiate their products or niche applications that cannot live with 10 GbE. But this doesn’t apply to most Tier 1 and Tier 2 network equipment providers. These providers’ main interest is to understand how much performance the new processors such as NetLogic XLP832, Cavium OCTEON II, and future Intel devices will deliver. As such, early 40 GbE backplane and switch technologies are suitable for evaluating new processors over the next four to six months. Only after this evaluation period can network equipment providers make realistic estimates of system architectures and data flows. Any kind of deployment is not expected until 2012.

Some of us may have convinced ourselves that network equipment providers have little time to make 40 GbE selections. This is not the case. Rather, network equipment providers have more time to select 40 GbE products that they did to select 10 GbE products. This is true despite savvy marketers of 40 GbE insisting that the platform and switch come first and that they need to be deployed today. In reality, the first step for network equipment providers is to find the processor for 40 GbE, benchmark, and then select a switch and platform. Tier 1 and Tier 2 companies know this approach forms a more effective and efficient path to 40 GbE rather than having to compromise on capabilities, especially around the switch, which is the heart of the system. Isn’t the system what network equipment designers envision and design first? Therefore it makes sense to wait for all the components to be in place and additionally for 40 GbE technology to reach some maturity, which will likely not be until later this year.


 Going back to the AT&T/iPhone example, regardless of network issues or bugs in early devices, AT&T definitely reaped the rewards of subscriber additions due to being first with the iPhone. However, early estimates show that Verizon Wireless has received 500,000 pre-orders for the iPhone (they expect to sell 1.1 million in Q1 2011). Who ultimately wins the iPhone award? Is it AT&T for being first or Verizon Wireless, who waited (due to the AT&T and Apple exclusivity period)? Verizon Wireless has had time to ensure its network is ready, research and prepare appropriate data plans, and take advantage of iPhones with fewer bugs (like the notorious antenna issue).

Rushing to be first with 40 GbE AdvancedTCA brings on a whole new set of complexities compared to 10 GbE and 1 GbE. It is our responsibility as AdvancedTCA vendors to guide network equipment providers along a pragmatic path. Our customers must understand that 10 GbE is a viable option today, and there is value in waiting for a total 40 GbE solution to be available.

Suzzana George is a Product Line Manager at Continuous Computing and is responsible for the company’s AdvancedTCA chassis products. Prior to Continuous Computing, she worked as an industry analyst in the telecom and semiconductor equipment manufacturing spaces for 10 years. Suzzana has a bachelor’s degree in International Business from San Diego State University.

Continuous Computing