Testing FCoE storage protocols

Yet another storage technology? Yes. Fibre Channel over Ethernet (FCoE) has arrived on the scene for a number of reasons, which Steve Looby of SANBlaze Technology, Inc., a designer of SAN Emulation test products and manufacturer of a complete line of AMC, PMC, AdvancedTCA, and CompactPCI board-level storage solutions, outlines here. Steve then goes on to discuss the new test capabilities for developing and deploying FCoE storage equipment.

Fibre Channel over Ethernet (FCoE) is a new technology that will allow Ethernet networks to carry Fibre Channel (FC) storage I/O data frames. FC is both a protocol and a specialized high-speed optical network, developed over a 20-year period. Most often used for storage networking, it forms the backbone of most modern data centers. Data center uptime, crucial to so many corporate operations, causes corporations to deploy a separate FC network in many cases because such networks are fast, reliable (loss-less), and boast very low latency. If FCoE lives up to its promise, data centers and AdvancedTCA blade server backplanes will soon have the option to consolidate future infrastructure investments and operate FC protocol on 10 Gigabit Ethernet (GbE) equipment.

Why FCoE?

Before diving into the FCoE test challenges, it might be helpful to understand why the industry would consider another Ethernet-based storage protocol when we already have iSCSI, FCIP, NFS, and CIFS. By their very nature, all storage protocols require guaranteed packet delivery. When a document is saved, there can be no ambiguity about whether the data arrives at the disk. Yet Ethernet is by definition a best-effort transport. It attempts to deliver all packets, but Ethernet may drop packets whenever the network becomes very busy.

To tolerate Ethernet’s intrinsic lossy nature, many applications employ TCP. TCP is effectively a “tally system” an application can use to guarantee data delivery between sender and receiver. With TCP, every packet is numbered and tracked as it travels through the IP network. TCP protocol detects dropped or lost Ethernet packets and retransmits whenever necessary. TCP software runs on both the transmitter and receiver and can sometimes consume a large percentage of the host computer’s CPU bandwidth.

FCoE abandons IP protocol and IP addresses (for example, 192.168.1.xx) and instead employs a new 802.3 Ethertype. The approach eliminates TCP altogether, replacing it with a hardware flow control scheme that guarantees packet delivery and can match performance metrics found on native Fibre Channel networks. Moreover, FCoE uses the same protocols as native FC, which allows inexpensive gateways to shuttle traffic between new FCoE equipment and legacy FC equipment. Costs shrink dramatically when consolidation results in fewer adapters, transceivers, and cables, less labor to manage a single infrastructure, and less energy and cooling. Consolidation can be especially useful in AdvancedTCA blade server environments, which often feature Ethernet XAUI signaling as the only means to conduct high-speed blade-to-blade I/O.

Two keys to high-performance FCoE

To match the performance metrics established by native Fibre Channel technology, users should look for Ethernet controllers with two key characteristics. First, look for controllers with PAUSE frame management implemented in hardware. As mentioned, FCoE is the only major Ethernet storage protocol that will not utilize TCP. To guarantee packet delivery, FCoE utilizes a flow control scheme defined in IEEE 802.3x called PAUSE commands. PAUSE will exist at the data link layer, and allows sender and receiver to temporarily pause traffic. Optionally, some Ethernet controllers will offer a priority pause scheme, which allows suspension of FCoE frames, but permits non-FCoE traffic to pass through the controller. The second key feature is jumbo frame support, already common on most commercial Ethernet controllers. Fibre Channel allows data payloads that are 2 KB in size. Jumbo frame support makes it possible for the FCoE network to carry the maximum size unaltered native FC frames.

FCoE storage hierarchy: Who’s boss?

The equipment or tools selected to validate FCoE must have the ability to understand block storage hierarchy. The foundation of FCoE storage is still the disk drive, constructed of platters that hold magnetic charges representing data. Disk firmware divides each platter into concentric circular tracks – much like an old LP record (for those old enough to remember). The disk divides each track into sectors, and each sector contains some number of blocks. (Figure 1) A block is the smallest addressable chunk of data on a disk – generally 512 bytes. (It is the block in what we call block I/O.) The disk linearly numbers each block from 0 to its maximum capacity. The index is called a Logical Block Address (LBA).

Figure 1

If you walk into a data center, you will see hundreds of disks installed into storage devices that provide virtualization and RAID protection services and ultimately make the storage visible to a host. Several vendors announced FCoE storage products in 2008, and users can expect product announcements to accelerate in 2009. Let us briefly examine what is happening inside an FCoE storage array:

1. Real physical disks are installed in the storage product.

2. Software in the storage product may apply RAID services to these disks.

3. Software in the storage product is used to create a virtual volume and virtual disks.

4. Software assigns virtual disks to hosts using Access Control Lists (ACLs).

5. Hosts access virtual disks using FC protocol.

6. FC commands are encapsulated in Ethernet packets (FCoE) for transmission.

The final piece of FCoE storage hierarchy is the FCoE protocol. As with the Russian dolls that nest one within the other, the outer “shell” is the Ethernet 802.3 packet. Ethernet switches use the address information to deliver the packet. The Ethernet frame carries a new frame type designated for FCoE traffic. The FCoE frame carries Fibre Channel frames. Within the FC frame (Figure 2) is the FC protocol, which carries an SCSI command. SCSI commands are the block I/O language that hosts use to communicate with storage products. The SCSI command set includes SCSI read, SCSI write, and dozens of commands for disk management that all exist to facilitate block I/O storage activities.

Figure 2

FCoE test challenges

Whether FCoE deploys in the data center or the backplane storage choice is a chassis, there will be servers, switches, and storage products. Each product category presents a slightly different test challenge, which requires validation of functionality, performance, and error path responses. Table 1 shows the minimum criteria for consideration that FCoE test equipment must support to achieve functional FCoE protocol exercise capability.

The best FCoE equipment will also include a four-corner I/O performance measurement and traffic generation (Figure 3).

Figure 3

Table 1

Optimal FCoE test equipment will feature both FCoE and native FC test ports for gateway validation. They will feature scriptable test execution to facilitate deterministic regression testing.

FC technology is already an integral part of most data centers, and suppliers are racing to launch FCoE products at all levels of the ecosystem. The transition may prove especially exciting for AdvancedTCA backplane environments that already feature 10 GbE XAUI connections.

Transitioning with confidence to FCoE

One example of a solution available to developers transitioning to FCoE is the VirtuaLUN emulation system from SANBlaze Technology, Inc. (See Figure 4.) This system delivers all the test features discussed in this article. Product developers as well as Q/A, test, and manufacturing professionals should look to test equipment that can improve time to market and product quality while at the same time reducing product development costs. With the right test equipment, the transition to FCoE-based storage can be made confidently and quickly.


Figure 4

Steve Looby is the Director of Product Management at SANBlaze Technology, Inc. responsible for the requirements, definition, and delivery of AMC and AdvancedTCA storage products. He has spent 20 years in the storage technology industry, with roles in engineering, product management, sales, and marketing. Steve holds a B.S.in Electrical Engineering from the University of Massachusetts.

SANBlaze Technology, Inc.



(978) 897-1888 x 222