Why that cornerstone for the NextGen building reads: ìDeep Packet Inspectionî

Network bandwidth is the multimedia service providers’ most valuable asset. Residential bandwidth usage is expected to almost triple from the reported 2008 statistics over the next four years[1]. Teaming up to drive this demand are technology acquisition and increasing application sophistication. We and our neighbors have bought Tivo and VoIP phones, and we’re using Skype, Twitter, Facebook, and the like.

More sophisticated multimedia applications typically initiate on a single well-known server port. But in order to deliver all the necessary multimedia resources, these endpoints tend to negotiate for multiple network bandwidth resources during execution to carry the necessary multimedia data.

Increased enterprise and residential network bandwidth usage multiplies opportunities and challenges for service providers. Being able to identify how individual users want to use the Internet and crafting a quality of service plan for those users can increase revenue and bandwidth availability, resulting in higher profitability, more efficient network usage, and stronger customer satisfaction ratings. Identifying the applications being used isn’t simple though – it requires looking deep into the packet and being able to understand what network resources are being allocated for that application. Another concern for service providers is the ability to comply with the Communications Assistance for Law Enforcement Act (CALEA) laws to enable lawful surveillance.

This month we’ll hear from two innovative companies about their architecture and approach for enabling Deep Packet Inspection (DPI) for service providers and law enforcement officials.

It only looks like a giant cluster

Greg Kopchinski is the Senior Product Manager for Bivio Networks. He points out that a key element of successful deep packet inspection platforms lies in the platform architecture. (See Figure 1.)

Figure 1: Bivio Platform Architecture


Three subsystems form the company’s platform architecture: a network processor, application processor, and management processor.

The applications being inspected can vary widely and change frequently, so it’s unwise to implement the data plane processing as a fixed-function ASIC. Therefore, a programmable network processor with data plane processing software is used with an Application Programming Interface (API) to the application processor subsystem. The API allows each application processor to program policies for which packets are relevant to the application processor, greatly reducing the amount of irrelevant traffic flowing to the application processor.

The application processor subsystem consists of a scalable number of application processor cards. Each application processor has a dual-core processor, and each core is independent with its own memory and operating system. It’s an asymmetric processing environment that allows each core on the application processor card to function independently and maximize flow analysis throughput. The data plane will also send all packets within a given bidirectional packet flow (IP address pair, port pair, and protocol) to a single application processor, making synchronized coordination between processors unnecessary.

The management processor subsystem allows for communication of statistics and remote management information by standard SNMP-based management stations.

A 40 Gbps fabric interconnect between the data plane and application processor cards enables high-speed transfer of relevant packets to the application processors.

Greg mentions that Bivio’s open platform can be used for any number of DPI-enabled applications. “The platform is architected so it looks like a standard Linux environment to the programmer. The box simply looks like a giant cluster of servers, all managed through a single management subsystem and interconnected through a switched Ethernet backplane.”

Bivio customers might elect to start with the basics by compiling and running standard Linux applications immediately on the platform. Then, as performance and complexity increase, the code can be enhanced to use the data plane subsystem. This approach allows the network processor to weed out the irrelevant packets and give back valuable cycles to the application processors for more complex, flow-based analysis of what’s going on.

Each application processor card has an acceleration module interface. This interface allows regular expressions parsing and similar tasks to be off-loaded from the software processing on the application processor. A single application processor delivers the bidirectional flow of packets that it has programmed as relevant through the data plane API. There is no packet buffering for cross-packet analysis or reassembly down in the data plane; doing those things in the data plane would introduce latency that could be detectable by the endpoints using the application. So these cross-packet and reassembly functions are best left up to the application processor software.

The Bivio platform uses a Cisco IOS-like CLI for configuring the appliance and internal network, so the product can be connected to an external management system network.

An other than AdvancedTCA redundancy plan

While the architecture is similar, the platform isn’t AdvancedTCA. The form factor is designed to be compact so it can be deployed as a “bump-on-the-wire” in virtually any environment. As opposed to AdvancedTCA where redundancy and fail-over is built into several components within the architecture, the Bivio platform implements redundancy by using multiple redundant appliances.

The Bivio platform is an ideal environment for deployments wishing to download, compile, and optionally enhance Linux open-source or custom applications within their network. The platform can be loaded to serve as a multifunction environment for things like intrusion detection, prevention, and mitigation as well as traffic classification for load balancing and Quality of Service (QoS) applications.

Going a step further than DPI to reach the how

Kevin Graves (Figure 2) is the Chief Technology Officer at IP Fabrics, a company with products that employ deep packet inspection techniques focused on network surveillance applications.

Figure 2: Kevin Graves


Kevin agrees that the deep packet inspection architecture must involve a programmable data plane processing component with a high-speed interconnect to multiple application processing elements and the IP Fabrics DeepSweep and DeepProbe products implement a very similar architecture to the Bivio platform. However, DeepSweep and DeepProbe focus on network surveillance applications and implement application software compliant with United States CALEA and other international intercept standards.

IP Fabrics defines network surveillance as one of four overlapping categories, as Figure 3 illustrates.

Figure 3: Network Surveillance Applications


Criminal investigations typically involve law enforcement obtaining a warrant for a suspect’s Internet activities. In such a cases, a U.S.-based network provider is required by law to be able to intercept some or all Internet activity (for example, emails, Voice over IP calls, chat, video conferencing) to and/or from that suspect. This also includes being able to automatically detect dynamic IP address assignment and auto-tracking of the suspect based on that person’s IP address information and what applications the individual under surveillance is running. National security uses include intelligence gathering, and counterterrorism activities. The cyber-crime category includes looking for scams, phishing, and inappropriate chat room activities. Network abuse involves detecting improper or illegal use of private or public networks.

Kevin points out that while deep packet inspection forms the foundation of the capabilities of the products, they also incorporate what he terms “Deep Application Protocol Inspection” or DAPI. According to Kevin, DAPI processing is the key ingredient that enables the IP Fabrics product line to be effective at network surveillance applications.

Deep packet inspection is important because it enables wire-speed processing of individual packets as the packets are examined for specific content. Such content might be, for example, RADIUS or DHCP IP address assignment and IP lease durations. That’s why companies like IP Fabrics and Bivio use architectures that ensure high-speed data plane processing. DAPI goes one step further than DPI by inspecting and understanding how applications are communicating.

For example, voice over IP protocols involve call set-up messaging where both endpoints negotiate which port(s) are to be used for the conversation as well as the audio encoding and any special data treatment that might be enabled. Deep packet inspection has the visibility into the pertinent packet bytes, but it doesn’t understand what they mean. The DAPI engines that run on the IP Fabrics application cores (called “Surveillance Machines”) use DPI to gather this information and communicate with other surveillance machines to auto-configure the IP address assignments, ports being used for phone calls or chat sessions, and which audio encoding governs the call. These DAPI state machines implement protocol processing and heuristics that piece together and decode the information from multiple packets.

Kevin explains that while things like voice over IP are standards based, some of the DAPI engines are not and must rely on reverse-engineering the protocols in order to figure out the messaging and the resulting resources being used by the application once the set-up phase is complete. “First you need to implement the protocol state machine. Sometimes this is a standard protocol like the Session Initiation Protocol (SIP) for voice over IP. Other times, like in the case of Web mail or chat protocols, it could mean reverse-engineering the end-to-end communication in order to implement the analysis necessary to track the subject.”

Table 1 summarizes some key aspects of DAPI and how it extends DPI to track more complex applications.

Table 1: DAPI Extensions to DPI

A turnkey solution for service providers looking to comply with the FCC’s recent broadband CALEA requirements, IP Fabrics  DeepSweep uses a Web-based GUI that allows law enforcement or service provider network administrators to configure a large variety of network surveillance activities for a given subject.

The DeepProbe product is an intelligent probe, generally under the control of a separate surveillance element. It is designed to be used in distributed surveillance environments, which are typically large, complex networks. In cases where address assignment is being done in one network, but communication happens on another network, multiple DeepProbes can be deployed at each relevant location.  The DeepProbes  communicate with each other, relaying things like address assignment once call set-up has completed. This enables a  group of probes  auto-update based on the knowledge gathered by each surveillance module within each of the deployed products.


Bivio (www.bivio.net) and IP Fabrics (www.ipfabrics.com) use an architecture involving fast, programmable data plane components with multiple general-purpose CPUs to handle application-level processing. The two companies’ flexible systems can adapt to an ever-changing multimedia applications world.

Bivio focuses on a high-speed platform and data plane software that enables product deployment for a variety of DPI applications. The IP Fabrics products illustrate how DPI and DAPI are used for products focused on network surveillance applications. While DPI is relatively new, it’s interesting to learn about two innovative companies and their approach to intelligent products that are ready to deal with the next generation of multimedia applications users.

For more information, contact Curt by e-mail at [email protected]



[1] Kozischek, David and Sawyer, William, Meeting Residential Bandwidth Demand, CED 1 January 2008 http://www.cedmagazine.com/Article-Residential-Bandwidth-Demand.aspx(http://www.cedmagazine.com/Article-Residential-Bandwidth-Demand.aspx