This article presents a high-level overview of Open vSwitch* with the Data Plane Development Kit (OvS-DPDK)—the high performance, open source virtual switch—and links to further technical articles that dive deeper into individual OvS-DPDK features. This article was written for users of OvS who want to know more about DPDK integration.
Note: Users can download a zip file of the OVS master branch or the 2.6 branch, as well as installation steps for the master branch or the 2.6 branch.
OvS-DPDK High-level Architecture
Open vSwitch is a production quality, multilayer virtual switch licensed under the open source Apache* 2.0 license. It supports SDN control semantics via the OpenFlow* protocol and its OVSDB management interface. It is available from openvswitch.org, GitHub*, and is also consumable through Linux* distributions.
Native Open vSwitch generally forwards packets via the kernel space data path (see Figure 1). In the kernel data path, the switching “fastpath” consists of a simple flow table indicating forwarding/action rules for packets that are received. Exception packets (first packet in a flow) do not match any existing entries in the kernel fastpath table and are sent to the user space daemon for processing (slowpath). After user space handles the first packet in the flow, the daemon will then update the flow table in kernel space so that subsequent packets in the flow can be processed in the fastpath and not sent to user space. Following this approach, native OvS can eliminate the costly context switch between kernel and user space for a large percentage of received packets. However, the achievable packet throughput is limited by the forwarding bandwidth of the Linux network stack, which is not suited for use cases requiring a high rate of packet processing; for example, Telco.
DPDK is a set of user space libraries that enable a user to create optimized performant packet processing applications (information available at DPDK.org). In practice, it offers a series of Poll Mode Drivers (PMDs), which enable direct transferral of packets between user space and the physical interface, bypassing the kernel network stack. This offers a significant performance boost over kernel forwarding, through the elimination of both interrupt handling and traversal of the kernel network stack. By integrating OvS with DPDK, the switching fastpath is in user space, and the exception path is the same path that is traversed by packets in the kernel fastpath case. The integration of DPDK with OvS is illustrated at a high level in Figure 1.
Figure 1: Integration of Data Plane Development Kit data plane with native Open vSwitch*.
Figure 2 below shows the high-level architecture of OvS-DPDK. OvS switching ports are represented by network devices (or netdevs). Netdev-dpdk is a DPDK-accelerated network device that uses DPDK to accelerate switch I/O, through three separate interfaces: one physical interface (handled by the librte_eth library within DPDK), and two virtual interfaces (librte_vhost and librte_ring). These interface with the physical and virtual devices connected to the virtual switch.
Other OvS architectural layers provide further functionality and interface with, for example, the SDN controller. Dpif-netdev provides user space forwarding and ofproto is the OvS library that implements an OpenFlow switch. It talks to OpenFlow controllers over the network and to switch hardware or software through an ofproto provider. The ovsdb server maintains the up-to-date switching table information for this OvS instance and communicates this to the SDN controller. The following section provides details of the switching/forwarding tables, with further information on the OvS architecture available through the openvswitch.org website.
Figure 2: Open vSwitch* with Data Plane Development Kit high-level architecture.
OvS-DPDK Switching Table Hierarchy
A packet entering OvS-DPDK from a physical or virtual interface receives a unique identifier or hash, based on its header fields, which is then matched against an entry in one of three main switching tables: the exact match cache (EMC), the data path classifier (dpcls), or the ofproto classifier. A packet’s identifier will traverse each of these three tables in order, unless a match is found, in which case the appropriate actions indicated by the match rule in the table will be executed and the packet forwarded out of the switch upon completion of all actions. This scheme is illustrated in Figure 3.
Figure 3: Open vSwitch* with Data Plane Development Kit switching table hierarchy.
The three tables have different characteristics and associated throughput performance/latency. The EMC offers fastest processing for a limited number of table entries. The packet’s identifier must exactly match the entry in this table for all fields—the 5-tuple of source IP and port, destination IP and port, and protocol—for highest speed processing or it will “miss” on the EMC and pass through to the dpcls. The dpcls contains many more table entries (arranged in multiple subtables) and enables wildcard matching of the packet identifier (for example, destination IP and port are specified but any source is allowed). This gives approximately half the throughput performance of the EMC and caters to a much larger number of table entries. Packet flows matched in the dpcls are installed in the EMC so that subsequent packets with the same identifier can be processed at the highest speed.
A miss on the dpcls results in the packet identifier being sent to the ofproto classifier so that the OpenFlow controller can decide on the action. This path is the least performant, >10x slower than the EMC. Matches in the ofproto classifier result in new table entries being established in the faster switching tables so that subsequent packets in the same flow can be processed more quickly.
OvS-DPDK Features and Performance
At the time of this writing, the following high-level OvS-DPDK features are available on the OvS master code branch:
- DPDK support for v16.07 (supported version increments with each new DPDK release)
- vHost user support
- vHost reconnect
- vHost multiqueue
- Native tunneling support: VxLAN, GRE, Geneve
- VLAN support
- MPLS support
- Ingress/egress QoS policing
- Jumbo frame support
- Connection tracking
- Statistics: DPDK vHost and extended DPDK stats
- Debug: DPDK pdump support
- Link bonding
- Link status
- VFIO support
- ODL/OpenStack detection of DPDK ports
- vHost user NUMA awareness
A recent performance comparison between native OvS and OvS-DPDK is highlighted in Figure 4. This shows the throughput in packets-per-second for the Phy-OvS-Phy use case, indicating a ~10x performance enhancement for OvS-DPDK over native OvS, increasing to ~12x with Intel® Hyper-Threading Technology (Intel® HT Technology) enabled (labelled 1C2T, or one physical core with two logical threads, in the figure legend). Similarly, the Phy-OvS-VM-OvS-Phy use case demonstrates a ~9x performance enhancement for OvS-DPDK over native OvS.
Figure 4: Performance comparison - native Open vSwitch* (OvS) and OvS with Data Plane Development Kit.
The hardware and software configuration for this data, along with further use case results, can be found in the Intel® Open Network Platform (Intel® ONP) performance report.
OvS-DPDK Availability
OvS-DPDK is available in the upstream openvswitch.org repository and is also available through Linux distributions as below. The latest milestone release is OvS 2.6 (September 2016), and releases are made with a six-month cadence.
Code is available for download as follows: OvS master branch; OvS 2.6 release branch. Installation steps for the master branch are available as well as installation steps for the 2.6 release branch.
Packaged versions of OvS with DPDK are available from:
Additional Information
To learn more about OvS-DPDK, check out the following videos and articles on Intel® Developer Zone, 01.org, Intel® Network Builders and Intel® Network Builders University.
User guides:
Developer guides:
Articles:
- Rate limiting configuration and usage for OvS with DPDK
- QoS configuration and usage for OvS with DPDK
- Configure vHost User multiqueue for OvS with DPDK
- vHost User NUMA awareness in OvS with DPDK
- DPDK Pdump in OvS with DPDK
- Enabling OvS with DPDK in OpenStack
OvS with DPDK milestone release webinars:
INB university:
White paper:
Have a question? Feel free to follow up with the query on the Open vSwitch discussion mailing thread.
About the Author
Robin Giller is a program manager with the Intel Network Platforms Group.