Overview
In our previous article Using Open vSwitch* with DPDK for Inter-VM NFV Applications, we discussed how to manually build and configure Open vSwitch with the Data Plane Development Kit (OVS-DPDK, from here on) to take advantage of the accelerated user-space datapath for network traffic.
It is not always convenient to manually build libraries and deploy them. Some distributions like Ubuntu have already released standard system packages for DPDK-enhanced Open vSwitch (openvswitch-switch-dpdk). In this article we discuss how to install, configure, and use this package for enhanced network throughput and performance.
Ubuntu has had OVS-DPDK support since Ubuntu 15.10 (Wily Werewolf), and even earlier via custom repos, but we found the vhost-user feature works best in the new release of the package, which you can find at https://launchpad.net/ubuntu/+source/openvswitch-dpdk.
The latest package release is available with Ubuntu 16.04 (Xenial Xerus), and also via PPAs for testing purposes at https://launchpad.net/~paelzer/+archive/ubuntu/dpdk-merge-2.2
With the new release of this package, OVS-DPDK has been updated to use the latest release of both the DPDK (v2.2) and Open Switch (v2.5) projects. We took it for a test drive and were impressed with how seamless and easy it is to use OVS-DPDK on Ubuntu.
We will repeat the same inter-virtual machine (VM) test case that we used in our previous article: We configure OVS-DPDK with two vhost-user ports and allocate them to two VMs. We then run a simple iperf3 test-case. The following diagram captures the setup.
In this article we will use Ubuntu 16.04 64 bit server version on the host for demonstration purposes.
Installing OVS-DPDK using Ubuntu packages
The following steps install the OVS-DPDK using default Ubuntu packages and update the default ovs-vswitchd to use the ovs-vswitchd-dpdk package.
sudo apt-get install openvswitch-switch-dpdk sudo update-alternatives --set ovs-vswitchd /usr/lib/openvswitch-switch-dpdk/ovs-vswitchd-dpdk
Configuring OVS-DPDK on Ubuntu
The OVS-DPDK package (primarily the “ovs-vswitchd-dpdk”) relies on the following config files:
- /etc/default/openvswitch-switch – Passes in DPDK command-line options to ovs-vswitchd
- /etc/dpdk/dpdk.conf – Configures hugepages
- /etc/dpdk/interfaces – Configures/assigns NICs for DPDK use
For our sample test case, the following config options are used:
The above options configure 64 2-MB hugepages for OVS-DPDK use and pass in DPDK command-line options of 0x3 as coremask (cores to run OVS-DPDK on), and 4 memory channels (default).
For details on why Hugepages are required, and how they can help in improved performance, please see http://dpdk.org/doc/guides/linux_gsg/sys_reqs.html#use-of-hugepages-in-the-linux-environment.
For detailed DPDK command-line options and their descriptions, please refer to following documentation: http://dpdk.org/doc/guides/testpmd_app_ug/run_app.html
The third config file (/etc/dpdk/interfaces) is not needed for this inter-VM use case. A follow-up article will cover this config file along with a different use case.
Reboot, or restart the ovs-vswitchd process, and ensure the ovs-vswitchd is being invoked with the requested DPDK options.
For our sample test case, we create a bridge and add two DPDK vhost-user ports:
sudo ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev sudo ovs-vsctl add-port br0 vhost-user1 -- set Interface vhost-user1 type=dpdkvhostuser sudo ovs-vsctl add-port br0 vhost-user2 -- set Interface vhost-user2 type=dpdkvhostuser
Ensure the bridge and vhost-user ports have been properly set up and configured:
Using DPDK vhost-user ports with VMs
Creating VMs is out of the scope of this document. Once we have two VMs created (for example, f21vm1.qcow2 and f21vm2.qcow2), the following commands show how to use the DPDK vhost-user ports we created earlier.
Ensure the qemu version on the system is v2.2.0 or above as discussed under “DPDK vhost-user Prerequisites“ in https://github.com/openvswitch/ovs/blob/master/INSTALL.DPDK.md
sudo sh -c "echo 1200 > /proc/sys/vm/nr_hugepages" sudo qemu-system-x86_64 -m 512 -smp 4 -cpu host -hda /home/user/f21vm1c1.qcow2 -boot c -enable-kvm -no-reboot -nographic -net none \ -chardev socket,id=char1,path=/run/openvswitch/vhost-user1 \ -netdev type=vhost-user,id=mynet1,chardev=char1,vhostforce \ -device virtio-net-pci,mac=00:00:00:00:00:01,netdev=mynet1 \ -object memory-backend-file,id=mem,size=512M,mem-path=/dev/hugepages,share=on -numa node,memdev=mem -mem-prealloc sudo qemu-system-x86_64 -m 512 -smp 4 -cpu host -hda /home/user/f21vm1c2.qcow2 -boot c -enable-kvm -no-reboot -nographic -net none \ -chardev socket,id=char1,path=/run/openvswitch/vhost-user2 \ -netdev type=vhost-user,id=mynet1,chardev=char1,vhostforce \ -device virtio-net-pci,mac=00:00:00:00:00:02,netdev=mynet1 \ -object memory-backend-file,id=mem,size=512M,mem-path=/dev/hugepages,share=on -numa node,memdev=mem -mem-prealloc
DPDK vhost-user inter-VM test case with iperf3
In the previous step, we configured 2 VMs each with a virtio NIC that is connected to OVS-DPDK bridge. After the VMs are powered-up, check the VM to ensure the NIC is properly initialized:
Configure the NIC IP address on both the VMs to be on same subnet. Install iperf3 from http://software.es.net/iperf and then run a simple network test-case.
On one VM, start iperf3 in server mode iperf3 -s and run the iperf3 client on another VM, iperf3 –c server_ip. The network throughput and performance varies depending on your system HW capabilities and configuration.
Summary
Ubuntu has standard packages available for using OVS-DPDK. In this article we discussed how to install, configure, and use this package for enhanced network throughput and performance. We also covered how to configure a simple OVS-DPDK bridge with DPDK vhost-user ports for an inter-VM application use case.
To read more about the DPDK with Open vSwitch, we suggest the following resource(s):
https://github.com/openvswitch/ovs/blob/master/INSTALL.DPDK.md
http://dpdk.org/doc/guides/index.html
Have a question? The SDN/NFV Forum is the perfect place to ask.