Introduction
This article looks at IPsec acceleration improvements in the FD.io VPP project based on the Data Plane Development Kit (DPDK) Cryptodev framework.
It gives a brief introduction to FD.io, VPP, DPDK, and the DPDK Cryptodev library, and shows how they are combined to provide enhanced IPsec performance and functionality.
It also shows how to install, build, configure, and run VPP with Cryptodev, and shows the type of performance gains that can be achieved.
Background
FD.io, the Fast Data Project, is an umbrella project for open source software aimed at providing high-performance networking solutions.
VPP, the Vector Packet Processing library, is one of the core projects in FD.io. VPP is an extensible framework that provides production-quality userspace switch/router functionality running on commodity CPUs.
The default implementation of VPP contains IPsec functionality, which relies on the OpenSSL library, as shown in Figure 1.
Figure 1. Default IPsec implementation in VPP
While this implementation is feature rich, it is not optimal from a performance point of view. In VPP 17.01, the IPsec implementation was extended to include support for DPDK’s Cryptodev API to provide improved performance and features.
DPDK is a userspace library for fast packet processing. It is used by VPP as one of its optional I/O layers.
The DPDK Cryptodev library provides a crypto device framework for management and provisioning of hardware and software crypto poll mode drivers, and defines generic APIs that support a number of different Crypto operations.
The extended DPDK Cryptodev IPsec Implementation architecture in VPP is shown in Figure 2.
Figure 2. DPDK Cryptodev IPsec Implementation in VPP
Enabling DPDK Cryptodev in VPP improves the performance for the most common IPsec encryption and authentication algorithms, including AES-GCM, which was not enabled in the default VPP implementation.
As such, enabling the DPDK Cryptodev feature not only increases performance but also provides access to additional options and flexibility such as:
- Devices with Intel® QuickAssist Technology (Intel® QAT) for hardware offloading of all supported algorithms, including AES-GCM.
- Intel® Multi-Buffer Crypto for IPsec Library for a heavily optimized software implementation.
- Intel® Intelligent Storage Acceleration Library (Intel® ISA-L) for heavily optimized software implementation of the AES-GCM algorithm.
The Intel QuickAssist Technology-based Crypto Poll Mode Driver provides support for a range of hardware accelerator devices. For more information see the DPDK documentation for Intel QuickAssist Technology.
Building VPP with DPDK Cryptodev
To try out VPP with DPDK and Cryptodev support you can download and build VPP as follows:
$ git clone https://gerrit.fd.io/r/vpp $ cd vpp $ git checkout v17.04 $ vpp_uses_dpdk_cryptodev_sw=yes make build-release -j
Note that the build command line enables DPDK Cryptodev support for software-optimized libraries and the specific VPP release. The build process will automatically download and build VPP, DPDK and the required software crypto libraries.
To start VPP with DPDK Cryptodev use the following command:
$ make run-release STARTUP_CONF=/vpp_test/vpp_conf/startup.conf
The “startup_conf” path should be changed to suit the specific location in the end-user’s environment.
Testing VPP with DPDK Cryptodev
Figure 3 represents a typical test configuration.
Figure 3. Test system setup
Pktgen, as shown in Figure 3, is a software traffic generator based on DPDK that is being used in this configuration to test the VPP IPsec.
For this example we used the following hardware configuration:
- 1 x Intel® Core™ i7-4770K series processor @ 3.5 GHz
- 2 x Intel® Ethernet Controller 10 Gigabit 82599ES NICs, 10G, 2 ports
- 1 x Intel® QuickAssist Adapter 8950
The VPP “startup.conf” configuration file used in the test setup is shown below:
unix {
nodaemon
interactive
exec /vpp_test/vpp_conf/ipsec.cli
}
cpu {
main-core 0
corelist-workers 1-2
}
dpdk {
socket-mem 1024
uio-driver igb_uio
dev 0000:06:00.0
{
workers 0
}
dev 0000:06:00.1
{
workers 1
}
# Option 1: Leave both options below commented out
# to fall back to the default OpenSSL.
# Option 2: Multi-Buffer Crypto Library.
#enable-cryptodev
#vdev cryptodev_aesni_mb_pmd,socket_id=0
# Option3: QAT hardware acceleration.
#enable-cryptodev
#dev 0000:03:01.1
#dev 0000:03:01.2
}
The configuration file allows these configuration options:
- The default VPP + OpenSSL option
- Cryptodev with the optimized Multi-Buffer software library
- Cryptodev with Intel QuickAssist Technology-based hardware acceleration
The example “startup.conf” file in Figure 4 sets up the system configuration. The devices “0000:06:00.0” and “0000:06:00.1” are the network ports and “0000:03:01.1” and “0000:03:01.2” are the Intel QuickAssist Technology-based device VFs (Virtual Functions). These should be changed to match the end-user system where the test is replicated.
The CPP “ipsec.cli” file used for testing is shown in below:
set int ip address TenGigabitEthernet6/0/0 192.168.30.30/24 set int promiscuous on TenGigabitEthernet6/0/0 set int ip address TenGigabitEthernet6/0/1 192.168.30.31/24 set int promiscuous on TenGigabitEthernet6/0/1 ipsec spd add 1 set interface ipsec spd TenGigabitEthernet6/0/1 1 ipsec sa add 10 spi 1000 esp tunnel-src 192.168.1.1 tunnel-dst 192.168.1.2 crypto-key 4339314b55523947594d6d3547666b45 crypto-alg aes-cbc-128 integ-key 4339314b55523947594d6d3547666b45 integ-alg sha1-96 ipsec policy add spd 1 outbound priority 100 action protect sa 10 local-ip-range 192.168.20.0-192.168.20.255 remote-ip-range 192.168.40.0-192.168.40.255 ipsec policy add spd 1 outbound priority 90 protocol 50 action bypass ip route add 192.168.40.40/32 via 192.168.1.2 TenGigabitEthernet6/0/1 set ip arp TenGigabitEthernet6/0/1 192.168.1.2 90:e2:ba:50:8f:19 set int state TenGigabitEthernet6/0/0 up set int state TenGigabitEthernet6/0/1 up
This file sets up the network interfaces and the IPsec configuration.
In the configuration shown above, incoming packets on interface “TenGigabitEthernet6/0/0” with the destination IP “192.168.40.40” will be encrypted using the AES-CBC-128 algorithm and authenticated using the SHA1-96 algorithm. The packets are then sent out on the “TenGigabitEthernet6/0/1” interface. Again, this configuration should to be adjusted to reflect the end-user’s environment.
Using the Pktgen traffic generator configured to send packets that match the IPsec configuration above (source 192.168.20.20, destination 192.168.40.40, port 80, and packets of variable size 64, 128, 256, 512, and 1024), performance data can be generated as shown in Figure 4. Note, the data shown is for illustration purposes only, actual values will depend on the hardware configuration and software versions. The “MB” data refers to the Multi-Buffer Crypto Library which is an optimized software library.
Figure 4. Example throughput vs. packet size for different configurations
The illustrative data in Figure 4 shows a significant improvement using software-optimized libraries and an even larger improvement using hardware offloading, with the throughput being capped by the line rate rather than processing power.
Conclusion
This article gives an overview of the DPDK Cryptodev framework in VPP and shows how it can be used to improve performance and functionality in the VPP IPsec implementation.
About the Author
Radu Nicolau is a network software engineer with Intel. His work is currently focused on IPsec-related development of data plane functions and libraries. His contributions include enablement of the AES-GCM crypto algorithm in the VPP IPsec stack, IKEv2 initiator support for VPP, and inline IPsec enablement in DPDK.