The OpenSSL* library v1.0.2 introduces the multibuffer enhancements for AES. Multibuffer significantly increases the performance of CBC mode encryption by processing multiple TLS records in parallel on a single hardware thread. As shown in the white paper Improving OpenSSL Performance, the multibuffer enhancements to AES-CBC encryption can result in a 50 to 100% increase in throughput. This makes it an attractive feature for web site administrators using these ciphers, and this article explains how to enable it under two popular, open source, Linux*-based web servers: Apache* HTTP Server from the Apache Software Foundation, and nginx* from Nginx, Inc.
Multibuffer Basics
Multibuffer works by encrypting the TLS records created during a file transfer in parallel. Because of the overhead involved, however, multibuffer only works when the SSL buffers are at least 16 KB in size (where 1 KB is defined as 1024 bytes). Since the current multibuffer implementations process four data streams in parallel this means that the minimum file size required for multibuffer to engage is 64 KB.
Each TLS record is encrypted separately with CBC encryption which means each record also requires a random IV. Since those TLS records are also processed simultaneously, the multibuffer solution demands random numbers at a higher frequency: all else being equal, multibuffer will consume the same number of random values as the non-multibuffer code path, but do so in less time. In OpenSSL 1.0.2, random numbers are supplied by the RAND_bytes() function which uses OpenSSL's software-based pseudorandom number generator (PRNG), mixed with values obtained from the RDRAND instruction if Intel® Data Protection with Secure Key is supported by the processor. Multibuffer throughput might benefit from using Secure Key as the sole random number provider, freeing up CPU cycles for the AES encryption. This can be accomplished by enabling the rdrand engine in OpenSSL, though this will do so for other areas of OpenSSL that depend on random numbers such as the SSL/TLS handshake, too.
Configuring Apache for Multibuffer
Apache uses dynamically sized SSL buffers that adapt to the file being transferred. As a result, Apache will automatically make use of multibuffer if the target file is 64 KB or larger. No configuration needs to be done.
To configure OpenSSL to use the rdrand engine for Apache, use the OpenSSLConf directive:
SSLCryptoDevice rdrand
More information can be found in the article "Configuring the Apache Web server to use RDRAND in SSL sessions".
Configuring nginx for Multibuffer
nginx uses a fixed SSL buffer size which is configured in the nginx.conf file. In order to enable multibuffer the configuration parameter ssl_buffer_size must be set to 64k or larger:
ssl_buffer_size 64k;
Larger sizes will result in greater throughput, but unfortunately it can have an impact on latency no matter which cipher is used. For small files (ones that would not benefit from multibuffer), this means incurring a small penalty in performance.
Unfortuantely, there is no single magic number that works well for all file sizes. This value should be chosen based on the workloads that are common and appropriate for the server, and administrators are encouraged to test different values to find the one that works best for their environment.
At the current time, nginx does not provide a way to configure specific engines within OpenSSL so there is no way to force the rdrand engine for random number generation.
§