Intel® Intelligent Storage Acceleration Library (Intel® ISA-L) Performance Under a Virtual Machine

Intel® ISA-L provides the tools to help accelerate and optimize storage on Intel® architecture (IA) for everything from small office NAS appliances to enterprise storage systems.  Intel® ISA-L can run on various Intel® server processors and provides operation acceleration through the following instruction sets:

•  Intel® Advanced Encryption Standard-New Instruction,
• Intel® Streaming SIMD Extensions,
•  Intel® Advanced Vector Extensions, and
• Intel® Advanced Vector Extensions 2.

In today’s private or public cloud infrastructure, software defined storage (SDS) uses some form of storage virtualization.  To answer questions from various storage developers who wanted to know how Intel® ISA-L performs under a virtual machine (VM), Intel ran ISA-L on a barebones system and under a VM. This experiment demonstrated that the performance running under a VM is as good as that when running on a barebones system.  This blog captures all the performance data and setup instructions for developers interested in reproducing this experiment in their own environment.  

The performance data in Figure 1 shows almost identical results (delta < 1%) when running Intel® ISA-L on a VM (via KVM) and on a barebones system. The data has been converted to - raw data (MB/s) running with VM / raw data (MB/s) running on the barebones system:

Figure 1- Running Intel® ISA-L on a barebones system vs. under a virtual machine

Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark, are measured using specific computer systems, components, software, operations and functions.  Any change to any of those factors may cause the results to vary.  You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more information go to http://www.intel.com/performance

How to Set Up a System Using a VM For Performance Tests with KVM

• Use Ubuntu* 14.04LTS for the guest OS of the KVM.   Install packages using apt-get.
• Important: Ensure BIOS settings and kernel modules are correct, or the system will fall back to QEMU emulation for a VM, which will be very slow and won’t enable AVX/AVX2 and other CPU extensions.

1. Install cpu-checker
    sudo apt-get install cpu-checker

2. Run sudo kvm-ok. You will either see that KVM acceleration can be used      

Figure 2: Successful KVM test

Or get this warning saying KVM acceleration can NOT be used. If you get this message, follow step 2b.

Figure 3: Failed KVM test

2b: If you get the “can NOT warning”, enable vmx / VT in the BIOS (example)

Figure 4: Enable Virtualization in the BIOS

3.  Install the KVM packages
    sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils virt-manager
    sudo virsh –c qemu:///system list 
            - If the command works, it will just show an empty list
            - Note: if you get errors trying to view the console and the machine is inside a firewall, make sure to run " export no_proxy=127.0.0.1"

4. Create a Virtual Machine (VM)

Figure 5 - Create a Virtual Machine

      a) Enter a name for the VM
      b) Ensure that the Connection:  line says localhost (QEMU/KVM)
      c) Choose the location of your VM's ISO image
      d) Once booted, the VM will install itself

5. Ensure the KVM Hypervisor is used

Figure 6: Confirm KVM as the hypervisor

6. Ensure the CPU configuration is correct. If Model: Haswell, set 'avx' to 'require'.

Figure 7:- Confirm the CPU configuration

• Note: If you change settings, you may need to restart the VM

7. Once booted, run"cat/proc/cpuinfo | grep aux" to check for cpu flags. If "avx', 'avx2' and etc. are not listed, you will have to fix the VM configuration.

Figure 8:- Confirm the CPU configuration

How To Set Up Intel® ISA-L On the System (with or without a VM)

1. To access the full suite of Intel® Storage Acceleration Library functions, please fill out and submit this request form.
   You will receive an email that provides information on how to get the complete ISA-L zip file.
2. Download and unzip the library source into the OS. The folder should contain the following:

Figure 9: Unzipped ISA-L directory listing

3. Read the ISA-L_Getting_Started.pdf and Release_notes.txt supplied with the source. From the Guide, choose and follow the instructions to build the source depending on your needs.
4. Note: Don't run ISA-L as root.

How to Run the Benchmarks

1. Run "make perfs". This will build all unit function tests set for 'cache cold - larger data set exceeds LLC size'
   Or
   Run "make perfs D="CACHED_TEST"" ('cache - smaller data set fits within cache')
2. Run "make perf_report". This will run each unit test supported by the platform architecture. Performance results will be output on the console.

Optional: Run “make other”. This will build additional functions including compression functions and unit tests. Compression tests (igzip_file_perf and igzip_stateless_file_perf) are run using each file of a standard corpus -The Calgary Corpus - as an input. It is available here. 

The following unit test results are reported in the above snapshot/overview data table:

(Unit tests run and data reported will depend on platform architecture and instructions supported)

• aes_xts_128_dec_cold:
• aes_xts_256_dec_cold:
• crc16_t10dif_cold:
• crc32_ieee_cold:
• crc32_iscsi_cold:
• erasure_code_decode_cold:
• md5_mb_avx2_cold:
• pq_gen_avx2_cold:
• sha1_avx2_mb_cold:
• sha256_avx2_cold:
• sha512_avx2_cold:
• xor_gen_avx_cold:
• igzip_file_perf with Calgary Corpus* as an input
• igzip_stateless_file_perf with Calgary Corpus* as an input

Each unit test will report results in MB/s. For normalization across platforms, cycle/B is reported based on the throughput and system frequency.

Compression test results are reported as a weighted average of the results from each file being compressed. The throughput is then also calculated to cycle/B.

Below is the platform configuration:

Figure 10 - Tested system configuration

• Intel® Storage Acceleration Library (Open Source Version)
• OPTIMIZING STORAGE SOLUTIONS USING THE INTEL® INTELLIGENT STORAGE ACCELERATION LIBRARY

INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL PRODUCTS. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. EXCEPT AS PROVIDED IN INTEL'S TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY WHATSOEVER AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO SALE AND/OR USE OF INTEL PRODUCTS INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT.

A "Mission Critical Application" is any application in which failure of the Intel Product could result, directly or indirectly, in personal injury or death. SHOULD YOU PURCHASE OR USE INTEL'S PRODUCTS FOR ANY SUCH MISSION CRITICAL APPLICATION, YOU SHALL INDEMNIFY AND HOLD INTEL AND ITS SUBSIDIARIES, SUBCONTRACTORS AND AFFILIATES, AND THE DIRECTORS, OFFICERS, AND EMPLOYEES OF EACH, HARMLESS AGAINST ALL CLAIMS COSTS, DAMAGES, AND EXPENSES AND REASONABLE ATTORNEYS' FEES ARISING OUT OF, DIRECTLY OR INDIRECTLY, ANY CLAIM OF PRODUCT LIABILITY, PERSONAL INJURY, OR DEATH ARISING IN ANY WAY OUT OF SUCH MISSION CRITICAL APPLICATION, WHETHER OR NOT INTEL OR ITS SUBCONTRACTOR WAS NEGLIGENT IN THE DESIGN, MANUFACTURE, OR WARNING OF THE INTEL PRODUCT OR ANY OF ITS PARTS.

Intel may make changes to specifications and product descriptions at any time, without notice. Designers must not rely on the absence or characteristics of any features or instructions marked "reserved" or "undefined". Intel reserves these for future definition and shall have no responsibility whatsoever for conflicts or incompatibilities arising from future changes to them. The information here is subject to change without notice. Do not finalize a design with this information.

The products described in this document may contain design defects or errors known as errata which may cause the product to deviate from published specifications. Current characterized errata are available on request.

Contact your local Intel sales office or your distributor to obtain the latest specifications and before placing your product order.
Copies of documents which have an order number and are referenced in this document, or other Intel literature, may be obtained by calling 1-800-548-4725, or go to:  http://www.intel.com/design/literature.htm

Intel, the Intel logo, AVX2, AVX, SIMD, AES-NI, Intel® ISA-L, and Intel® Xeon® are trademarks of Intel Corporation in the U.S. and other countries.

*Other names and brands may be claimed as the property of others

Copyright© 2014 Intel Corporation. All rights reserved.

This sample source code is released under the Intel Sample Source Code License Agreement

1. Enter a name for the VM

2. Ensure that the Connection: line says localhost (QEMU/KVM)

3. Choose the location of your VM’s ISO image.

4. Once booted, the VM will install itself.

5. Ensure the KVM hypervisor is used

6. Ensure the CPU configuration is correct. If Model: Haswell, set ‘avx’ to ‘require’.

• Note: If you change settings, you may need to restart the VM.

1.  Once booted, run “cat /proc/cpuinfo | grep aux” to check for cpu flags. If ‘avx’, ‘avx2’ etc. are not listed, you will have to fix the VM configuration.

2. To access the full suite of Intel® Storage Acceleration Library functions, please fill out and submit this request form.
   You will receive an email that provides information on how to get the ISA-L zip file.

3. Download and unzip the library source into the OS. The folder should contain the following:

4. Read the ISA-L_Getting_Started.pdf and Release_notes.txt. From the Guide, choose and follow the instructions to build the source depending on your needs.

5. Don’t run ISA-L as root.

6. Run “make perfs”. This will build all unit function tests set for ‘cache cold – larger data set exceeds LLC size’

   Or

   Run “make perfs D=”CACHED_TEST”” (‘cache – smaller data set fits within cache’)

7. Run “make perf_report”. This will run each unit test supported by the platform architecture. Performance results will be output on the console.

• Intel® Storage Acceleration Library (Open Source Version)

• OPTIMIZING STORAGE SOLUTIONS USING THE INTEL® INTELLIGENT STORAGE ACCELERATION LIBRARY

• Intel® AES-NI – Intel® Advanced Encryption Standard - New Instruction

• Intel® SSE – Intel® Streaming SIMD Extensions

• Intel® AVX – Intel® Advanced Vector Extensions

• Intel® AVX2 - Intel® Advanced Vector Extensions 2