Introduction
NOTE: The OpenVINO™ toolkit was formerly known as the Intel® Computer Vision SDK
The OpenVINO™ toolkit is a comprehensive toolkit for quickly developing applications and solutions that emulate human vision. Based on Convolutional Neural Networks (CNNs), the toolkit extends CV workloads across Intel® hardware, maximizing performance.
The OpenVINO™ toolkit:
- Enables the CNN-based deep learning inference on the edge.
- Supports heterogeneous execution across Intel CV accelerators, using a common API for the CPU, Intel® Integrated Graphics, Intel® Movidius™ Neural Compute Stick, and FPGA.
- Speeds time-to-market through an easy-to-use library of CV functions and pre-optimized kernels.
- Includes optimized calls for CV standards, including OpenCV*, OpenCL™, and OpenVX*
New and Changed in This Release
Model Optimizer Changes
The Model Optimizer component has been replaced by a Python*-based application, with a consistent design across the supported frameworks. Key features are listed below. See the Model Optimizer Developer Guide for more information.
- General changes:
- Several CLI options have been deprecated since the last release. See the Model Optimizer Developer Guide for more information.
- More optimization techniques were added.
- Usability, stability, and diagnostics capabilities were improved.
- Microsoft* Windows* 10 support was added.
- A total of more than 100 public models are now supported for Caffe*, MXNet*, and TensorFlow* frameworks.
- A framework is required for unsupported layers, and a fallback to the original framework is available for unsupported layers.
- Caffe* changes:
- The workflow was simplified, and you are no longer required to install Caffe.
- Caffe is no longer required to generate the Intermediate Representation for models that consist of standard layers and/or user-provided custom layers. User-provided custom layers must be properly registered for the Model Optimizer and the Inference Engine. See Using the Model Optimizer to Convert Caffe* Models for details and a list of standard layers.
- Caffe is now only required for unsupported layers that are not registered as extensions in the Model Optimizer.
- TensorFlow* support is significantly improved, and now offers a preview of the Object Detection API support for SSD*-based topologies.
Inference Engine
- Added Heterogeneity support:
- Device affinities via API are now available for fine-grained, per-layer control.
- You can now specify a CPU fallback for layers that the FPGA does not support. For example, you can specify
HETERO: FPGA, CPUas a device option for Inference Engine samples. - You can use the fallback for CPU + Intel® Integrated Graphics if you have custom layers implemented only on the CPU, and you want to execute the rest of the topology on the Intel® Integrated Graphics without rewriting the custom layer for the Intel® Integrated Graphics.
- Asynchronous execution: The Asynchronous API improves the overall application frame rate, allowing you to perform secondary tasks, like next frame decoding, while the accelerator is busy with current frame inference.
- New customization features include easy-to-create Inference Engine operations. You can:
- Express the new operation as a composition of existing Inference Engine operations or register the operation in the Model Optimizer.
- Connect the operation to the new Inference Engine layer in C++ or OpenCL™. The existing layers are reorganized to “core” (general primitives) and “extensions” (topology-specific, such as
DetectionOutputfor SSD). These extensions now come as source code that you must build and load into your application. After the Inference Engine samples are compiled, this library is built automatically, and every sample explicitly loads the library upon execution. The extensions are also required for the pre-trained models inference.
- Plugin support added for the Intel® Movidius™ Neural Compute Stick hardware (Myriad2).
- Samples are provided for an increased understanding of the Inference Engine, APIs, and features:
- All samples automatically support heterogeneous execution.
- Async API showcase in Object Detection via the SSD sample.
- Minimalistic
Hello, classificationsample to demonstrate Inference Engine API usage.
OpenCV*
- Updated to version 3.4.1 with minor patches. See the change log for details. Notable changes:
- Implementation of on-disk caching of precompiled OpenCL kernels. This feature reduces initialization time for applications that use several kernels.
- Improved C++ 11 compatibility on source and binary levels.
- Added subset of OpenCV samples from the community version to showcase the toolkit capabilities:
bgfg_segm.cpp- background segmentationcolorization.cpp- performs image colorization using DNN module (download the network from a third-party site)dense_optical_flow.cpp- dense optical flow using T-API (Farneback, TVL1)opencl_custom_kernel.cpp- running custom OpenCL™ kernel via T-APIopencv_version.cpp- the simplest OpenCV* application - prints library version and build configurationpeopledetect.cpp- pedestrian detector using built-in HOGDescriptor
OpenVX*
- A new memory management scheme with the Imaging and Analytics Pipeline (IAP) framework drastically reduces memory consumption.
- Introduces intermediate image buffers that result in a significant memory footprint reduction for complex Printing and Imaging (PI) pipelines operating with extremely large images.
- Deprecated tile pool memory consumption reduction feature. Removed from the Copy Pipeline sample.
- The OpenVX* CNN path is not recommended for CNN-based applications and is partially deprecated:
- CNN AlexNet* sample is removed.
- CNN Custom Layer (FCN8) and Custom Layers library are removed.
- The OpenVX* SSD-based Object Detection web article is removed.
- OpenVX* FPGA plugin is deprecated. This is part of the CNN OVX deprecation.
- The VAD tool for creating OpenVX* applications is deprecated and removed.
- New recommendation: Use Deep Learning Inference Engine capabilities for CNN-based applications.
Examples and Tutorials
- Model downloader for the OpenVINO™ toolkit public models in Caffe format:
- densenet-121, densenet-161, densenet-169, densenet-201 - Densely Connected Convolutional Networks. https://github.com/shicai/DenseNet-Caffe
- squeezenet 1.0, squeezene 1.1 - SqueezeNet: AlexNet-level accuracy with 50 times fewer parameters and 0.5 MB model size. https://github.com/DeepScale/SqueezeNet
- mtcnn-p, mtcnn-r, mtcnn-o - Multi-task Cascaded Convolutional Networks: Proposal, Refine, Output https://github.com/DuinoDu/mtcnn/tree/master/model
- mobilenet-ssd – Common object detection architecture https://github.com/chuanqi305/MobileNet-SSD
- vgg16, vgg19 - Very Deep Convolutional Networks For Large-Scale Image Recognition https://arxiv.org/pdf/1409.1556.pdf
- ssd300, ssd512 - Single Shot MultiBox Detector https://arxiv.org/pdf/1512.02325.pdf
- Cross-check tool: To debug the model inference both in whole and layer-by-layer, comparing accuracy and performance between CPU, Intel® Integrated Graphics, and the Intel® Movidius™ Neural Compute Stick.
- CNN pre-trained models (prototxt) + pre-generated Intermediate Representations (
.xml+.bin):age-gender-recognition-retail: Age and gender classification.face-detection-retail: Face Detection.person-detection-retail: Person detection.license-plate-recognition-barrier: Chinese license plate recognition.face-detection-adas: Face Detection.person-detection-retail: Person Detection.head-pose-estimation-adas: Head and yaw + pitch + roll.vehicle-attributes-recognition-barrier: Vehicle attributes (type/color) recognition.person-vehicle-bike-detection-crossroad: Multiclass (person, vehicle, non-vehicle) detector.vehicle-license-plate-detection-barrier: Multiclass (vehicle, license plates) detector.
Known Issues
| ID | Description | Component | Workaround |
|---|---|---|---|
| 1 | Releasing a non-virtual vx_array object after it has been used as a parameter in a graph and before graph execution, may result in slow vxProcessGraph and data corruption. | OpenVX* | N/A |
| 2 | When a graph is abandoned due to a failure in a user node, the callbacks that are attached to skipped nodes are called. | OpenVX | N/A |
| 3 | The OpenVX* volatile kernels extensions API are subject to change. | OpenVX | N/A |
| 4 | Multiple user node access the same array cause application crash. | OpenVX | N/A |
| 5 | Intel® Integrated Graphics equalize histogram node partially runs on CPU. | OpenVX | N/A |
| 6 | User node hangs when calling Intel® Intetegated Performance Primitives if the node is linked to IAP.so | OpenVX | N/A |
| 7 | Edge Tracing part of IPU Canny Edge detection runs on CPU. | OpenVX | N/A |
| 8 | The Harris Corners* Kernel Extension produces inaccurate results when the sensitivity parameter is set outside the range of [0.04; 0.15] | OpenVX | N/A |
| 9 | The API vxQueryNode() returns zero for custom Intel® Integrated Graphics nodes when queried for the attribute VX_NODE_ATTRIBUTE_PERFORMANCE. | OpenVX | N/A |
| 10 | Node creation methods do not allow using the NULL pointer for non-optional parameters. | OpenVX | N/A |
| 11 | The vx_delay object doesn’t the support vx_tensor and vx_object_array types | OpenVX | N/A |
| 12 | The vx_delay object is not supported as a user node input parameter | OpenVX | N/A |
| 13 | Scalar arguments are not changing dynamically in several nodes in ColorConvert node on Intel®Integrated Graphics in Runtime | OpenVX | N/A |
| 14 | The OpenCL™ out of order queue feature might slow down a single nodes graph | OpenVX | N/A |
| 15 | On CPU in vxConvolutionLayerrounding_police parameter ignored, TO_ZERO rounding is used in any case | OpenVX | N/A |
| 16 | On CPU in vxFullyConnecedLayerrounding_police parameter ignored, TO_ZERO rounding is used in any case | OpenVX | N/A |
| 17 | On CPU in vxTensorMultiplyNode rounding_policy parameter ignored, TO_ZERO policy is used in any case | OpenVX | N/A |
| 18 | Unsupported Dynamic Shapes for Caffe* layers | Model Optimizer | N/A |
| 19 | Some TensorFlow operations are not supported, but only a limited set of different operations can be successfully converted. | Model Optimizer | Enable unsupported ops through Model Optimizer extensions and IE custom layers |
| 20 | Only TensorFlow models with FP32 Placeholders. If there is non FP32 Placeholder, the next immediate operation after this Placeholder should be a Cast operation that converts to FP32. | Model Optimizer | Rebuild your model to include a FP32 placeholder only or add cast operations |
| 21 | Only TensorFlow models with FP32 weights are supported. | Model Optimizer | Rebuild your model to have FP32 weights only |
| 22 | The recent version of TensorFlow Detection API is not supported. Only SSD models frozen in versions prior r1.6.0 of the detection API can be converted. | Model Optimizer | N/A |
| 23 | Pre-build protobuf binary distributed as egg-file with Model Optimizer breaks Python 3.5.2 installation. It shouldn't be used with Python 3.5.2. | Model Optimizer | Build protobuf binary yourself (recommended), or use Python version of protobuf (slow |
| 24 | TensorFlow models with trainable layers such as Conv2D or MatMul that re-use weights from the same Const operations cannot be successfully converted. | Model Optimizer | Rebuild a model with duplicated Const operations to avoid weights sharing |
| 25 | Embedded preprocessing in Caffe models is not supported and is ignored. | Model Optimizer | Pass preprocessing parameters through MO CLI parameters |
| 26 | Offloading computation to TensorFlow using the following command line parameters doesn't work with TensorFlow 1.8:
| Model Optimizer | N/A |
| 27 | Releasing the the plugin's pointer before inference completion might cause a crash. | Inference Engine | Release the plugin pointer at the end of the application, when inference is done. |
| 28 | Altera* OpenCL* 17.1 might not be installed properly. Follow the Installation guide. | Inference Engine | Use the instructions in the FPGA installation guide |
| 29 | FP11 bitstreams can be programmed to the boards using the flash approach only. | Inference Engine | Use the instructions in the FPGA installation guide |
| 30 | If Intel OpenMP was initialized before OpenCL, OpenCL will hang. This means initialization or executing the FPGA will hang, too. | Inference Engine | Initialize FPGA or Heterogeneous with the FPGA plugin priority before the CPU plugin. |
| 31 | The performance of the first iteration of the samples for networks executing on FPGA is much lower than the performance of the next iterations. | Inference Engine | Use the -ni <number> -pc to tet the real performance of inference on FPGA. |
| 32 | To select the best bitstream for a custom network, evaluate all available bitstreams and choose the bitstream with the best performance and accuracy. Use validation_app to collect accuracy and performance data for the validation dataset. | Inference Engine | |
| 33 | The Intel® Movidius™ Myriad™ Vision Processing Unit plugin supports batch=1 only | Inference Engine | Infer the batch of images one at a time, or use multiple Intel® Movidius™ Myriad™ Vision Processing Units |
| 34 | Myriad plugin may fail to open device when several processes try to do inference the same time and several NCS devices are available | Inference Engine | Use threads within same process to utilize multiple devices. |
| 35 | The setBatch method works only for topology which has batch as first dimension for all tensors | Inference Engine | |
| 36 | Multiple OpenMP runtime initialization is possible if you are using MKL and Inference Engine simultaneously | Inference Engine | Use apreloaded iomp5 dynamic library |
| 37 | Completion Callback is called in case of succesfull execution of infer request only | Inference Engine | Use Wait to get notfied about errors in infer request |
Included in This Release
The OpenVINO™ toolkit is available in three versions:
- OpenVINO™ toolkit for Windows
- OpenVINO™ toolkit for Linux
- OpenVINO™ toolkit for Linux with FPGA Beta Support
| Install Location/File Name | Description |
|---|---|
| Deep Learning Model Optimizer | Model optimization tool for your trained models. |
| Deep Learning Inference Engine | Unified API to integrate the inference with application logic |
| OpenCV* 3.4.1 library | OpenCV Community version compiled for Intel hardware. Includes PVL libraries for computer vision |
| Intel® Media SDK libraries (open source version) | Eases the integration between the OpenVINO™ toolkit and the Intel® Media SDK. |
| Intel OpenVX* 1.1 runtime | Directory containing bundled OpenVX* runtime that supports CPU, Intel® Integrated Graphics, and IPU devices |
| OpenCL™ NEO driver | Improves usability |
| Intel® FPGA Deep Learning Acceleration Suite, including pre-compiled bitstreams | Implementations of the most common CNN topologies to enable image classification and ease the adoption of FPGAs for AI developers. Includes pre-compiled bitstream samples for the Intel® Programmable Acceleration Card with Intel® Arria® 10 GX FPGA and the Arria® 10 GX FPGA Development Kit. |
| Intel® FPGA SDK for OpenCL™ software technology | The Intel® FPGA RTE for OpenCL™ provides utilities, host runtime libraries, drivers, and RTE-specific libraries and files |
| OpenVINO™ toolkit documentation | Developer guides and other documentation. Available from the OpenVINO™ toolkit product site |
| Pre-trained Deep Learning Models | Ten pre-trained models for prototxt and generated Intermediate Representation Files. You can use these for demonstrations, to help you learn the product, or for product development. |
| Computer Vision Samples | Samples that illustrate use of or application computer vision application creation for the Inference Engine, OpenCV, and OpenVX. |
Where to Download This Release
https://software.intel.com/en-us/OpenVINO-toolkit/choose-download
System Requirements
Development Platform
Processors
6th-8th Generation Intel® Core™ & Xeon™ processor
Operating Systems:
- Ubuntu* 16.04.3 long-term support (LTS), 64-bit
- CentOS* 7.4, 64-bit
- Windows* 10, 64-bit
Target Platform (choose one processor with one corresponding operating system)
Your requirements may vary, depending on which product version you use.
CPU processors with corresponding operating systems
- 6th-8th Generation Intel® Core™ & Xeon™ processor with operating system options:
- Ubuntu* 16.04.3 long-term support (LTS), 64-bit
- CentOS* 7.4, 64-bit
- Windows* 10, 64-bit
- Intel® Pentium® processor N4200/5, N3350/5, N3450/5 with Intel® HD Graphics
- Yocto Project* Poky Jethro* v2.0.3, 64-bit
Intel® Integrated Graphics processors with corresponding operating systems (GEN Graphics)
NOTE: This installation requires drivers that are not included in the Intel® openVINO-toolkitpackage
- 6th Generation Intel® Core™ processor with Intel® Iris® Pro graphics and Intel® HD Graphics
- Ubuntu* 16.04.3 long-term support (LTS), 64-bit
- CentOS* 7.4, 64-bit
- 6th Generation Intel® Xeon™ processor with Intel® Iris® Pro graphics and Intel® HD Graphics (without e5)
- Ubuntu* 16.04.3 long-term support (LTS), 64-bit
- CentOS* 7.4, 64-bit
FPGA processors with corresponding operating systems
NOTES:
Only for the OpenVINO™ toolkit for Linux with FPGA Beta Support download
OpenCV* and OpenVX functions must be run against the CPU or Intel® Integrated Graphics to get all required drivers and tools
- Intel® Arria® FPGA 10 GX development kit
- Ubuntu* 16.04.3 long-term support (LTS), 64-bit
- CentOS* 7.4, 64-bit
Intel® Movidius™ Neural Compute Stick processor with corresponding operating systems
- Intel® Movidius™ Neural Compute Stick Neural Compute Stick
- Ubuntu* 16.04.3 long-term support (LTS), 64-bit
- CentOS* 7.4, 64-bit
Helpful Links
Note: Links open in a new window.
OpenVINO™ toolkit Home Page: https://software.intel.com/en-us/OpenVINO-toolkit
Intel® OpenVINO™ toolkit Documentation: https://software.intel.com/en-us/OpenVINO-toolkit/documentation/featured
Legal Information
You may not use or facilitate the use of this document in connection with any infringement or other legal analysis concerning Intel products described herein. You agree to grant Intel a non-exclusive, royalty-free license to any patent claim thereafter drafted which includes subject matter disclosed herein.
No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document.
All information provided here is subject to change without notice. Contact your Intel representative to obtain the latest Intel product specifications and roadmaps.
The products described may contain design defects or errors known as errata which may cause the product to deviate from published specifications. Current characterized errata are available on request.
Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Learn more at http://www.intel.com/ or from the OEM or retailer.
No computer system can be absolutely secure.
Intel, Arria, Core, Movidius, Xeon, OpenVINO, and the Intel logo are trademarks of Intel Corporation in the U.S. and/or other countries.
OpenCL and the OpenCL logo are trademarks of Apple Inc. used by permission by Khronos
*Other names and brands may be claimed as the property of others.
Copyright © 2018, Intel Corporation. All rights reserved.