The Intel® Computer Vision SDK (Intel® CV SDK) is a comprehensive toolkit that you can use to develop and deploy vision-oriented solutions on Intel platforms. You can use the Intel® CV SDK in things like autonomous vehicles, digital surveillance cameras, robotics, and mixed-reality headsets.
The figure below shows the end-to-end computer vision application process. Some of the components in the figure are not part of the Intel® CV SDK, but are included in the diagram to illustrate the full end-to-end computer vision process.
After your model is trained, you are ready to use the Intel® CV SDK in a developer environment to write an application and optimize its performance on Intel® hardware. In this part of the development, you use the Model Optimizer.
In addition to optimizing your model, in this development phase, you integrate the components within your application. Depending on your specific needs, you might use other tools in your application, such as the Intel® Media SDK.
If you need to retrain your model after you optimize it with the Intel® CV SDK, you will need to return to the tasks listed in the first two columns of the figure to retrain your model, and then run the Model Optimizer again .
Once you have used the Model Optimizer and integrated the applicable components, you are ready to deploy your application at the edge. This piece of the process uses the Inference component of the Intel® CV SDK. In this tutorial, deployment is represented by using the video provided with this tutorial instead of using data directly from a video camera or on a separate piece of hardware.
Product Contents
Deep Learning Model Optimizer tool to help you with deploying Convolutional Neural Networks. It's a cross-platform command line tool that performs static model analysis and adjusts deep learning models for optimal execution on end-point target devices.
- Deep Learning Inference Engine
- Deep Learning Inference Engine Samples
- Deep Learning Deployment Toolkit provides the Inference Engine and Model Optimizer tools as a separate package
- Intel-optimized implementation of the Khronos* OpenVX* 1.1 API, extended with both additional API and kernels, including support for Convolutional Neural Networks (CNN)
- Pre-built and fully-validated community OpenCV* 3.3 binaries with additional fast and accurate Face capabilities
Intel® CV SDK Components
Model Optimizer
The Model Optimizer is the first of two key components of the Intel® CV SDK. The second key component is the Inference Engine.
The Model Optimizer is a cross-platform command-line tool that converts trained models into Immediate Representations (IRs). In the process of optimization, the Model Optimizer:
- Performs horizontal fusion of the network layers
- Merges the network layers
- Prunes unused branches in the network
- Applies weight compression methods
Note: Layers require the use of a framework, like Caffe*
Three-stage model processing
The Model Optimizer uses three-stage model processing to transform a deep learning network:
Learning stage:
- Iteratively runs the networks on a set of input samples (typically, the validation set) and collects the network statistics on each layer output. This allows the Model Optimizer to estimate the dynamic range of all layer activations, weights and biases. This is required only if the target data type differs from the original data type that the network was trained with.
- Reports collected statistics for offline analysis that contains the following metrics: min, max, standard deviation, mean, percentiles (99%, 99.5%, 99.95%).
- Builds an optimal configuration for the target precision network and creates an inference network converted to run in the target data type
Feedback stage:
- Simulates the inference process to estimate any potential accuracy loss. Each layer in the produced network has a bit accurate implementation for a specific Intel® platform, which simulates both the mode of operation of the hardware, and the required data type precision.
- Reports the network performance in terms of accuracy and loss. These metrics are identical to those that would have reported using the dedicated scoring API (for example, OpenVX*, Intel® Math Kernel Library for Deep Neural Networks (Intel® MKL-DNN) and so on).
Deployment stage: The output provided by the Model Optimizer is an Intermediate Representation (IR) of the network. The IR is an input into the Inference Engine. The IR consists of two files:
- A topology file - an XML file that describes the network topology
- A trained data file - a .bin file that contains the weights and biases binary data
Workflow
Use the following typical workflow diagram to see where the Model Optimizer fits in when performing inference on a trained deep neural network model.
A brief description of using the Model Optimizer workflow is:
- Optional / if you use layers: Configure the Model Optimizer for the deep learning framework that you used to train the model.
- Provide as input a trained network that contains the network topology, parameters, and the adjusted weights and biases.
- Run the Model Optimizer to perform specific model optimizations, such as horizontal fusion of specific network layers.
Inference Engine
The Inference Engine is the second of the two key components of the Intel® CV SDK.
The Inference Engine uses the Intermediate Representation files that result from running the Model Optimizer and provides an optimized C++ application on embedded platforms. The Engine helps application execution with computational graph analysis, scheduling, model compression.
The Inference Engine has:
- A core library
- Four hardware-specific libraries
- A plugin for Intel® Xeon® and Intel® Core™ processors with Intel® AVX2, Intel Atom® processors ("CPU Plugin")
- A plugin for Intel® HD Graphics ("GPU Plugin")
- A plugin for Intel® Arria® A10 discrete cards ("FPGA Plugin")
- Third party libraries
Use the following typical workflow diagram to see where the Inference Engine fits in when performing inference on a trained deep neural network model:
Inference Engine Workflow
A brief description of the Inference Engine workflow is:
- Use the model as input. The model is in the form of Intermediate Representation (IR) that was produced by Model Optimizer.
- Optimize inference execution for target hardware.
- Deliver the inference solution to one or more embedded inference platforms.