This paper addresses how the Smart Video (SV) system architecture is increasing in complexity and evolving into new industries and use cases. It articulates the advantage of using Intel® software tools such as Intel® Media SDK, Intel® Computer Vision SDK, software libraries, Intel® AMT, Intel® vPro technology and Intel® hardware to enable these smart video systems. The main areas this article covers are Smart cameras that hardware accelerate encoding, transcoding and decoding, smart video gateway’s hardware and software, video analytics, deep learning and integration with the Internet of Things.
Introduction to Intel® Smart Video Technologies
The next generation of Smart Video Systems have complex and integrated computer architectures that are rapidly evolving. Technologies from computer fields such as HD graphics, real-time video encoding and transcoding, scalable video streaming and storage, artificial intelligence and the Internet of Things are expanding the abilities of modern Smart Video Systems.
Smart Video Systems are also expanding into new market segments such as autonomous vehicles, smart lighting, industrial machine vision and drones. They are increasingly used in city surveillance, retail interactions, banking and finance surveillance, traffic monitoring, railway supervision, education management, airport security and many other market applications (some are industry markets i.e. retail some are application i.e. airports). For example, an automotive dealership may use a smart video system in order to ensure the security of property on their auto lot. A factory may use a smart video system to monitor equipment and determine if there are any malfunctions that need to be serviced. A hospital or a government building may use a V system to monitor the people that are entering restricted areas. And retail stores often use them to monitor customer transactions and provide reporting to police and authorities in the case of a robbery.
Each of these scenarios not only changes how the SV system will be architected and installed, but also they affect the policies and procedures enforced during the use of the system.
Intel focuses on enabling technologies that are flexible, horizontal and scalable so that businesses are able to implement their own regulations and procedures when deciding how to use the recorded video.
Overview of Intel® Smart Video Technologies
Current video systems use camera sources to create video and stream it to local or cloud-based storage where it is analyzed and any necessary actions are taken. The scalability of the system is limited by the amount of available bandwidth and storage, as well as, the ability to rapidly analyze and determine if alerts or actions need to be taken.
Intel® technologies address customer challenges by focusing on each area of the technology stack and providing solutions that can be mixed and integrated to build next-generation systems.
Let’s focus on four areas where Smart Video System technology is rapidly evolving:
- Smart cameras that hardware accelerate encoding, transcoding and decoding
- Smart Video gateway and software
- Video analytics and Deep Learning
- Integration with the Internet of Things
Smart Cameras
Smart cameras powered by Intel technology will continue to increase resolutions to HD and 4K/UHD. Intel® processors based on next generation Intel® architecture, such as SkyLake, provide hardware acceleration for media operations in HEVC, H.264 to 4K30, VP9, and JPEG based on Intel® Quick Sync Video technology (GPU). Higher resolutions are achieved with lower power and bandwidth requirements.
The Intel® Media SDK delivers a higher level set of libraries, tools, and sample code that define cross-platform APIs. This SDK provides a unified abstraction layer for media decode and encode. That is capable of accessing hardware acceleration and includes an OpenCL interface to allow custom media processing capabilities.
Higher resolutions and video pipelines enable the creation of multiple video streams that lower bandwidth requirements, enabling more cameras to be deployed in a smart video system ultimately producing an increase in the number of simultaneous video feeds.
Smart Video Gateways
Smart Video Gateways serve as aggregation points for analytics services, network management, security management, and integration with automation services and the Internet of Things.
By moving high-end computation closer to the video sources, Intel based gateways will be able to analyze the video onsite without moving video over the Internet or to centralized cloud based data centers. This reduces Smart Video System bandwidth requirements and increases scalability from tens of video streams to hundreds or thousands of simultaneous video streams.
Intel offers smart video systems based on a range of processors so the computational needs of your Smart Video System can grow with your business needs. Systems based on the Intel Atom® platform, the Intel® Core™ processor family and the Intel® Xeon® platform span the range of video computational needs.
Here is a profile of a potential mid-level and an advanced gateway.
Entry-level NVR, DVR, Transcoder (8/16 HDD, 16-20ch decode)
Intel® Celeron® processor family +24ch 1080p decoding
APL: 4k H.265, 16-20ch 1080p decode capability and VA
Intel Advantages:
- Intel® graphics with Intel® Quick Sync Video
- Headroom for Apps, Video Analytics, Software RAID
- Ability to Scale
- IOT Portfolio: Silicon, Software, Security, Manageability, Ecosystem
Performance
Intelligent Video Server, Transcoder
(Data Center / High-Performance NVR )
(>16 HDD, >20ch decode [more encode], and Deep Learning)
Intel® Xeon® processor E3-1225v5: Hi End NVR/Transcoder/VA Server
Intel® Core™ i3/5/7 family-based NVR with VA
Intel Advantages
- Intel® Quick Sync Video – HW Accelerated De/Trans-code w/ Codecs
- Heterogenous Compute Resources – CPU/GPU/FPGA
- Analytics Performance from Processor and Memory (Intel® SSD)
- Big Data SW Ecosystem, Open Standards, Libraries & Frameworks
- Server-class Reliability, Optimized on Various Workloads
Smart Video Gateway Software
Intel provides a large number of libraries that system integrators and vendors can use to create smart video services. The algorithms span a range of applications from providing statistical information about the video to allowing higher level descriptions and actionable information.
Hardware Accelerated Analytics
These smart video libraries make it possible to build custom analytics algorithms that are hardware optimized for image processing, cryptography, math processing & neural networks routines, data analytics and high availability threading:
• Intel® Integrated Performance Primitives (IPP)
• Intel® Math Kernel Library (MKL)
• Intel® Data Analytics Acceleration Library (DAAL)
• Intel® Threading Building Blocks (TBB)
Classification, Analytics and Computer Vision
Higher level analytics can be customize for different situations. For example, if several smart cameras are monitoring an area that only allows restricted personal, artificial intelligence and deep learning can be used to train the system to identify authorized people and sound the alarm for unauthorized people.
Intel provides the Intel® Computer Vision SDK based OpenVX*, OpenCL™ software technology, and OpenCV. Intel® Computer Vision SDK is a specialized set of libraries for enabling computer vision processing.
Reducing Latency
Computational power near the video source is also important in reducing latency in timing sensitive application. In autonomous vehicle applications, pedestrian recognition needs to happen immediately and avoid the latencies associated with the cloud.
Enabling analysis of video nearer the smart camera reduces network transit for high-bandwidth video streams. Low bandwidth alerts and automatic actions can be triggered.
Higher Level Software Libraries
Intel® Distribution for Caffe* is a set of libraries from machine learning to enable deep learning applications that can be deployed on a Smart Video System. By using advanced smart cameras and video gateways, multiple video streams can be encoded in an efficient manner.
Deep learning techniques are used in tasks such as object recognition, object classification, object or person tracking, intrusion detection, scene analysis, counting people, facial detection, facial recognition, recognition of age gender or behavior, license plate recognition, vehicle detection, 3D depth analysis, navigation and many other useful applications of video analysis.
Smart Video Manageability
Once you have the correct hardware and software layers you also need a manageability layer. Intel provides Intel® AMT technology, enabled by Intel® vPro™ Technology, which will allow you to remotely administer your entire smart video system. This includes upgrading firmware, managing deployment and decommissioning lifecycle steps, and monitoring in real-time the status of each of the components in your DDS system.
Intel silicon and software is enabling next-generation architecture of smart video systems and security and surveillance systems.