In this article, we are going to discuss the Intel Neural Compute Stick 2 (NCS 2) that is based on the Intel Movidius Myriad X chip. Learn about the advantages of using the NCS 2 for computing artificial intelligence (AI) on the edge.
Specifically, we will cover:
- What is the Intel Neural Compute Stick 2
- Performance of the Movidius Myriad X
- Introduction to OpenVINO and the Myriad Development Kit (MDK)
- Benefits of the Intel Neural Compute Stick 2
What is Intel Neural Compute Stick 2?
The edge is becoming an increasingly popular destination for deploying computer vision or deep learning models. Edge computing provides advantages such as local data processing, filtered data transfer to the cloud, or faster decision-making.
The Intel Neural Compute Stick 2 is powered by the Intel Movidius X VPU to power on-device AI applications at high performance with an ultra-low power consumption. With new performance enhancements, the Intel Movidius Myriad X VPU is a power-efficient solution revolving around edge computing that brings computer vision and AI applications to edge devices such as drones, smart cameras, smart home, security, VR/AR headsets, and 360-degree cameras.
The NCS 2 is a small, fanless neural network training and deployment device that can be used for AI programming at the edge.
AI accelerators like the Intel Stick 2 VPU are useful for accelerating data-intensive deep learning inference on edge devices in a very cost-effective way. These accelerators work by assisting the edge device computer processing unit (CPU) by taking over the mathematical burden needed for running deep learning models.
Edge accelerators allow for deep learning models to be run at low costs, low power consumption, and faster speeds. The benefits are primarily measured using throughput, value, latency and efficiency.
Movidius Myriad X – High Performance Computer Vision Inferencing
The Intel Movidius Myriad X Vision Processing Unit (VPU) is Intel’s first VPU to feature the Neural Compute Engine — a dedicated hardware accelerator for deep neural network inference. The Movidius Myriad X VPU is sold as a chip embedded in a Neural Compute Stick (similar to a USB drive) that is specifically built for processing images and video inputs. The USB containment of the chip allows it to be easily compatible with a Raspberry Pi or Intel NUC device (any popular computing architecture such as x86 PC’s).
The Intel Movidius Neural Compute Stick engine is a chip-form hardware piece designed to run deep neural networks at high speed and low power without compromising accuracy, enabling computer vision processes in real time.
This engine is implemented within the USB casing along with Movidius Myriad X. When used concurrently, the Neural Compute Engine, 16 powerful SHAVE cores, and high throughput memory fabric make the Intel Movidius Myriad X neural compute stick ideal for training and deploying deep neural networks and computer vision products.
This information shows the relative success rates of various machine learning algorithms on different inferencing engines. If we zoom in on the specs concerning the ssd300-CF (Caffe backend), the throughput rate is directly proportional to the accuracy, while the latency rate is inversely proportional to quality.
As we can see, the Movidius accelerator has higher throughput for the ssd300-CF than any of the core i3 to i9’s (except for the Intel Core i9-10920X). It has lower latency than any inferencing engine, which makes it a relatively robust platform for running artificial intelligence computer vision models.
OpenVINO with Movidius Neural Compute Stick 2
The Intel Distribution of OpenVINO (Open Visual Inference and Neural Network) allows computer vision models trained in the cloud to be run at the edge. The OpenVINO developer’s toolkit contains a full suite of development and deployment tools best used in conjunction with Movidius Myriad X. The toolkit aims to facilitate faster inference of deep learning models by creating cost-effective and robust computer vision applications.
Supporting a large number of deep learning models out of the box, it expedites the computer vision application production process by cutting down on raw creation and setup time. The pre-trained models, which range from deep learning frameworks such as YOLO (You Only Look Once) to R-CNN and ResNet, allow developers to create models that carry out complex computer vision applications such as face and person detection, vehicle detection, and people counting.
The Myriad Development Kit (MDK) further includes necessary development tools, frameworks, and APIs to implement custom vision, imaging, and deep neural network workloads on the chip. Existing Convolutional Neural Network (CNN) models can be converted into OpenVINO Intermediate Representation (IR) which drastically reduces the size of the model while simultaneously optimizing it for inferencing.
Advantages of Intel Neural Compute Stick 2
The Intel NCS 2 can easily be attached to an Ubuntu 16.04 PC or Raspberry Pi running Raspbian Stretch OS, making it very easy and affordable to get started.
As an edge computing solution, it offers all the benefits of edge AI hardware such as privacy through local processing, performance due to low latency and reliability (decentralized, does not depend on network connections).
The Intel Movidius Neural Compute Stick features 16 Programmable 128-bit VLIW Vector Processors which allows the user to run multiple imaging and vision application pipelines at the same time. The 16 vector processors optimize the system for flexibility surrounding computer vision workloads. As a member of the Movidius VPU family, the Movidius Myriad X VPU is known for extremely low, decreased power consumption.
Based on the maximum performance of operations-per-second over all available compute units, the Intel Neural Compute Stick 2 is able to deliver a total performance of over 4 trillion operations-per-second.
What makes Intel Movidius Myriad X specifically suitable for computer vision is the enhanced vision accelerators. 20 accelerators allow the Neural Compute Stick to be capable of performing tasks such as optical flow and stereo depth without introducing additional compute overhead.
The new stereo depth accelerator on Movidius Myriad X can concurrently process 6 camera inputs (3 stereo pairs) each running 720p resolution at 60 Hz frame rate, which is comparable and competitive with edge accelerators such as Google’s Coral TPU.
If you enjoyed this article, we suggest you read more about: