Applications of Deep Learning have shown great potential in the automotive sector, both inside the vehicle, for example, for advanced driving assistance systems and autonomous driving, and outside the vehicle, such as during manufacturing, sales, and after-sales processes.
In this article, we will explore the following sections:
- Key technology trends in computer vision
- Use cases of computer vision in automotive manufacturing
- In-vehicle computer vision applications
- Get started
Key Technology Trends in AI Vision
Deep learning and computer vision have many applications in the automotive industry, such as autonomous driving and robotics, optimization in the manufacturing process, monitoring quality issues, and connected vehicle services.
The Era of Deep Learning
The increased deployment of mobile edge devices and IoT sensors has led to the creation of a massive amount of image and video data. To effectively analyze this kind of data, new machine learning methods, in particular, deep learning, are required to process the increasing amount of unstructured data (video, images), from cameras in or on vehicles, machines, or factories. The combination of IoT and AI enables the implementation of scalable, intelligent systems (AIoT).
Compared to traditional machine vision systems, deep learning models achieve far greater performance, robustness, and flexibility. Hence, deep learning is the gold standard for most modern vision systems. The technology utilizes large neural networks with many hidden layers, so-called Deep Neural Networks (DNN), for feature generation, learning, classification, and prediction.
In the automotive industry, the use of Convolutional Neural Networks (CNNs) for vehicle and lane detection has recently dramatically improved. This made it possible to replace some expensive sensors, such as LIDAR, with cameras.
The Future of AI Is at the Edge
Distributed AIoT systems combine the Internet of Things with AI technology to improve the data collection process and build distributed intelligent systems. Recent advances in AI hardware and AI model optimization (lightweight models) made it possible to move machine learning towards the network edge, a concept called Edge AI.
Instead of sending all visuals to the Cloud and processing the data there, it is much more efficient, secure, robust, and scalable, to perform machine learning at the source of the data, and on-device (Edge Intelligence). Hence, it becomes possible to build and deploy powerful physical-world computer vision applications.
Autonomous driving is a premier example of an Edge AI application. The smart vehicles are connected to the Cloud but process all data locally. The latency needs to be ultra-low, and the systems need to function even while temporarily losing the connection to the internet.
As a result, distributed cloud-edge systems are very scalable. Increasing the number of system endpoints (edge devices) is much less likely to cause bandwidth bottlenecks.
Types of Computer Vision Systems
In manufacturing, most computer vision systems fall into one of the following categories:
- #1 Type: Automation systems, to conduct tasks otherwise carried out by human operators, reduce human error, as well as production line robots.
- #2 Type: Intelligent systems, to generate insights, early-detect issues that lead to costly business interruptions, facilitate planning, and monitor the shop floor and inventory to make better decisions faster.
- #3 Type: Quality control systems, to automatically inspect possible production failures or part defects during assembly or in the final product.
Deep Learning in Automotive Manufacturing
Automotive manufacturing assembly tasks require visual inspections such as scratch identification on machined surfaces, part identification, and selection, to ensure product and process quality. Traditionally, visual inspection and quality control have been conducted manually, by trained operators. With the rise of AI and automation technologies in manufacturing, the artificial perception of computer vision systems is rapidly gaining in popularity.
Traditional machine vision systems greatly depend on lighting, enclosure, occlusion, and quality of image video feed. As a result, the settings needed to be highly standardized. Hence, adjustments were very costly, and improperly designed vision systems can negatively impact the operating cycle time of the factory.
Modern deep learning-based technologies are much more flexible, robust, and scalable. AI models can be utilized to conduct and assist in visual inspection tasks while leaving very little footprints in the manufacturing environment.
Automatic Vision System for Visual Defect Detection
In automotive industries, computer vision is broadly used in various applications to improve product quality. Most of the customer returns of defective products are due to appearance defects, often related to the painting. In general, the visual defect detection process is conducted by operators. The manual inspection is subjective, difficult, and time-consuming.
Automatic computer vision systems can inspect the surface of manufactured components, for example, wheels. Multiple cameras placed over the production line can be used for defect detection in real time. The systems monitor the wheel coating intensity and anomalies such as a small decreasing amount of paint that would reveal a sudden problem in the painting process.
Deep Learning Part Inspection in the Assembly Lines
Part inspection and fault localization with deep learning has enormous potential in automotive applications of AI vision. For example, for brake parts, it is of great importance to detect damaged manufactured parts before they are assembled in any vehicle. Here manual inspection is extremely difficult and hard to conduct without assistance.
Compared to conventional image processing, deep learning methods (Single Shot Detector – SSD, Faster RCNN) are more robust in detecting multiple faults. When training a deep learning algorithm using transfer learning on a custom collected dataset for fault detection, such algorithms achieved an accuracy of 95.6% on cylindrical grey shade brakes.
Robotics and Smart Machines
Robotics requires high-performing computer vision sub-systems. Deep learning is widely used to recognize features from camera images to control the machine. While near real-time object detection algorithms are already widely advanced, the field of object tracking still poses many challenges.
Researchers are working on using deep learning to enable self-learning robots that become more intelligent over their lifetime (read more about semi-supervised machine learning).
Track Tool Movement in Automotive Assembly Plants
In automobile assembly, technicians use torque tools to mount bolts to parts of vehicles. Different bolts require varying torque levels to be fastened correctly. Hence, intelligent systems deliver torque levels in a specific order that must be strictly followed by the technician. Because of the pre-specified order, it is critical to follow the correct order precisely.
Computer vision applications in automotive have been tested to identify, correct, and document human errors in the bolt-securing process. Therefore, the application helps to detect incorrect ordering on an automotive factory assembly line. Visual AI can select a desired order of bolts, detect the visitation of a torque tool to each bolt location, and report errors made in the sequence of actions in the vehicle assembly line.
Ergonomic Risk Assessment
In automotive manufacturing, ergonomic risk assessments help to increase occupational health and safety by reducing the risk of work-related musculoskeletal disorders. Digital video analysis with computer vision and machine learning techniques is used to perform accurate ergonomic risk assessments. Therefore, camera systems with deep learning can perform unsupervised ergonomic assessment ergonomics of multiple workers simultaneously, even in sub-optimal conditions (poor illumination, occlusion).
Neural networks detect the worker’s skeletons and body-joint positions and infer angles. The information can be used to perform Rapid Upper Limb Assessment (RULA) evaluation scores.
In-Vehicle Computer Vision
Autonomous Driving
Different aspects of autonomous driving vehicles require machine learning and computer vision technologies. An important part includes the processing of an immense amount of data from cameras, in combination with other sensors (e.g., Lidar), and the learning of driving car situations and driver behavior.
Deep Head Pose Estimation
An in-car automation application of deep learning involves head pose estimation from 2D or 3D depth data through a CNN. Common use cases include detecting driver inattention, one of the main causes of traffic accidents. Therefore, real-time driver monitoring systems help to improve driver safety with computer vision models.
Real-time Traffic Road Signs Detection
The recognition of traffic road signs uses computer vision algorithms to detect road signs and their shape (triangle, square, and rectangle). Traffic sign recognition is an important field of computer vision, especially relevant for autonomous vehicles and Advanced Driver Assistance Systems (ADAS).
Cameras for traffic sign detection are utilized in many other applications, such as road safety, or highway asset maintenance and management, to check the condition of signs on major roads.
Real-time prediction of automotive collision risk
Multiple automotive applications, including Advanced Driver Assistance Systems for collision avoidance and warnings, require estimating the future risk of a driving scene. Visual AI systems process the video stream of conventional, dashboard-mounted cameras to predict the collision risk over an intermediate time horizon and support absolute speed estimation.
Get Started
Computer vision systems in the automotive industry help to improve operational control. Connected AI vision systems provide the infrastructure to detect and react to manufacturing issues that would lead to costly business interruptions. With visual AI, it becomes possible to reduce human error, increase operational productivity by automating manual tasks, and improve employee safety.
Need AI vision at your organization? At viso.ai, we power a computer vision platform to build AI vision applications in automotive and industrial manufacturing. Check out the most complete end-to-end platform Viso Suite with industry-leading AI model scalability at the Edge.
Contact our team to get started.
Read more about related topics:
