viso.ai
Search
Close this search box.

Top 18 Applications of Computer Vision in Security and Surveillance

computer vision surveillance security applications

Build, deploy, operate computer vision at scale

  • One platform for all use cases
  • Connect all your cameras
  • Flexible for your needs
Contents

In the last decade, computer vision has evolved as a key technology for numerous applications replacing human supervision and monitoring. This article provides a research overview of state-of-the-art computer vision in video surveillance and AI security monitoring.

We will discuss the following topics:

  • State-of-the-art AI Video Security Technology
  • Anomaly Detection and Response with Computer Vision
  • AI Vision Applications in Security and Surveillance
  • Starting with Computer Vision – Software Solution

About us: Viso Suite is the world’s only end-to-end Computer Vision Platform. The solution helps global organizations to develop, deploy, and scale all computer vision applications in one place. Get a demo for your organization.

State of the Art in AI Video Surveillance

Computer vision uses a combination of technologies to analyze and understand video data with computers. In surveillance and security industry applications, the primary goal of computer vision is to automate human supervision. The ability to capture and digitize real-life scenes provides new opportunities to detect threats better and earlier, quantify risk, and provide real-time security assessments.

The list of computer vision applications is expanding rapidly, driven by new Machine Learning, Edge Computing, Artificial Intelligence (AI), and IoT technologies that make AI-powered vision much more powerful, flexible, and scalable.

Edge AI for Computer Vision in Surveillance

Applying computer vision has only recently become possible through advances in deep learning and edge computing. Deep learning is a subfield of machine learning that enables machines to learn from training data and apply those algorithms to new data.

Edge Computing is the systematic process of moving computing tasks from the cloud to the network edge near the data source (camera). As a result, edge computing eliminates the challenges of connected cameras and devices, such as network congestion, constant connectivity, latency, robustness, privacy, and data management.

Modern computer vision systems use edge computing to process video without sending video data to the cloud or another storage unit. The combination of on-device machine learning and edge computing is also called Edge AI or Edge Intelligence. In surveillance and security applications of computer vision, those emerging technologies play an important role in enabling real-life AI applications. Also, Edge AI vision infrastructure allows significant cost reductions in large-scale and real-time computer vision systems.

 

High Level concept of Edge AI architectures
High-level concept of Edge AI architectures with local AI inferencing.

 

Intelligent Surveillance Cameras

With the widespread use of security cameras in public places, AI video analytics and scene understanding with computer vision have become essential features of surveillance systems. Visual data from camera streams contain rich information compared to other sources such as mobile location, GPS, radar signals, etc. Large-scale video analytics systems can collect statistical information about the status of road traffic, public places, buildings, or private areas.

Modern AI vision software can analyze video feeds from virtually any network camera. Depending on its hardware configuration, a single-edge device can process the video feed of multiple cameras. Powerful edge servers can analyze dozens to hundreds of cameras.

Some IP camera manufacturers or turnkey point solutions provide on-camera intelligence with an integration of the computing processor directly into the camera. However, enterprise systems usually separate AI computation from the camera – for several reasons.

First, businesses need to stay vendor-independent and maintain the ability to negotiate. Then, companies must avoid technology lock-in and ensure extensibility and integration with their systems. Also, cameras with integrated AI processing don’t allow scaling up hardware resources if a company needs to extend features or increase AI performance.

In addition, most businesses operate video systems with cameras from various brands, generations, and types (Sony, Panasonic, Axis, Hikvision, Dahua, Samsung, and so on). Replacing all cameras is too expensive, and standardizing would lead to lock-in costs. Also, most camera products are periodically replaced with a new model every two years.

 

Computer Vision Projects with Object detection
Computer Vision to analyze a camera stream

 

AI Video Surveillance Systems

In traditional video surveillance, systems fully depend on human operators and individual judgment and attentivity. Intelligent AI analysis supports human operators with fast, objective, and consistent information. Depending on the use case, the AI vision software performs detection and prediction tasks for traffic congestion, security threats, accidents, and other anomalies.

A typical computer vision system integrates multiple software capabilities, from data input acquisition to image preprocessing, deep learning inference, output aggregation, communication, and visualization. Such a computer vision system can run one or multiple applications that address specific problems (anomaly detection, pose estimation, object detection, etc.).

Computer Vision Applications vs. AI Models

An AI model is not a computer vision application; the terms are often used interchangeably, albeit incorrectly. A computer vision application contains a computer vision pipeline (or flow) with one or multiple AI models.

The AI model requires upstream functions to fetch and preprocess image data before feeding it into the model. Only then can an AI model perform an algorithmic task to transform video frames into specific metadata (e.g., classes of “person” with confidence). The raw model output requires interpretation and aggregation with logic to be useful for solving a business or security problem.

 

MindSpore can be implemented for a number of computer vision tasks including object detection.
A model for hardhat detection in construction use cases.

 

 

Anomaly Detection in AI Vision

What is Anomaly Detection?

In computer vision, anomaly detection is a sub-field of behavior understanding from surveillance scenes. Anomalies are typically aberrations of scene entities (people, vehicles, environment) and their interactions from normal behavior. Anomaly detection methods learn the normal behavior via training. Anomaly detection typically uses unsupervised or unsupervised learning, or a combination, of semi-supervised learning.

Use Cases of Anomaly Detection

Anything deviating significantly from normal behavior can be considered “anomalous.” Examples include vehicle presence on walkways, the sudden dispersal of people within a gathering, a person falling while walking, jaywalking, signal bypassing in traffic, or U-turns of vehicles at red signals.

Anomaly Detection Solution

An anomaly detection system leverages data acquisition, feature extraction, scene or activity learning, and behavioral understanding. Anomaly detection systems are architected towards specific use cases, optimized for particular deployment environments and camera positioning.

Systems for anomaly event detection are nontrivial and require a set of techniques that span multiple research fields. In general, such systems process video data to perform scene analysis using video processing techniques, vehicle and person detection and tracking, multi-camera-based techniques, intelligent event detection, and more.

Anomaly Types

In AI vision, there are different types of anomalies. The three types include point anomalies, contextual anomalies, and collective anomalies. Point anomalies include, for example, a non-moving car on a busy road or in a tunnel. Contextual anomalies could be normal in a different context.

For example, in slow-moving traffic, if a vehicle moves faster than others, what would be normal behavior in less dense situations? Collective anomalies occur when a group of instances together may cause an anomaly even though individually they may be normal, for example, a group of people dispersing within a short period.

Scope and Class of Anomalies

In the context of visual surveillance, it is common to see anomalies classified as global and local anomalies. Global anomalies can be present in a frame or segment of the video without specifying exactly where an event occurred (no localization).

Local anomalies usually happen within a specific area of the scene but may be missed by global anomaly detection algorithms. Some methods can detect both global and local anomalies.

Anomaly Detection in Security Monitoring

AI video analysis is used for anomaly detection in traffic, subway, campus, trains, boats, buildings, and public places. Examples of CCTV monitoring for anomaly detection in visual AI include stopped vehicle detection, panic detection, intrusion detection, or abnormal pedestrian activity recognition.

 

Deep learning anomaly detection for stopped vehicle detection
Deep learning anomaly detection for stopped vehicle detection – Viso Suite

 

Applications of Video Surveillance and Security

Intelligent video surveillance includes a wide range of applications and use cases for anomaly detection, object detection and tracking, movement analysis technologies, monitoring systems, prevention, identification, and warning systems. Cooperative video surveillance enables large-scale AI vision systems that integrate numerous cameras at remote locations.

Person Detection

People detection systems use object detection algorithms (for example the popular YOLOv7) to localize people in video feeds. Hence, automated single-person and multiple-person detection are key features of intelligent video surveillance systems.

Human detection also includes crowd analysis to estimate scene density and evaluate moving object interaction in crowded and uncrowded scenes (for example, at mega events).

 

People Detection and Tracking with privacy face blur applied
People Detection with privacy-preserving facial blur – Built with Viso Suite
People Movement Analysis

Path learning combines human detection with path modeling techniques and clustering to perform person movement analysis. For example, in Smart City applications, movement analysis is used to perform motion prediction and analyze vehicle behavior, pedestrian behavior, acceleration, movement speed, and trajectories.

Person Recognition and Biometric Security

Modern security systems for video surveillance use facial recognition technology for automated person identification. On a high level, such biometric security technologies perform a sequence of tasks, also called AI vision pipeline, to (1) detect people, (2) crop the face area, and (3) apply image classification to compare it to images of a database.

To comply with legal requirements, face recognition algorithms require sophisticated privacy and security infrastructure. In general, facial recognition technology is sensitive because it could be used to identify people in videos and pictures without their knowledge and permission.

Note: Our technology, the Viso Suite platform provides industry-leading privacy and security capabilities for real-world computer vision. The computer vision platform enables organizations worldwide to meet the strictest privacy requirements in the most challenging environments.

 

Face Detection with Facial Blurring
Real-time face recognition with facial blurring – Built with Viso Suite

 

Behavioral Biometrics in Computer Vision

Another alternative method of human recognition includes behavioral biometrics. Computer vision for behavioral biometrics is a modern technological research field that seeks to identify people using their habitual physical and mental traits. It has become a novel form of identity authentication as it can accurately distinguish individuals from one another without requiring any physical contact.

By analyzing subtle, individual characteristics such as facial expressions, hand gestures, body movements, and gait recognition – also called gait biometrics – this technology can detect and identify individuals. There is a wide range of applications that can be used for various purposes such as security and access control, personal recognition systems, customer service, fraud detection, deepfake detection, and general surveillance.

Human Behavior Understanding

Person detection, classification, and person tracking can perform human behavior understanding in video-based surveillance applications. Specific behavior patterns can be learned with classification models to recognize specific human actions.

This can perform aggression detection, brawl detection, robbery or theft detection, and more. Applications to analyze human behavior also include trajectory clustering using different machine learning and imaging technologies such as multi-camera detection.

 

Trajectory analysis with computer vision
Trajectory analysis with computer vision in RetailViso Suite
Illegal Activity Detection

Human action recognition and motion pattern detection can detect suspicious activities, events, or behaviors. Popular techniques include pose estimation, 3D sensing, learning, and classification to detect violations of guidelines or the law. Illegal activities can include littering, loitering, begging, and more.

Driver and Traffic Safety Applications

Stationary or vehicle-mounted camera systems can perform different types of anomaly detection. Applications include lane departure warning, pedestrian detection, and adaptive warning systems. In-vehicle safety and alarm systems include driver monitoring, for example, seatbelt detection or gaze recognition to analyze tiredness and fatigue.

 

Gaze recognition with the OpenVINO framework
Gaze recognition with the OpenVINO framework

 

AI Security Systems

Dangerous Objects and Weapon Detection

Real-time object detection uses deep learning to detect and localize specific objects in video scenes. Common object recognition applications in security include weapon detection (firearms or knives) or protective equipment detection.

As for many computer vision applications, object detection in real-life settings is very challenging to implement; we will discuss the reasons later in this article.

 

Automated weapon detection with deep learning
Weapon detection with deep learning, using training data from movie scenes.
Intrusion Detection and Virtual Fencing

Virtual fencing of sensitive locations is a popular feature of AI vision surveillance systems. Specific regions of interest mark virtual fences to detect intrusion events, prompting the system to send alerts to security teams.

 

People detection with video analytics
Virtual fence with a region of interest (ROI)
Automated Video Summarization

Video summarization is a process that allows people to quickly gain an understanding of the content and key messages within a video without needing to watch it in its entirety. For example, in shopping malls, fitness centers, or transportation hubs, operators spend a significant amount of time watching live and recorded video streams.

AI vision algorithms are used to perform video summarization, synopsis generation, and content-based video retrieval. In video surveillance, the historical data output of the deep learning models can be used to identify specific events and find related video material.

 

Infrastructure Security

Accident and Traffic Incident Detection

In traffic management and surveillance, vehicle detection and tracking algorithms are used to identify specific incidents and events. Such systems are popular in smart city applications, also for traffic parameter collection, vehicle counting, video-based tolling, traffic flow analysis, and behavior understanding. Other use cases include accident detection, highway vehicle detection, and vehicle classification (profiling).

Common methods apply foreground segmentation or background subtraction in combination with convolutional neural networks (CNN) for deep learning tasks.

 

Traffic analytics
Traffic analytics with deep learning – Viso Suite

 

Smart Parking and Vehicle Surveillance

Vehicle or moving object detection and tracking are used in combination with license plate recognition. Image classification algorithms can determine vehicle model and type, color, or logo recognition. CCTV camera-based vehicle surveillance is popular in smart parking lot analytics to identify and track the occupancy of multiple parking spaces with computer vision.

 

parking lot detection
Parking lot detection with computer vision

 

Unattended Object Detection

Abandoned object detection in security is a critical aspect of keeping locations, properties, and safety monitoring. It involves the use of automated systems that can detect any items left behind in public or private areas without human intervention.

A deep learning model can be trained to detect items like bags, boxes, packages, and other objects that could pose potential security threats. When objects are detected, an alarm or other alert can be sent to a monitoring center or security personnel. With this alert, they perform potential threat detection before it becomes a real problem.

 

Security application for Abandoned Object Detection
Security application for Abandoned Object Detection – Viso Suite

 

Vehicle Identification and Numberplate Recognition

Vision-based vehicle identification uses automatic number plate recognition (ANPR) and vehicle feature detection (color, type) to identify and count individual vehicles using cameras. ANPR is also called LPR (License Plate Recognition).

Vehicle identification software first detects the vehicle with object detection, locates the license plate, and finally reads the number plate using optical character recognition (OCR).

 

Real-time multi ANPR
Real-time video-based automated number plate recognition (ANPR)

 

Safety Assessment

AI camera systems can implement vision-based gap analysis, threat assessment, risk, conflict, and accident detection. Deep learning models perform recognition to digitize real-world situations and gather data to model and predict threat situations.

The ability to digitize the visual world and translate it into metadata can help provide high-level security assessments, which are also important for insurance applications. For example, we can create dynamic reports that provide information about vehicle or person movements, interactions, or terrain information.

Infrastructure Security

Visual surveillance uses computer vision for roadside warning systems and decision support in the security monitoring of public places, critical infrastructure, and transportation infrastructure. Modern AI vision systems integrate with almost all CCTV monitoring and VMS systems. For example, our computer vision platform Viso Suite can acquire and process the video streams of existing cameras. This can be done across different camera models and vendors.

 

Small object detection in traffic analysis with computer vision
Real-time object detection in traffic analysis with computer vision

 

Emergency Management With Computer Vision

AI vision systems can perform emergency classification of natural events including storm detection, flooding detection, or smoke and fire detection. In addition, “abrupt event detection” can identify anomalies across different locations and cameras in large-scale settings. When the system detects an emergency, it can send an alert directly to law enforcement, prompting an appropriate emergency response.

Surveillance and security AI systems can also detect human-made emergencies such as road accidents, dangerous crows, weapon threat recognition, drowning person detection, and injured person or falling person detection.

 

Challenges of AI vision

Video-based anomaly detection in camera security systems is very challenging. Several factors make the real-life applications of computer vision very difficult to implement and scale:

  • Lack of real-life data: There is a huge need for real-life data collection to train effective algorithms and build computer vision applications that perform well in real-world settings.
  • Illumination: Managing variations of illumination is difficult because trained features are hard to extract from the videos.
  • Pose and perspective: The camera angles that determine the surveillance area have a substantial impact on the performance of deep learning algorithms. This is because the appearance of objects or people may change depending on their distance from the camera.
  • Heterogenous objects: Learning the movement of heterogeneous objects and entities in a scene can be difficult at times. The variability of appearance considerably lowers the application performance.
  • Sparse vs. Dense: The methods used for detecting anomalies in sparse and dense conditions are different. Some methods are suitable for event recognition in sparse environments but can generate many false negatives in dense scene-based conditions, for example, with large crowds.
  • Occlusion: Detection and tracking under occlusion with partially or fully hidden instances (people or objects) is very challenging though this task is comparably easy for humans.

 

Starting With Computer Vision

Computer vision is a nontrivial technology, and applications are almost always unique and require a high level of integration and configuration. Eventually, organizations are looking for ways to consolidate their computer vision systems, to overcome the sprawl of isolated point solutions, different hardware, and individual tools.

Our computer vision platform Viso Suite is the most powerful, end-to-end computer vision platform. Leading organizations worldwide use Viso technology to deliver their computer vision applications on one unified, automated infrastructure. To learn more about the solutions Viso Suite can provide for your project, book a demo with our team.

 

Get the Viso Suite Whitepaper 2022
Get the Viso Suite Whitepaper