Video analytics use artificial intelligence to complete various tasks by applying computer vision and deep learning to video footage or live video streams. Video analytics is also sometimes referred to as video content analysis or intelligent video analytics.
Novel video analytics technologies are rapidly gaining popularity. Key adopters are both companies seeking to use the latest AI technologies to solve long-standing problems and those that have been operating video surveillance systems before the emergence of Artificial intelligence (AI).
Advances made in Deep and Machine Learning, both subsets of AI, have made it possible for video analytics to transform the task automation landscape that once required human interference to be successfully automated.
Trends in the video analytics market have been ever-evolving. The most recent advances in video analytics using Deep Learning for video content analysis, the ability to conduct real-time video processing, and the heightened accuracy of video recognition software.
Video Analytics Processes
How does object detection in video analytics happen?
Real-time Object Detection in video feeds has been a possibility for years by the emergence of algorithms such as Mask R-CNN or YOLO. These algorithms come pre-made and ready to detect the difference between objects in a field of view. For example, they allow video analyzing programs to detect and track objects such as vehicles, people, traffic lights, etc., in real-time. These objects are labeled and can be used for tasks such as vehicles or people-counting in crowded areas.
Motion detection and video analytics
Video motion detection is a way of defining activity in a scene by analyzing the differences in a series of images. Video motion detection is usually carried out by processes such as frame referencing or pixel matching.
Frame referencing and pixel matching involve detecting horizontal or vertical changes between video frames and regarding them as detections. The technique is common for analyzing videos using motion detection. It can either be built into a network video product (such as IP/CCTV cameras) or made available with video management software.
Video Analytics Market
Traditional players in the video analytics market include Cisco, Avigilon, Axis Communications, Aventura Systems, Genetec, IBM, IntelliVision, Bosch Security, Huawei, and more. The video analytics market is segmented as services and software. Most companies specialize in creating video analytics products that can be consumed (services) or software necessary for products to be successful (software).
The largest applications in the video analytics market involve security: incident detection, intrusion management, people counting, traffic monitoring, Automatic Number Plate Recognition (ANPR), facial recognition, AR, ego-motion estimation. In addition, video analytics has been useful for industries such as security, retail, healthcare and hospitality, and more.
Recently, novel computer vision platforms have been introduced, allowing businesses to deliver customized video analytics applications. Video analytics solutions built with low-code development platforms help businesses to adopt custom video analytics solutions while offering the functionality, speed, simplicity, and flexibility of ready-made software solutions.
The latest AI low-code platform, designed specifically for computer vision, is powered by the Swiss tech company viso.ai. Viso’s technology is worth mentioning because it is probably instructive of where the entire video analytics software space may be headed. Today, viso.ai has been a leader in AI computer vision software to create deep learning video analytics solutions that process video feeds of numerous cameras in real-time with deployed AI algorithms.
Video Analytics in Specified Industries
Video analytics has been working to provide solutions for security by creating a general means for identifying and detecting different objects in video streams. Such technology is useful for tracking people or objects of interest in videos or identifying and detecting intruders. Using video analytics for these purposes allows certain objects to be flagged and alarms to be raised on suspicious behavior.
Vertical Motion Detection
A specific instance of video analytics for security could be a fence-climbing detection system. Security staff is usually trained to know that people walking outside a fence is considered regular, but climbing on top of or struggling with the fence is irregular.
Video analytics software trained to recognize the subtle differences in motion direction between the regular and irregular behavior involving the fence can be linked to the real-time video feed from security cameras.
If someone were to begin climbing the fence, the software will recognize the vertical motion as an abnormal occurrence and create an alarm of some sort. Comparatively, if someone were to walk next to the fence, they would be creating horizontal motion, which is not classified as suspicious activity by the detection system.
There exist multiple video analytics applications in different variations. For example, Bosch Video Analytics can be used to detect a person climbing a fence in an area of view. In this application, the video analytics capabilities are based on integrated object detection algorithms that run directly on-device instead of an external server to perform detection in real-time (Edge Computing).
A variety of rules can be run simultaneously and can send alerts directly from the camera via text message, email, or to a video management system.
Video Feed Object Classification
Video feed object classification involves detecting dangerous objects in a live camera feed or given video. Small differences between objects that are sometimes even hard to see by security guards in front of a camera can be detected by video analytics programs trained to find minuscule differences that could make the difference between a hazardous and safe object.
X-ray security screening, for example, can use video analytics programs trained to do object classification on real-time feeds of baggage at security check-ins to identify specific objects of interest, such as sharp tools or weapons. Such technology has already been implemented worldwide as its accuracy increases. The Transportation Security Administration (TSA) introduced computed tomography scanners (CT) with state-of-the-art 3-D technology at U.S. airport checkpoints.
The technology is currently being enhanced to increase the accuracy to detect objects in video frames and has yet been successfully tested using images.
Similar to the motion detection discussed in the fence example, other kinds of behavior are also relevant grounds for video analytics to be able to classify. For example, behavior tracking involves human behavior in relation to both themselves and larger objects, such as vehicles, and what it entails for the safety of a general area. The following are two smaller-scale examples of behavior tracking implemented in video analytics.
- Loitering Detection: Video analytics are trained to notice when people or vehicles remain in a defined zone longer than the user-defined time allows. For the safety of the area, an alarm could be activated depending on the preferences of the program implementer. This behavior is effective in the real-time notification of suspicious behavior around pharmacy departments, ATMs, narcotic dispensaries, and other locations.
- Stopped Vehicle Detection: This portion of video analytics is useful for preventing vehicles from idling or stopping in an unauthorized location for prolonged periods of time. Vehicles stopped near a sensitive area longer than the user-defined time allows are detected. This behavior is ideal for stopping vehicles from obstructing loading and receiving docks, enforcing parking rules, and decreasing vehicle wait time at valet services or parking gates. Stopped vehicles in moving roadways can also indicate unreported accidents or vehicle issues, and such technology can alert proper authorities of the instances.
- Camera Sabotage: Advanced video loss detection can recognize when a live video stream has been compromised or tampered with. For example, if a vandal paints or covers a lens or reaches to move a fixed camera away from an intended scene, an alarm is triggered.
The retail industry can implement AI analytics for video streams in multiple situations. These components of retail management help streamline operations and create better customer experiences without increasing human responsibility or adding other operational costs relating to expensive equipment.
- Queue management: Video analytics provides information on better policies for checkouts and can even set stores up for check-out free capability. It allows stores to conduct self-checkout and honor-code activities without the fear of shoplifting or other nefarious infringements. Queue management can also provide insights into what is and isn’t working to manage the size of queues throughout stores. During the pandemic, for example, queue management could be essential for preventing spread.
- People counting: People counting can be conducted using video analytics. Retail involves a lot of experimenting with displays and marketing strategies. Observing or having access to how many customers come in and when is helpful for stores to know what is working in terms of marketing and product overview. In addition, noticing how many customers spend prolonged periods of time near which displays are useful for the store because it improves the customer experience and business for the store. In terms of people counting, video analytics provides operational insights and branding insights and reveals a host of other aspects of customer relationships.
Healthcare institutions have always prioritized having up-to-date technology to streamline costs and ensure the safety of their practices because healthcare as an industry is moderated by strict government and corporate legislation. Video analytics implementations can be useful in healthcare for mental health, the accuracy of diagnosis, and monitoring elderly or young patients in hospitals.
- At-home monitoring: Surveillance technology makes monitoring of elderly patients in care homes feasible and convenient for caretakers. Falls are a major cause of injury and death in older persons, which is why at-home monitoring is useful for detecting unusual positions or periods of time in which a person is on the floor or incapacitated. Personal medical devices can detect falls efficiently but require being worn at all times for effectiveness. Video analytics provides a more hands-free solution and can be modified to do more than just detect falls. For example, such a system could also determine if an elderly took a given medication when they were supposed to.
- Mental health: Combining advanced video analytics and machine learning with facial analysis and the expertise of human clinicians could enhance a healthcare provider’s ability to get to the right conclusion about a patient’s state of mental health. Video analytics can be trained to pinpoint differences in normal and abnormal facial or physical behavior. Healthcare professionals often record these nonverbal communications as part of their prognosis, but in a fairly subjective manner and only if they notice them. Video analytics in mental health applications ensure that subtle hints in a patient’s behavior do not go unnoticed.
- Biotechnology: Early screening of foodborne pathogens is a key to ensure food safety. Biosensors that aim to detect salmonella through smartphone video processing and fluorescence labeling are currently being researched. Video analytics can also analyze live feeds of bacteria and identify certain bacteria from others, making it useful for identifying differences in bacterial composure.
Real-time video analysis with deep learning algorithms has prominent use cases in smart cities. Read our article about a state-of-the-art list of the best and most valuable applications of computer vision in smart cities.
Multiple companies involved in video analytics are trying to develop more integrated solutions that have to do with cities. For example, in February 2019, Cisco initiated the prospect of developing a model to explore smart city solutions in Norway. A “smart city” uses digital technology to connect, protect, and enhance the lives of citizens. IoT sensors, video cameras, social media, and other inputs act as a nervous system, providing the city operator and citizens with constant feedback to make informed decisions.
City agencies can gain more citizen engagement and optimize operations through real-time data intelligence and intra-agency collaboration. From an economic standpoint, smart city facilitations drive new revenue streams and economic development by enhancing customer activity and behavior awareness.
Video analytics are useful for cities and towns that manage crowds of people and are part of the smart city model. Automatic Number Plate Recognition and traffic monitoring are two examples of video analytics being used within cities. These applications streamline otherwise cumbersome processes that require sufficient human intervention.
- ANPR: Automatic Number Plate Recognition (ANPR) consists of accurate systems capable of reading vehicle number plates without human intervention. Using high-speed image capture with supporting the illumination makes it possible for video analytic systems to detect and read plate numbers in near real-time. Therefore, characters of the license plates are recognized using Optical Character Recognition (OCR), converting the images into digital text strings. This makes it possible for video analytic systems to detect and record plate numbers. Modern ANPR programs create metadata sets for every detected license plate for the authorities to re-use in other systems. ANPR is useful for recording cars running red lights, traffic mishaps, and more.
- Traffic monitoring: Video analytics can provide insights useful for analyzing traffic and monitoring traffic jams. In addition to detecting dangerous accidents and situations, traffic monitoring gives quantitative insights on the number of vehicles in areas at specific times and traffic patterns. In the case of an accident, these analytical systems involving traffic analysis later provide police with assistance for collecting evidence in case of litigation.
- Vehicle counting: This aspect of video analytics involves differentiating between cars, trucks, buses, taxis in order to generate helpful statistics used to obtain insights about traffic. Speed cameras can record the concentration of fast-moving cars in one area compared to another, which can be helpful for the city to know which implementations of traffic control are effective. Vehicle counting also provides insights for when future road maintenance needs to occur.
Video analytics remains an interesting aspect and application of computer vision as a part of visual artificial intelligence.
If you enjoyed this article, we suggest you read more about the applications of computer vision. Articles we recommend include:
- Learn about Computer Vision, the technology behind video analytics
- Explore the 56 Most Popular Computer Vision Applications in 2021
- Everything you need to know about Image Recognition (Guide)