Video analytics use artificial intelligence to complete various tasks by applying computer vision and deep learning to video footage or live video streams. Video analytics is also sometimes referred to as video content analysis or intelligent video analytics.
Novel video analytics technologies are rapidly gaining popularity. Key adopters are both companies seeking to use the latest AI technologies to solve long-standing problems and those that have been operating video surveillance systems before the emergence of Artificial intelligence (AI).
Advances made in deep and machine learning, both subsets of AI, have made it possible for video analytics to transform the task automation landscape that once required human interference to be successfully automated.
Trends in the video analytics market have been ever-evolving, with the most recent advances in video analytics involving the use of deep learning for video content analysis, ability to conduct real time video processing, and heightened accuracy of video recognition softwares.
Video Analytics Processes
How does object detection in video analytics happen?
Real time object detection in video feeds has been a possibility for years by the creation of algorithms such as Mask R-CNN or YOLO. These algorithms come pre-made and ready to use for detecting the difference between objects in a field of view. For example, they allow video analyzing programs to detect objects such as vehicles, people, traffic lights, etc. in real-time. These objects are labelled, and can be used for tasks such as vehicle or people-counting in crowded areas.
Motion detection and video analytics
Video motion detection is a way of defining activity in a scene by analyzing the differences that occur in a series of images. Video motion detection is usually carried out by processes such as frame referencing or pixel matching. Frame referencing and pixel matching involves detecting horizontal or vertical changes between frames of video and regarding them as detections. The functionality is common for analyzing videos using motion detection and can either be built into a network video product (such as IP/CCTV cameras) or made available with video management softwares.
Video Analytics Market
Traditional players in the video analytics market include Cisco, Avigilon, Axis Communications, Aventura Systems, Genetec, IBM, IntelliVision, Bosch Security, Huawei, and more. The video analytics market is segmented as services and software, where most companies specialize in creating video analytics products that can be consumed (services) or software necessary for products to be successful (software). The largest applications in the video analytics market involve security: incident detection, intrusion management; people counting, traffic monitoring, Automatic Number Plate Recognition (ANPR), facial recognition, AR, ego-motion estimation. Video analytics has been useful for industries such as security, retail, healthcare and hospitality, and more.
Recently, novel computer vision platforms have been introduced, allowing businesses to deliver customized video analytics applications. Video analytics solutions built with low-code development platforms help businesses to adopt custom video analytics solutions while offering the functionality, speed, simplicity, and flexibility of ready-made software solutions.
The latest AI low-code platform, designed specifically for computer vision, is powered by the Swiss tech company viso.ai. Viso’s technology is worth mentioning because it is probably instructive of where the entire video analytics software-space may be headed. Today, viso.ai has been known as a leader in AI computer vision software to create deep learning video analytics solutions that process video feeds of numerous cameras in real-time with deployed AI algorithms.
Video Analytics in Specified Industries
Video analytics have been working to provide solutions for security by creating a general means for identifying and detecting different objects in video streams. Such technology is useful for tracking people or objects of interest in videos, or identifying and detecting intruders. Using video analytics for these purposes allow certain objects to be flagged, and alarms to be raised on suspicious behavior.
Vertical Motion Detection
A specific instance of video analytics for security could be a fence-climbing detection system. Security staff are usually trained to know that people walking on the outside of a fence is considered regular, but climbing on top of or struggling with the fence is irregular. The difference between climbing a fence as compared to walking next to it is the direction of motion.
Video analytics software that is trained to recognize the subtle differences in motion direction between the regular and irregular behavior involving the fence can be linked to the real-time video feed from security cameras. If someone were to begin climbing the fence, the software will be able to recognize the vertical motion as an abnormal occurrence, and create an alarm of some sort. Comparatively, if someone were to walk next to the fence, they would be creating horizontal motion, which is not suspicious in relation to the fence and detection system.
There exist multiple video analytics applications in different variations. For example, Bosch Video Analytics can be used to detect a person climbing a fence in an area of view. In this application, the video analytics capabilities are based on integrated object detection algorithms that run directly on-device instead of an external server, to perform detection in real-time (Edge Computing). A variety of rules can be run at the same time and can send alerts directly from the camera via text message, email, or to a video management system.
Video Feed Object Classification
Video feed object classification involves detecting hazardous or dangerous objects in a live camera feed or given video. Small differences between objects that are sometimes even hard to see by security guards in front of a camera can be detected by video analytics programs trained to find miniscule differences that could make the difference between a hazardous and safe object.
X-ray security screening, for example, can use video analytics programs trained to do object classification on real-time feeds of baggage at security check-ins to identify specific objects of interest such as sharp tools or weapons.
Such technology has already been implemented world wide, as accuracy increases. Transportation Security Administration (TSA) announced its plan to test computed tomography scanners (CT) in 2018, a state-of-the-art 3-D technology at select U.S. airport checkpoints. As the agency continues to raise the baseline for aviation security, the technology intends to enhance critical explosives and other threat items detection capabilities at airport checkpoints. The technology is currently being enhanced to increase accuracy to detect objects in video frames, and has yet been successfully tested using images.
Similar to the motion detection discussed in the fence example, other kinds of behavior are also relevant ground for video analytics to be able to classify. Behavior tracking involves human behavior in relation to both themselves and larger objects, such as vehicles, and what it entail for the safety of a general area. The following are two smaller-scale examples of behavior tracking implemented in video analytics.
- Loitering Detection: Video analytics are trained to notice when people or vehicles remain in a defined zone longer than the user-defined time allows. For the safety of the area, an alarm could be activated depending on the preferences of the program implementer. This behavior is effective in real-time notification of suspicious behavior around pharmacy departments, ATMs, narcotic dispensaries and other locations.
- Stopped Vehicle Detection: This portion of video analytics is useful for preventing vehicles from idling or stopping in an unauthorized location for prolonged periods of time. vehicles stopped near a sensitive area longer than the user-defined time allows are detected. This behavior is ideal for stopping vehicles from obstructing loading and receiving docks, enforcing parking rules, and decreasing vehicle wait time at valet services or parking gates. Stopped vehicles in moving roadways can also be an indicator of unreported accidents or vehicle issues, and such technology can alert proper authorities of the instances.
- Camera Sabotage: Advanced video loss detection can recognize when a live video stream has been compromised or tampered with. For example, if a vandal paints or covers a lens, or reaches to move a fixed camera away from an intended scene, an alarm is triggered.
The retail industry can implement AI analytics for video streams in multiple situations. These components of retail management help streamline operations and create better customer experiences without increasing human responsibility or adding other operational costs relating to expensive equipment.
- Queue management: Video analytics provides information on better policies for checkouts and can even set stores up for check out free capability. It allows stores to conduct self-checkout and honor-code activities without the fear of shoplifting or other nefarious infringements. Queue management can also provide insights on what is and isn’t working in terms of managing the size of queues throughout stores. During the pandemic, for example, queue management could be essential for preventing spread.
- People counting: People counting can be conducted using video analytics. Retail involves a lot of experimenting with displays and marketing strategies. Observing or having access to how many customers come in and when is helpful for stores to know what is working in terms of marketing and product overview. Noticing how many customers spend prolonged periods of time near which displays are useful for the store because it improves the customer experience and business for the store. Video analytics in terms of people counting provides operational insights and branding insights, as well as revealing a host of other aspects of customer relationships.
Healthcare institutions have always placed utmost priority on having up to date technology to streamline costs and ensure safety of their practices because healthcare as an industry is so moderated by strict government and corporate legislation. Video analytics implementations can be useful in healthcare for mental health, accuracy of diagnosis, and monitoring elderly or young patients in hospitals.
- At-home monitoring: Surveillance technology makes monitoring of elderly patients in care homes feasible and convenient for caretakers. Falls are a major cause of injury and death in older persons which is why at-home monitoring is useful for detecting unusual positions or periods of time in which a person is on the floor or incapacitated. Personal medical devices can detect falls efficiently but require being worn at all times for effectiveness. Video analytics provides a more hands-free solution and can be modified to do more than just detect falls. For example, such a system could also determine if an elderly took a given medication when they were supposed to.
- Mental health: Combining advanced video analytics and machine learning with facial analysis and the expertise of human clinicians could enhance a healthcare provider’s ability to get to the right conclusion about a patient’s state of mental health. Video analytics can be trained to pinpoint differences in normal and abnormal facial or physical behavior. These nonverbal communications are often recorded by healthcare professionals as part of their prognosis, but in a fairly subjective manner and only if they notice them. Video analytics in mental health applications ensure that subtle hints in a patient’s behavior do not go unnoticed.
- Biotechnology: Early screening of foodborne pathogens is a key to ensure food safety. Biosensors that aim to detect salmonella through smartphone video processing and fluorescence labeling are currently being researched. Video analytics can also analyze live feeds of bacteria and identify certain bacteria from others, making it useful for identifying differences in bacterial composure.
Multiple companies involved in video analytics are trying to develop more integrated solutions that have to do with cities. In February 2019, Cisco initiated the prospect of developing a model to explore smart city solutions in Norway. A “smart city” uses digital technology to connect, protect, and enhance the lives of citizens. IoT sensors, video cameras, social media, and other inputs act as a nervous system, providing the city operator and citizens with constant feedback so they can make informed decisions. City agencies can gain more citizen engagement and optimize operations through real-time data intelligence and intra-agency collaboration. From an economic standpoint, smart city facilitations drive new revenue streams and economic development by enhancing awareness of customer activity and behavior.
Video analytics are useful for cities and towns that manage crowds of people and are part of the smart city model. Automatic Number Plate Recognition and traffic monitoring are two examples of video analytics being used within cities. These applications streamline otherwise cumbersome processes that require sufficient human intervention.
- ANPR: Automatic Number Plate Recognition (ANPR) consists of accurate systems capable of reading vehicle number plates without human intervention. Using high speed image capture with supporting illumination, detection of characters within the images provided, verification of the character sequences as being those from a vehicle license plate, character recognition to convert image to text, and more makes it possible for video analytic systems to detect and record plate numbers. These programs create sets of metadata that identify an image containing a vehicle license plate and the associated decoded text of that plate for authorities to later analyze. ANPR is useful for recording cars running red lights, traffic mishaps, and more.
- Traffic monitoring: Video analytics can provide improved and developed insights useful for analyzing traffic and monitoring traffic jams. In addition to detecting dangerous accidents and situations, traffic monitoring gives quantitative insights on the number of vehicles in areas at specific times as well as traffic patterns. In the case of an accident, these analytical systems involving traffic analysis later provide police with assistance for collecting evidence in case of litigation.
- Vehicle counting: This aspect of video analytics involves differentiating between cars, trucks, buses, taxis, in order to generate helpful statistics used to obtain insights about traffic. Speed cameras can record the concentration of fast moving cars in one area as compared to another, which can be helpful for the city to know which implementations of traffic control are effective or not. Vehicle counting also provides insights for when future road maintenance needs to occur.
Video analytics remains an interesting aspect and application of computer vision as a part of visual artificial intelligence.
If you enjoyed this article, we suggest you read more about the applications of computer vision. Articles we recommend include:
- Learn about Computer Vision, the technology behind video analytics
- Explore the 56 Most Popular Computer Vision Applications in 2021
- Everything you need to know about Image Recognition (Guide)