The latest threat to global health is the outbreak of COVID-19, a respiratory disease named Coronavirus Disease 2019. As a subfield of Artificial Intelligence, Computer Vision has enjoyed recent success in solving various complex problems in health care. Therefore, it has the potential to contribute to the fight against COVID-19.
This article will provide an easy-to-read summary of recent research on the use of computer vision technology to combat COVID-19. Particularly, you will learn:
- Innovation and emerging AI technologies to address COVID-19 challenges
- The use of Computer Vision technology to combat coronavirus
- A list of applications for COVID-19 diagnosis, prevention, and treatment
Providing this review and summary of great research in this field, we hope to accelerate the adoption of AI technologies and existing efforts.
Background and Challenges of COVID-19
Coronavirus is a virus that causes COVID-19, an infectious disease. ‘CO’ stands for corona, ‘VI’ for virus, and ‘D’ for disease. The COVID-19 pandemic outbreak drives the need to fight against an invisible threat that is challenging to detect and contain.
The virus outbreak has posed critical challenges for public health, governments, and medical communities across the world. The use of emerging AI technology may help to contribute to the fight against the threat to the human population.
Computer Vision Technology to Combat Coronavirus
Researchers and governments worldwide are trying to explore new diagnostic and treatment techniques to battle the challenges of COVID-19. Computer Vision deals with how computers can analyze and understand visual information. With advances in deep learning, a subset of machine learning, computer vision methods have shown enormous scope in multiple application areas and industries, especially in healthcare and medical research.
Some of the most popular computer vision applications in healthcare include disease diagnosis, prognosis, surgery, therapy, medical image analysis, and drug discovery. Hence, there is a vast potential to use AI vision technology to fight COVID-19 for disease diagnosis, prognosis, prevention, control, treatment, and disease management.
In the following, we provide an overview of recent applications of computer vision for coronavirus control. Most of the applications listed are based on emerging technologies and by no means production-ready solutions. More medical research and data will be needed to build highly accurate yet practical deep-learning products.
Computer Vision for COVID-19 Diagnosis
A fast, reliable, widely available, and affordable diagnostic process is an essential part of the fight against COVID-19. The goal of diagnosis is “the making of a judgment about the exact character of the disease”, and the prognosis is “a doctor’s judgment of the expected development of a disease and the chances of getting better”.
1.) X-Ray Radiography
Digital chest x-ray radiography (CXR) comes with lower costs and a wider availability for detecting chest pathology compared to CT imaging. Hence, digital X-ray imagery with computer-aided diagnoses can be used for the detection of COVID-19 features as it is used for different diseases such as cancer and cardiac diseases. However, the visual detection of COVID-19 features is challenging and requires pre-processing the visuals to enhance the poor contrast of soft tissues in X-ray imagery.
Multiple deep-learning computer vision models exist for x-ray-based COVID-19 diagnosis. The most popular one for detecting COVID-19 cases with chest radiography images is named COVID-Net and was developed by Darwin AI, Canada. It is trained with a dataset referred to as COVIDx and is based on a total of 16’756 chest X-ray images across 13,645 patient cases. The COVID-Net computer vision model achieved 92.4% accuracy for COVID-19 diagnosis.
The goal is to aid clinicians in deciding which treatment strategy to employ depending on the cause of infection. The current COVIDx dataset is constructed by open-source chest radiography datasets.
2.) Computed Tomography
Computed Tomography (CT) is a non-invasive test to generate a precise image of a patient’s chest using a radiology examination. The method provides a more detailed image than a conventional x-ray and gives a detailed view of bones, fats, muscles, and organs. Researchers identified observable CT features of COVID-19 pneumonia, such as ground-glass opacification, and consolidation affecting the lower lobes – with additional features to be observable depending on the stage of the disease.
Diagnosis of COVID-19 as an image segmentation problem with deep learning aims to identify the infected area in CT images between healthy and infected patients. The Renmin Hospital of Wuhan University (China) used the trained model to accelerate the diagnosis of new COVID-19 cases. It is based on a CT image dataset of 46’096 images of both healthy and infected patients that expert radiologists have labeled. The dataset was collected from 106 patients (51 confirmed COVID-19 patients and 55 control patients).
The method is based on the UNet++ semantic segmentation model, used to extract valid areas in the images. The model achieved a per-patient accuracy of 95.24%, with a positive prediction value of 84.62% and a negative prediction value of 100%.
COVNet is another method for COVID-19 diagnosis. As a binary classification problem, the model was made available online for public use. It extracts visual features from volumetric chest CT using transfer learning based on the ResNet50. Lung image segmentation was performed as a pre-processing task using the U-Net model. The model used a dataset of 4’356 chest CT exams from 3’322 patients collected from 6 hospitals. The sensitivity and specificity for COVID-19 are 90% with a 95% confidence interval (CI).
There are publicly available datasets, potentially, the largest of its kind is the COVID-CT dataset of the University of San Diego. The dataset provides 349 CT images containing clinical findings of COVID-19. The AI model trained to demonstrate its potential achieved 85% accuracy.
Computer Vision for COVID-19 Prevention
The World Health Organisation provides guidelines on infection prevention and control (IPC) strategies to reduce the risk factors. Coronavirus prevention aims to limit transmission in healthcare settings using early recognition, source control, standard precautions for all patients, administrative controls, and environmental and engineering controls. Computer Vision applications can be used to support the implementation of such IPC strategies.
3.) Masked Face Recognition
A key strategy in the early stage of disease progression is the use of face masks and personal protective equipment to limit the respiratory virus spread, especially in older adults who are at higher risk of developing it. Multiple countries implemented the use of masks as a control strategy. Computer Vision systems can be used to facilitate its implementation greatly.
A Masked Face Recognition approach based on a multi-granularity masked face recognition model achieved 95% accuracy on a masked face image dataset. The data was made publicly available for research and contained three types of masked face datasets:
- The Masked Face Detection Dataset (MFDD) is used to train a masked face detection model that is the basis for the subsequent masked face recognition task.
- The Real-world Masked Face Recognition Dataset (RMFRD) is supposed to be the world’s largest real-world masked face dataset, with 5’000 pictures of 525 people wearing masks and 90’000 images of the same 525 subjects without masks.
- The Simulated Masked Face Recognition Dataset (SMFRD) is a simulated masked face data set covering 500’000 face images of 10’000 people.
4.) Thermography
Infrared thermography is an early detection strategy to identify people infected with COVID-19, mainly applied in public places such as airports or malls. Medical applications of infrared thermography include fever screening.
An approach is the use of a mobile platform for automatic fever screening based on infrared forehead temperature.
Infection screening is possible with systems using thermography and CCD cameras for non-contact vital signs measurements by Feature matching and MUSIC Algorithms.
For earlier SARS spread control, computer vision systems could be used for fever screening using thermography stations at entrances.
5.) Pandemic Drones
Pandemic drones have been used for prevention and control purposes. Hence, drones use remote sensing and digital imagery to identify infected people.
Such systems have previously been used for remote life sign monitoring in disaster management. Another similar application is vision-guided robot control for 3D object recognition.
6.) Germ Screening
Germ scanning can be used to combat COVID-19: A Convolutional Neural Network (CNN) for germ scanning, such as the identification of bacteria light-sheet microscopy image data, achieved over 90% accuracy.
AI Vision for COVID-19 Treatment
To date, there is no specific treatment for the COVID-19 virus. However, many of the symptoms can be treated. Therefore, treatment will depend on identifying the patient’s clinical condition.
7.) Disease Progression Score
Clinical management practices can be improved by classifying patients based on the severity of the disease. For example, clinical institutions can use computer vision to identify critically ill patients to direct medical attention (critical patient screening).
A disease progression score can be used to classify different types of infected patients. The score is calculated by measuring infected areas from CT images to measure the progression of patients over time and identify the most critically ill patients.
Depth cameras and deep learning have been used as abnormal respiratory patterns classifiers to perform an accurate and unobtrusive yet large-scale screening of people infected with the COVID-19 virus. The Respiratory Simulation Model (RSM) is first proposed to fill the gap between a large amount of training data and scarce real-world data.
People infected with COVID-19 are found to have more rapid respiration. Hence, the model is based on Gated recurrent units (GRUs) neural network to classify six clinically significant respiratory patterns to identify critically ill patients. The model can classify respiratory patterns with an accuracy of 94.5%.
8.) Support Covid-19 Vaccination Development
Deep feature representation learning can be used for Quantitative structure-activity relationship (QSAR) analysis by incorporating 360° images of molecular conformations into deep learning. QSAR (Quantitative structure-activity relationship) analysis using deep learning based on a novel molecular image input technique can be used for drug discovery and thus support the development of vaccines.
What’s Next?
Due to the multidisciplinary nature of computer vision technology, it can support teams working in different fields and industries to combat the coronavirus pandemic challenges. As a result, AI vision approaches are becoming increasingly popular, and new applications are currently being introduced and tested at a high pace.
If you enjoyed reading this article, we suggest reading about:
- Popular computer vision applications
- Learn how to evaluate the performance of machine learning models
- Read about why computer vision is difficult to implement
- How to get started with computer vision (Guide)
- A Beginner Guide to Generative Adversarial Networks (GANs)
