Face blur methods are used to develop privacy-preserving deep learning and computer vision applications. The use of face blurring for face anonymization has shown little impact on the accuracy of vision recognition models. In this article you will learn about:
- Why we need privacy-preserving AI
- How to use face blur technology
- Effects on computer vision tasks
- Alternative methods to anonymize faces with AI
Why We Need Privacy-Preserving AI
The unprecedented accuracy of deep learning methods has turned them into the foundation of new AI-based applications. Hence, companies collect user data on a large scale since the success of deep learning techniques is directly proportional to the amount of data that can be used for training. However, massive data collection required for deep learning comes with inevitable privacy issues.
Users’ personal, highly sensitive data such as photos and videos are kept indefinitely by the companies that collect it. Users can neither delete it nor restrict the purposes for which it is used. Also, such data may potentially be subject to legal matters.
Many data owners, for example, medical institutions, that want to apply deep learning methods to sensitive data such as clinical records, are prevented by privacy and confidentiality concerns. Hence, the development of privacy-preserving large-scale deep learning comes with big benefits for multiple parties involved.
Preventing unauthorized access to sensitive information in private datasets became an important matter. But also public, and widely used datasets like the ImageNet include a high number of images that contain people (as categories or co-occurring with other objects in images).
Face obfuscation is used to mitigate privacy issues based on face images. Object recognition research typically assumes access to complete, unobfuscated images. In visual datasets, even if most categories are usually not people categories, many incidental people appear in the images, and their privacy is a concern.
Using Face Blur in Image Dataset
Face obfuscation such as face blurring has been shown to be effective for privacy protection. However, object recognition, object detection and image segmentation typically use complete, unobfuscated images. The following steps were applied by a method to blur faces in the ImageNet dataset.
1. Detect Faces with Face Annotation
The focus is on a prominent type of private information – faces. In an initial step, the faces are annotated using automatic face detectors such as Amazon Rekognition to detect the faces. Then, the results were refined through crowdsourcing on Amazon Mechanical Turk to obtain accurate face annotations.
Faces are ubiquitous in datasets, even for images that are not people categories directly, which indicates that privacy-aware recognition is an important topic for public datasets. When annotating the 1.4 million images of the ILSVRC dataset, over 550’000 faces were identified in 240’000 images (17% of all images have at least one face).
2. Blur Detected Faces for Face Obfuscation
In a subsequent step, sensitive image areas are obfuscated using blurring, a widely used method for preserving privacy. This allowed building a face-blurred version of the ILSVRC dataset.
The face obfuscation has minimal impact on the accuracy of recognition models. Benchmarking with multiple deep neural networks on face-blurred images showed that the overall recognition accuracy drops only slightly (≤0.68%).
Effects of Face Blur on Computer Vision Tasks
Visual features learned on ImageNet are effective for a wide range of computer vision tasks. The results of benchmarking deep neural networks on face-blurred images demonstrated that face obfuscation offers privacy protection with minimal impact on accuracy.
Hence, face anonymization using face blurring does not significantly compromise accuracy on both image classification and downstream tasks. Such computer vision downstream tasks are object recognition, scene recognition, object detection, and face attributes classification.
Accordingly, the AI is still able to recognize a car even when people inside have their faces blurred. This makes the application of facial blur a feasible method to implement privacy-aware visual recognition.
Alternative Methods to Anonymize Faces with AI
Multiple techniques for visual identity obfuscation have evolved from simply covering the face with often unpleasant occluders, such as black boxes or mosaics, to more advanced methods that produce natural images.
- Face covering. Face anonymization methods similar to face-blurring are for example covering the face region with occluders, such as mosaic or a black bar. These methods are still the predominant techniques for visual obfuscation in videos or photos. However, they recently became less effective due to the improvement of CNN-based recognition methods (Convolutional Neural Networks).
- Face replacement. A new approach obfuscates identities in photos by head replacement, achieving highly realistic outputs while preserving a high similarity to the original image content.
- Removing of moving people. An alternative to blurring is a method to automatically remove and inpaint faces and license plates (e.g. pedestrians, vehicles) in Google street-view imagery. A moving object segmentation algorithm was used to detect, remove and inpaint moving objects with information from other views, to obtain a realistic output image in which the moving object is not visible anymore.
Obfuscation techniques such as face blurring have minimal impact on the accuracy of vision recognition models. Hence, we expect such face anonymization techniques to be more broadly implemented to support the development of privacy-preserving deep learning applications.
We recommend you to read more about related topics: