Face Blur for Data Privacy in Deep Learning

Face blur methods are used to develop privacy-preserving deep learning and computer vision applications. The use of face blurring for face anonymization has shown little impact on the accuracy of image recognition models. In this article, you will learn about:

1. Why we need privacy-preserving AI
2. How to use face blur technology
3. Effects on computer vision tasks
4. Alternative methods to anonymize faces with AI

About us: viso.ai provides Viso Suite, the world’s only end-to-end Computer Vision Platform. The technology helps global organizations to develop, deploy, and scale all computer vision applications in one place, and meet privacy requirements. Request a demo.

Why We Need Privacy-Preserving AI

The unprecedented accuracy of deep learning methods has turned them into the foundation of new AI-based applications. Hence, companies collect user data on a large scale since the success of deep learning techniques is directly proportional to the amount of data that can be used for training. However, the massive data collection required for deep learning comes with inevitable privacy issues.

Users’ personal, highly sensitive data, such as photos and videos, are kept indefinitely by the companies that collect them. As a result, users can neither delete it nor restrict the purposes for which it is used. Also, such data may potentially be subject to legal matters, and the collection is governed by a set of laws such as GDPR (General Data Protection Regulation) in Europe.

Many data owners, for example, medical institutions that want to apply deep learning methods to sensitive data like clinical records, are prevented by privacy and confidentiality concerns. Hence, the development of privacy-preserving large-scale deep learning comes with big benefits for multiple parties involved.

Preventing unauthorized access to sensitive information in private datasets became an important matter. But also public and widely used datasets like the ImageNet include a high number of images that contain people (as categories or co-occurring with other objects in images).

Face obfuscation is used to mitigate privacy issues based on face images. Object detection research typically assumes access to complete, unobfuscated images. In visual datasets, even if most categories are usually not people categories, many incidental people appear in the images, and their privacy is a concern.

Using Face Blur in Image Dataset

Face obfuscation, such as face blurring, has been shown to be effective for privacy protection. However, object recognition, object detection, and image segmentation typically use complete, unobfuscated images. Therefore, the following steps were applied by a method to blur faces in the ImageNet dataset.

1. Detect Faces with Face Annotation

Faces are ubiquitous in datasets, even for images that are not people categories directly, which indicates that privacy-aware image recognition is an important topic for public datasets. For example, when annotating the 1.4 million images of the ILSVRC dataset, over 550’000 faces were identified in 240’000 images (17% of all images have at least one face).

The first step for face obfuscation is face detection. Therefore, all faces are annotated using models to detect the individual faces, using a face detector such as Face Detection with SSD, OpenCV, Dlib, MTCNN Face Detection with TensorFlow, RetinaFace, MediaPipe or – if privacy policies allow – even Cloud APIs such as Amazon Rekognition or Google Cloud Vision API.

The obtained results can be further refined through synthetic data generation and data augmentation, along with accurate face image annotations and ML dataset evaluation techniques.

2. Blur Detected Faces for Face Obfuscation

In a subsequent step, a face segmentation model is trained to obfuscate sensitive image areas with pixel-based blurring, a widely used method for preserving privacy. This made it possible to create face-blurred versions of the ILSVRC or ImageNet dataset.

Face obfuscation has shown minimal impact on the accuracy of recognition models. Benchmarking with multiple deep neural networks on face-blurred images showed that the overall recognition accuracy drops only slightly (≤0.68%).

Effects of Face Blur on Computer Vision Tasks

Visual features learned on ImageNet are effective for a wide range of computer vision tasks. The results of benchmarking deep neural networks on face-blurred images demonstrated that face obfuscation offers privacy protection with minimal impact on accuracy.

Hence, face anonymization using face blurring does not significantly compromise accuracy on both image classification and downstream tasks. Such computer vision downstream tasks are object recognition, scene recognition, object detection, and face attributes classification.

Accordingly, the AI can still recognize a car even when people inside have their faces blurred. This makes the application of facial blur a feasible method to implement privacy-aware visual recognition.

Alternative Methods to Anonymize Faces with AI

Multiple techniques for visual identity obfuscation have evolved from simply covering the face with often unpleasant occluders, such as black boxes or mosaics, to more advanced methods that produce natural images.

• Face covering. Face anonymization methods similar to face-blurring are, for example, covering the face region with occluders, such as mosaic or a black bar. These methods are still the predominant techniques for visual obfuscation in videos or photos. However, they recently became less effective due to the improvement of CNN-based recognition methods (Convolutional Neural Networks).
• Face replacement. A new approach obfuscates identities in photos by head replacement, achieving highly realistic outputs while preserving a high similarity to the original image content.
• Removing of moving people. An alternative to blurring is a method to automatically remove and inpaint faces and license plates (e.g., pedestrians, vehicles) in Google street-view imagery. A moving object segmentation algorithm was used to detect, remove and inpaint moving objects with information from other views to obtain a realistic output image in which the moving object is not visible anymore.

What’s Next?

Obfuscation techniques such as face blurring have minimal impact on the accuracy of vision recognition models. Hence, we expect such face anonymization techniques to be more broadly implemented to support the development of privacy-preserving deep learning applications.

We recommend you to explore the following related topics:

• Overview of Privacy-Preserving Deep Learning
• Read Pytorch vs. Tensorflow: A Head-to-Head Comparison
• The COCO Dataset – What you need to know
• Deep Reinforcement Learning: How It Works and Real-World Examples