Facial Demorphing via Identity Preserving Image Decomposition
0
Sign in to get full access
Overview
- This paper proposes a method for "facial demorphing" - the process of converting a morphed, mixed-identity face image into its component face identities.
- The key innovation is an "identity-preserving image decomposition" technique that can separate a morphed face into its individual identity components while preserving the original facial features.
- This method has applications in areas like image forensics, face recognition, and photo editing.
Plain English Explanation
The researchers have developed a way to "unblend" a face image that has been morphed or combined from multiple people's faces. This could be useful for things like detecting fake or manipulated photos, improving face recognition systems, or allowing people to edit their own face photos.
The key idea is that they can take a mixed-identity face image and separate it back into the individual faces that were combined to create it. Their method is able to do this while preserving the original facial features, so the resulting individual face images still look natural.
For example, if you had a photo where two people's faces were blended together, this technique could analyze the image and pull out the two separate faces that were used to create the blended image. This could be very helpful in situations where you need to identify the original individuals, like in forensics or security applications.
Technical Explanation
The core of the researchers' approach is an "identity-preserving image decomposition" technique. This involves taking a morphed face image and breaking it down into its underlying facial identity components while preserving the original facial features and structure.
The method works by first learning a generative model that can create realistic face images from a latent representation. It then uses an optimization process to find the latent representations that, when combined, best reconstruct the input morphed face image. Crucially, these latent representations correspond to the individual facial identities that were blended together in the original.
The key innovation is that this decomposition is done in a way that preserves the original facial features - the resulting individual face images retain the same expressions, poses, lighting, etc. as the input morphed face. This allows the method to separate the identity information from the other visual factors in the image.
Critical Analysis
The researchers acknowledge several limitations of their approach. First, it relies on having a strong pre-trained generative model of faces, which may not always be available. The performance also degrades when dealing with more extreme face morphs or low-quality input images.
Additionally, while the method preserves facial features, it does not preserve other contextual information like background elements. This means it may not be suitable for all applications, like verifying the provenance of a full scene image.
An open question is how well this approach would generalize to morphs involving more than two identities. The experiments only considered binary blends, so further research would be needed to assess its performance on more complex composite faces.
Conclusion
This paper presents a novel technique for "facial demorphing" - separating a mixed-identity face image back into its individual component faces while preserving the original facial structure and features. This could be a valuable tool for applications like image forensics, face recognition, and photo editing.
The key innovation is an identity-preserving image decomposition method that can analyze a morphed face and extract the underlying facial identities that were blended together. While the approach has some limitations, it represents an important step forward in being able to reliably unmix and analyze composite face images.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
0
Facial Demorphing via Identity Preserving Image Decomposition
Nitish Shukla, Arun Ross
A face morph is created by combining the face images usually pertaining to two distinct identities. The goal is to generate an image that can be matched with two identities thereby undermining the security of a face recognition system. To deal with this problem, several morph attack detection techniques have been developed. But these methods do not extract any information about the underlying bonafides used to create them. Demorphing addresses this limitation. However, current demorphing techniques are mostly reference-based, i.e, they need an image of one of the identities to recover the other. In this work, we treat demorphing as an ill-posed decomposition problem. We propose a novel method that is reference-free and recovers the bonafides with high accuracy. Our method decomposes the morph into several identity-preserving feature components. A merger network then weighs and combines these components to recover the bonafides. Our method is observed to reconstruct high-quality bonafides in terms of definition and fidelity. Experiments on the CASIA-WebFace, SMDD and AMSL datasets demonstrate the effectiveness of our method.
Read more8/21/2024
🛠️
0
Leveraging Diffusion For Strong and High Quality Face Morphing Attacks
Zander W. Blasingame, Chen Liu
Face morphing attacks seek to deceive a Face Recognition (FR) system by presenting a morphed image consisting of the biometric qualities from two different identities with the aim of triggering a false acceptance with one of the two identities, thereby presenting a significant threat to biometric systems. The success of a morphing attack is dependent on the ability of the morphed image to represent the biometric characteristics of both identities that were used to create the image. We present a novel morphing attack that uses a Diffusion-based architecture to improve the visual fidelity of the image and the ability of the morphing attack to represent characteristics from both identities. We demonstrate the effectiveness of the proposed attack by evaluating its visual fidelity via the Frechet Inception Distance (FID). Also, extensive experiments are conducted to measure the vulnerability of FR systems to the proposed attack. The ability of a morphing attack detector to detect the proposed attack is measured and compared against two state-of-the-art GAN-based morphing attacks along with two Landmark-based attacks. Additionally, a novel metric to measure the relative strength between different morphing attacks is introduced and evaluated.
Read more4/11/2024
🖼️
0
Unrecognizable Yet Identifiable: Image Distortion with Preserved Embeddings
Dmytro Zakharov, Oleksandr Kuznetsov, Emanuele Frontoni
Biometric authentication systems play a crucial role in modern security systems. However, maintaining the balance of privacy and integrity of stored biometrics derivative data while achieving high recognition accuracy is often challenging. Addressing this issue, we introduce an innovative image transformation technique that effectively renders facial images unrecognizable to the eye while maintaining their identifiability by neural network models, which allows the distorted photo version to be stored for further verification. While initially intended for biometrics systems, the proposed methodology can be used in various artificial intelligence applications to distort the visual data and keep the derived features close. By experimenting with widely used datasets LFW and MNIST, we show that it is possible to build the distortion that changes the image content by more than 70% while maintaining the same recognition accuracy. We compare our method with previously state-of-the-art approaches. We publically release the source code.
Read more8/29/2024
0
Greedy-DiM: Greedy Algorithms for Unreasonably Effective Face Morphs
Zander W. Blasingame, Chen Liu
Morphing attacks are an emerging threat to state-of-the-art Face Recognition (FR) systems, which aim to create a single image that contains the biometric information of multiple identities. Diffusion Morphs (DiM) are a recently proposed morphing attack that has achieved state-of-the-art performance for representation-based morphing attacks. However, none of the existing research on DiMs have leveraged the iterative nature of DiMs and left the DiM model as a black box, treating it no differently than one would a Generative Adversarial Network (GAN) or Varational AutoEncoder (VAE). We propose a greedy strategy on the iterative sampling process of DiM models which searches for an optimal step guided by an identity-based heuristic function. We compare our proposed algorithm against ten other state-of-the-art morphing algorithms using the open-source SYN-MAD 2022 competition dataset. We find that our proposed algorithm is unreasonably effective, fooling all of the tested FR systems with an MMPMR of 100%, outperforming all other morphing algorithms compared.
Read more4/10/2024