SoK: Gradient Leakage in Federated Learning
2404.05403
0
0
Abstract
Federated learning (FL) enables collaborative model training among multiple clients without raw data exposure. However, recent studies have shown that clients' private training data can be reconstructed from the gradients they share in FL, known as gradient inversion attacks (GIAs). While GIAs have demonstrated effectiveness under emph{ideal settings and auxiliary assumptions}, their actual efficacy against emph{practical FL systems} remains under-explored. To address this gap, we conduct a comprehensive study on GIAs in this work. We start with a survey of GIAs that establishes a milestone to trace their evolution and develops a systematization to uncover their inherent threats. Specifically, we categorize the auxiliary assumptions used by existing GIAs based on their practical accessibility to potential adversaries. To facilitate deeper analysis, we highlight the challenges that GIAs face in practical FL systems from three perspectives: textit{local training}, textit{model}, and textit{post-processing}. We then perform extensive theoretical and empirical evaluations of state-of-the-art GIAs across diverse settings, utilizing eight datasets and thirteen models. Our findings indicate that GIAs have inherent limitations when reconstructing data under practical local training settings. Furthermore, their efficacy is sensitive to the trained model, and even simple post-processing measures applied to gradients can be effective defenses. Overall, our work provides crucial insights into the limited effectiveness of GIAs in practical FL systems. By rectifying prior misconceptions, we hope to inspire more accurate and realistic investigations on this topic.
Create account to get full access
Overview
- This paper provides a systematic overview of gradient inversion, a technique used to recover the original training data from the gradients shared during the federated learning process.
- Federated learning is a machine learning approach that allows multiple devices to collaboratively train a model without sharing their raw data.
- Gradient inversion poses a significant privacy risk, as it can potentially expose sensitive information about the participants' data.
Plain English Explanation
The paper examines a technique called "gradient inversion" in the context of federated learning. Federated learning is a way for multiple devices to work together to train a machine learning model without having to share their raw data. This is important for protecting the privacy of the participants.
However, gradient inversion is a method that can be used to try to recover the original training data from the gradients that are shared during the federated learning process. This poses a significant privacy risk, as it could potentially expose sensitive information about the participants' data.
The paper provides a comprehensive overview of how gradient inversion works and the various approaches that have been developed to perform this task. By understanding the different techniques and their potential impacts, researchers and practitioners can better assess the privacy risks associated with federated learning and develop more robust privacy-preserving solutions.
Technical Explanation
The paper presents a systematic overview of gradient inversion, a technique used to recover the original training data from the gradients shared during the federated learning process.
The authors first describe the system model for federated learning, which involves a central server coordinating the training process across multiple client devices. They then categorize the different gradient inversion techniques based on the information available to the attacker, such as the model architecture, the optimization algorithm, and the gradients themselves.
The paper also discusses various defenses against gradient inversion, including gradient compression, gradient perturbation, and adversarial training. The authors analyze the strengths and weaknesses of these defenses and highlight the need for more comprehensive protection mechanisms to ensure the privacy of federated learning participants.
Critical Analysis
The paper provides a thorough and well-structured overview of gradient inversion techniques in the context of federated learning. However, it is important to note that the field of federated learning is rapidly evolving, and new attacks and defenses may emerge over time.
While the paper discusses several defense mechanisms, it acknowledges that they may not be sufficient to fully protect against gradient inversion attacks. Additionally, the authors mention that the effectiveness of these defenses can be influenced by factors such as the model architecture and the training data distribution.
Further research is needed to develop more robust and comprehensive privacy-preserving techniques for federated learning. Potential areas for future work include exploring federated learning approaches that are inherently more resilient to gradient leakage, as well as investigating the impact of gradient inversion on different types of machine learning models and applications.
Conclusion
This paper offers a comprehensive systematization of gradient inversion techniques in the context of federated learning. By understanding the different approaches to gradient inversion and the associated privacy risks, researchers and practitioners can work towards developing more secure and privacy-preserving federated learning systems. The insights provided in this paper can inform the design of future federated learning architectures and contribute to the ongoing efforts to address the privacy challenges in this emerging field.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
🏅
Dealing Doubt: Unveiling Threat Models in Gradient Inversion Attacks under Federated Learning, A Survey and Taxonomy
Yichuan Shi, Olivera Kotevska, Viktor Reshniak, Abhishek Singh, Ramesh Raskar
0
0
Federated Learning (FL) has emerged as a leading paradigm for decentralized, privacy preserving machine learning training. However, recent research on gradient inversion attacks (GIAs) have shown that gradient updates in FL can leak information on private training samples. While existing surveys on GIAs have focused on the honest-but-curious server threat model, there is a dearth of research categorizing attacks under the realistic and far more privacy-infringing cases of malicious servers and clients. In this paper, we present a survey and novel taxonomy of GIAs that emphasize FL threat models, particularly that of malicious servers and clients. We first formally define GIAs and contrast conventional attacks with the malicious attacker. We then summarize existing honest-but-curious attack strategies, corresponding defenses, and evaluation metrics. Critically, we dive into attacks with malicious servers and clients to highlight how they break existing FL defenses, focusing specifically on reconstruction methods, target model architectures, target data, and evaluation metrics. Lastly, we discuss open problems and future research directions.
5/20/2024
Breaking Secure Aggregation: Label Leakage from Aggregated Gradients in Federated Learning
Zhibo Wang, Zhiwei Chang, Jiahui Hu, Xiaoyi Pang, Jiacheng Du, Yongle Chen, Kui Ren
0
0
Federated Learning (FL) exhibits privacy vulnerabilities under gradient inversion attacks (GIAs), which can extract private information from individual gradients. To enhance privacy, FL incorporates Secure Aggregation (SA) to prevent the server from obtaining individual gradients, thus effectively resisting GIAs. In this paper, we propose a stealthy label inference attack to bypass SA and recover individual clients' private labels. Specifically, we conduct a theoretical analysis of label inference from the aggregated gradients that are exclusively obtained after implementing SA. The analysis results reveal that the inputs (embeddings) and outputs (logits) of the final fully connected layer (FCL) contribute to gradient disaggregation and label restoration. To preset the embeddings and logits of FCL, we craft a fishing model by solely modifying the parameters of a single batch normalization (BN) layer in the original model. Distributing client-specific fishing models, the server can derive the individual gradients regarding the bias of FCL by resolving a linear system with expected embeddings and the aggregated gradients as coefficients. Then the labels of each client can be precisely computed based on preset logits and gradients of FCL's bias. Extensive experiments show that our attack achieves large-scale label recovery with 100% accuracy on various datasets and model architectures.
6/26/2024
🛸
GI-SMN: Gradient Inversion Attack against Federated Learning without Prior Knowledge
Jin Qian, Kaimin Wei, Yongdong Wu, Jilian Zhang, Jipeng Chen, Huan Bao
0
0
Federated learning (FL) has emerged as a privacy-preserving machine learning approach where multiple parties share gradient information rather than original user data. Recent work has demonstrated that gradient inversion attacks can exploit the gradients of FL to recreate the original user data, posing significant privacy risks. However, these attacks make strong assumptions about the attacker, such as altering the model structure or parameters, gaining batch normalization statistics, or acquiring prior knowledge of the original training set, etc. Consequently, these attacks are not possible in real-world scenarios. To end it, we propose a novel Gradient Inversion attack based on Style Migration Network (GI-SMN), which breaks through the strong assumptions made by previous gradient inversion attacks. The optimization space is reduced by the refinement of the latent code and the use of regular terms to facilitate gradient matching. GI-SMN enables the reconstruction of user data with high similarity in batches. Experimental results have demonstrated that GI-SMN outperforms state-of-the-art gradient inversion attacks in both visual effect and similarity metrics. Additionally, it also can overcome gradient pruning and differential privacy defenses.
5/7/2024
⛏️
Federated Learning Privacy: Attacks, Defenses, Applications, and Policy Landscape - A Survey
Joshua C. Zhao, Saurabh Bagchi, Salman Avestimehr, Kevin S. Chan, Somali Chaterji, Dimitris Dimitriadis, Jiacheng Li, Ninghui Li, Arash Nourian, Holger R. Roth
0
0
Deep learning has shown incredible potential across a vast array of tasks and accompanying this growth has been an insatiable appetite for data. However, a large amount of data needed for enabling deep learning is stored on personal devices and recent concerns on privacy have further highlighted challenges for accessing such data. As a result, federated learning (FL) has emerged as an important privacy-preserving technology enabling collaborative training of machine learning models without the need to send the raw, potentially sensitive, data to a central server. However, the fundamental premise that sending model updates to a server is privacy-preserving only holds if the updates cannot be reverse engineered to infer information about the private training data. It has been shown under a wide variety of settings that this premise for privacy does {em not} hold. In this survey paper, we provide a comprehensive literature review of the different privacy attacks and defense methods in FL. We identify the current limitations of these attacks and highlight the settings in which FL client privacy can be broken. We dissect some of the successful industry applications of FL and draw lessons for future successful adoption. We survey the emerging landscape of privacy regulation for FL. We conclude with future directions for taking FL toward the cherished goal of generating accurate models while preserving the privacy of the data from its participants.
5/7/2024