Hiding in Plain Sight: Disguising Data Stealing Attacks in Federated Learning
2306.03013
0
0
š
Abstract
Malicious server (MS) attacks have enabled the scaling of data stealing in federated learning to large batch sizes and secure aggregation, settings previously considered private. However, many concerns regarding the client-side detectability of MS attacks were raised, questioning their practicality. In this work, for the first time, we thoroughly study client-side detectability. We first demonstrate that all prior MS attacks are detectable by principled checks, and formulate a necessary set of requirements that a practical MS attack must satisfy. Next, we propose SEER, a novel attack framework that satisfies these requirements. The key insight of SEER is the use of a secret decoder, jointly trained with the shared model. We show that SEER can steal user data from gradients of realistic networks, even for large batch sizes of up to 512 and under secure aggregation. Our work is a promising step towards assessing the true vulnerability of federated learning in real-world settings.
Create account to get full access
Overview
- Researchers demonstrate that existing "malicious server" attacks can steal user data from federated learning, even when using large batch sizes and secure aggregation, which were previously considered private.
- However, they also find that these attacks are detectable by clients using principled checks.
- The researchers propose a new attack framework called SEER that satisfies the necessary requirements for a practical, undetectable attack.
Plain English Explanation
The paper explores a type of attack on federated learning systems called "malicious server" attacks. In these attacks, the server running the federated learning system is compromised and tries to steal private user data from the updates (gradients) sent by the clients.
The researchers show that even in settings with large batch sizes and secure aggregation - which were thought to protect privacy - these malicious server attacks can still steal user data. However, they also find that the clients can detect these attacks using some basic checks.
To address this, the researchers propose a new attack framework called SEER. SEER uses a "secret decoder" that is trained jointly with the main model. This allows SEER to steal data from client gradients without being detected by the clients' checks. The researchers demonstrate that SEER can steal data effectively, even for large batch sizes and under secure aggregation.
Overall, this work highlights the vulnerabilities of federated learning systems to malicious server attacks, even in settings considered private. The SEER attack framework represents a step towards understanding the true security risks in real-world federated learning applications.
Technical Explanation
The paper first demonstrates that existing "malicious server" (MS) attacks, which aim to steal user data from federated learning gradients, can be effective even in settings with large batch sizes and secure aggregation, which were previously thought to provide privacy. However, the researchers find that these MS attacks can be detected by clients using principled checks.
To address this, the researchers propose a new attack framework called SEER. The key insight of SEER is the use of a "secret decoder" that is jointly trained with the shared model. This secret decoder allows SEER to extract user data from the gradients without being detected by the clients' checks.
The researchers show that SEER can successfully steal user data from the gradients of realistic neural network models, even for large batch sizes of up to 512 and under secure aggregation. This demonstrates that SEER satisfies the necessary requirements for a practical, undetectable MS attack.
Critical Analysis
The paper provides a thorough analysis of the client-side detectability of malicious server attacks on federated learning systems. While the researchers demonstrate that existing attacks can be detected, their proposed SEER framework appears to address this limitation. However, some caveats and areas for future research are worth noting.
The paper focuses on gradient-based attacks, but there may be other attack vectors, such as those targeting the feature space, that were not considered. Additionally, the paper only evaluates SEER on a limited set of neural network architectures, and its effectiveness may vary with different models and tasks.
Further research is needed to explore the broader applicability of SEER and to assess its robustness against potential countermeasures. The precision-guided approach to mitigating data poisoning attacks, for example, may also be relevant for defending against malicious server attacks like SEER.
Overall, this work represents an important step in understanding the gradient leakage vulnerabilities of federated learning systems and highlights the need for continued research to ensure the security and privacy of these distributed learning systems.
Conclusion
This paper demonstrates that existing "malicious server" attacks can steal user data from federated learning gradients, even in settings with large batch sizes and secure aggregation that were previously considered private. However, the researchers also find that these attacks can be detected by clients using principled checks.
To address this, the researchers propose a novel attack framework called SEER that satisfies the necessary requirements for a practical, undetectable malicious server attack. SEER's use of a secret decoder allows it to effectively steal data from client gradients without being caught by the clients' detection mechanisms.
This work highlights the ongoing challenges in ensuring the security and privacy of federated learning systems, even as they are adopted for a growing number of real-world applications. The SEER attack framework represents a concerning development, but also an important step towards a more comprehensive understanding of the true vulnerabilities of federated learning in practice.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
Robust Federated Learning Mitigates Client-side Training Data Distribution Inference Attacks
Yichang Xu, Ming Yin, Minghong Fang, Neil Zhenqiang Gong
0
0
Recent studies have revealed that federated learning (FL), once considered secure due to clients not sharing their private data with the server, is vulnerable to attacks such as client-side training data distribution inference, where a malicious client can recreate the victim's data. While various countermeasures exist, they are not practical, often assuming server access to some training data or knowledge of label distribution before the attack. In this work, we bridge the gap by proposing InferGuard, a novel Byzantine-robust aggregation rule aimed at defending against client-side training data distribution inference attacks. In our proposed InferGuard, the server first calculates the coordinate-wise median of all the model updates it receives. A client's model update is considered malicious if it significantly deviates from the computed median update. We conduct a thorough evaluation of our proposed InferGuard on five benchmark datasets and perform a comparison with ten baseline methods. The results of our experiments indicate that our defense mechanism is highly effective in protecting against client-side training data distribution inference attacks, even against strong adaptive attacks. Furthermore, our method substantially outperforms the baseline methods in various practical FL scenarios.
4/5/2024
Breaking Secure Aggregation: Label Leakage from Aggregated Gradients in Federated Learning
Zhibo Wang, Zhiwei Chang, Jiahui Hu, Xiaoyi Pang, Jiacheng Du, Yongle Chen, Kui Ren
0
0
Federated Learning (FL) exhibits privacy vulnerabilities under gradient inversion attacks (GIAs), which can extract private information from individual gradients. To enhance privacy, FL incorporates Secure Aggregation (SA) to prevent the server from obtaining individual gradients, thus effectively resisting GIAs. In this paper, we propose a stealthy label inference attack to bypass SA and recover individual clients' private labels. Specifically, we conduct a theoretical analysis of label inference from the aggregated gradients that are exclusively obtained after implementing SA. The analysis results reveal that the inputs (embeddings) and outputs (logits) of the final fully connected layer (FCL) contribute to gradient disaggregation and label restoration. To preset the embeddings and logits of FCL, we craft a fishing model by solely modifying the parameters of a single batch normalization (BN) layer in the original model. Distributing client-specific fishing models, the server can derive the individual gradients regarding the bias of FCL by resolving a linear system with expected embeddings and the aggregated gradients as coefficients. Then the labels of each client can be precisely computed based on preset logits and gradients of FCL's bias. Extensive experiments show that our attack achieves large-scale label recovery with 100% accuracy on various datasets and model architectures.
6/26/2024
š
Make Split, not Hijack: Preventing Feature-Space Hijacking Attacks in Split Learning
Tanveer Khan, Mindaugas Budzys, Antonis Michalas
0
0
The popularity of Machine Learning (ML) makes the privacy of sensitive data more imperative than ever. Collaborative learning techniques like Split Learning (SL) aim to protect client data while enhancing ML processes. Though promising, SL has been proved to be vulnerable to a plethora of attacks, thus raising concerns about its effectiveness on data privacy. In this work, we introduce a hybrid approach combining SL and Function Secret Sharing (FSS) to ensure client data privacy. The client adds a random mask to the activation map before sending it to the servers. The servers cannot access the original function but instead work with shares generated using FSS. Consequently, during both forward and backward propagation, the servers cannot reconstruct the client's raw data from the activation map. Furthermore, through visual invertibility, we demonstrate that the server is incapable of reconstructing the raw image data from the activation map when using FSS. It enhances privacy by reducing privacy leakage compared to other SL-based approaches where the server can access client input information. Our approach also ensures security against feature space hijacking attack, protecting sensitive information from potential manipulation. Our protocols yield promising results, reducing communication overhead by over 2x and training time by over 7x compared to the same model with FSS, without any SL. Also, we show that our approach achieves >96% accuracy and remains equivalent to the plaintext models.
4/16/2024
Lurking in the shadows: Unveiling Stealthy Backdoor Attacks against Personalized Federated Learning
Xiaoting Lyu, Yufei Han, Wei Wang, Jingkai Liu, Yongsheng Zhu, Guangquan Xu, Jiqiang Liu, Xiangliang Zhang
0
0
Federated Learning (FL) is a collaborative machine learning technique where multiple clients work together with a central server to train a global model without sharing their private data. However, the distribution shift across non-IID datasets of clients poses a challenge to this one-model-fits-all method hindering the ability of the global model to effectively adapt to each client's unique local data. To echo this challenge, personalized FL (PFL) is designed to allow each client to create personalized local models tailored to their private data. While extensive research has scrutinized backdoor risks in FL, it has remained underexplored in PFL applications. In this study, we delve deep into the vulnerabilities of PFL to backdoor attacks. Our analysis showcases a tale of two cities. On the one hand, the personalization process in PFL can dilute the backdoor poisoning effects injected into the personalized local models. Furthermore, PFL systems can also deploy both server-end and client-end defense mechanisms to strengthen the barrier against backdoor attacks. On the other hand, our study shows that PFL fortified with these defense methods may offer a false sense of security. We propose textit{PFedBA}, a stealthy and effective backdoor attack strategy applicable to PFL systems. textit{PFedBA} ingeniously aligns the backdoor learning task with the main learning task of PFL by optimizing the trigger generation process. Our comprehensive experiments demonstrate the effectiveness of textit{PFedBA} in seamlessly embedding triggers into personalized local models. textit{PFedBA} yields outstanding attack performance across 10 state-of-the-art PFL algorithms, defeating the existing 6 defense mechanisms. Our study sheds light on the subtle yet potent backdoor threats to PFL systems, urging the community to bolster defenses against emerging backdoor challenges.
6/11/2024