Attacks on fairness in Federated Learning

Overview

• Federated Learning is a distributed training paradigm that keeps data private on clients.
• Backdoor attacks on Federated Learning models are well-understood, where a small subset of clients can introduce a backdoor.
• This paper presents a new type of attack that compromises the fairness of the trained model, rather than introducing a backdoor.
• Fairness refers to the attribute-level performance distribution of the trained model, which is important in domains where unfair discrimination could have serious consequences.

Plain English Explanation

Federated Learning is a way of training AI models without collecting all the data in one place. Instead, the training happens on individual devices, like phones or computers, and the model is updated by combining the changes from many devices. This keeps the data private on the individual devices.

Previous research has shown that backdoor attacks are possible in Federated Learning. This means that even if only a small number of devices are compromised, the attacker can introduce a backdoor into the final trained model.

This new paper looks at a different type of attack, one that compromises the fairness of the trained model, rather than introducing a backdoor. Fairness means that the model performs equally well across different groups or attributes, like age, gender, or race. Unfair models can have serious consequences in domains like healthcare or finance.

The paper shows that using a similar threat model to a backdoor attack, an attacker can influence the final model to have an unfair performance distribution between different attributes. Importantly, the attacker only needs to control a single client device to pull this off.

While dealing with natural unfairness in Federated Learning has been discussed before, this artificial unfairness introduced by an attacker has been overlooked. The paper argues that defending against attacks on fairness should be a critical consideration whenever unfairness in a model could benefit someone who participated in the training.

Technical Explanation

The paper presents a new type of attack on Federated Learning models that compromises the fairness of the trained model, rather than introducing a backdoor.

The threat model is similar to that of a backdoor attack - the attacker controls only a small subset of the participating clients. However, instead of injecting a backdoor, the attacker aims to influence the aggregated model to have an unfair performance distribution across different attributes.

The authors demonstrate that this "fairness attack" is possible by controlling just a single client. They show how the attacker can strategically update the local model on the compromised client to skew the final aggregated model towards unfair performance.

While prior work has addressed naturally-occurring unfairness in Federated Learning, this paper highlights that artificially-induced unfairness through adversarial attacks has been overlooked. The authors argue that defending against fairness attacks should be a critical consideration in any scenario where unfairness in the trained model could benefit a participant in the training process.

Critical Analysis

The paper makes an important contribution by identifying a new type of attack on Federated Learning models that targets fairness, rather than introducing a backdoor. This is a significant concern, as unfair models can have severe real-world consequences in domains like healthcare, finance, and criminal justice.

However, the paper does not address some key limitations and open questions:

• The proposed attack relies on the attacker having perfect knowledge of the target attributes and being able to strategically update the local model. In practice, this level of information and control may be difficult to achieve.
• The paper does not explore potential defenses against fairness attacks. Developing effective countermeasures should be a priority for future research in this area.
• The scope is limited to Federated Learning, but unfairness in AI models is a broader issue that affects centralized training as well. Expanding the analysis to other distributed training paradigms could yield additional insights.

Despite these limitations, the paper successfully highlights the importance of fairness as a security and privacy concern in Federated Learning. Further research is needed to fully understand the risks and develop robust defenses against this new class of attacks.

Conclusion

This paper presents a novel attack on Federated Learning models that compromises the fairness of the trained model, rather than introducing a backdoor. The authors demonstrate that an attacker can strategically update a single compromised client to skew the final aggregated model towards unfair performance across different attributes.

While prior work has addressed naturally-occurring unfairness in Federated Learning, this paper argues that artificially-induced unfairness through adversarial attacks has been overlooked. Defending against fairness attacks should be a critical consideration whenever unfairness in the trained model could benefit a participant in the training process.

The paper makes an important contribution by identifying this new threat, but further research is needed to fully understand the risks and develop effective countermeasures. Expanding the analysis beyond Federated Learning to other distributed training paradigms could also yield valuable insights.