Think Twice Before Selection: Federated Evidential Active Learning for Medical Image Analysis with Domain Shifts

Overview

• Federated learning allows multiple medical institutions to collaborate on training a global model without centralizing their data.
• However, the high cost of annotating local data remains a challenge.
• Federated active learning methods try to address this by selecting a small amount of informative local data for annotation.
• Existing methods focus on local data from the same domain, which can be unreliable when there are domain shifts across different clients.
• This paper proposes a new method called Federated Evidential Active Learning (FEAL) to address this issue.

Plain English Explanation

Federated learning is a way for different medical institutions to work together to train a single machine learning model, without having to share their private patient data. This is useful because hospitals and clinics often can't share data due to privacy concerns.

However, one problem with federated learning is that it can be expensive for each institution to carefully label or "annotate" their local data before sharing it. Federated active learning tries to solve this by having the global model identify the most important local data that needs to be annotated, so institutions only have to label a small amount.

But the existing federated active learning methods have a limitation - they assume all the local data comes from the same type of medical cases or "domain". In the real world, different hospitals and clinics may have data from quite different types of patients. This "domain shift" can cause problems for the active learning approach.

To address this, the researchers propose a new method called "Federated Evidential Active Learning" (FEAL). FEAL uses a special way of modeling the uncertainty in the predictions of both the local and global models. This allows it to better identify informative local data, even when there are differences between the data at different institutions. FEAL also has a strategy to reduce redundancy and maintain diversity in the selected local data.

Technical Explanation

The key technical innovation of the FEAL method is the use of a Dirichlet prior distribution in both the local and global models. This allows the models to output a distribution over the probability simplex, rather than just a single prediction. This distribution can capture both the "aleatoric" uncertainty (inherent randomness in the data) and the "epistemic" uncertainty (uncertainty due to limited knowledge).

FEAL then uses the epistemic uncertainty to calibrate the aleatoric uncertainty, in order to better assess the informativeness of local data samples, even when there are domain shifts across clients. Additionally, FEAL employs a diversity relaxation strategy to reduce data redundancy and maintain diversity in the selected local samples.

Extensive experiments on five real-world medical image datasets demonstrate that FEAL outperforms state-of-the-art federated active learning methods, especially in scenarios with domain shifts across clients. The code for FEAL will be available on GitHub.

Critical Analysis

The paper makes a valuable contribution by addressing the challenge of domain shifts in federated active learning, which is an important practical concern for real-world deployment of these methods. The use of the Dirichlet distribution to model uncertainty is a principled approach, and the diversity relaxation strategy is a sensible way to ensure the selected local data is informative.

However, the paper does not provide a deep analysis of the limitations of FEAL. For example, it would be helpful to understand how FEAL performs when the domain shifts are more severe, or when there are multiple distinct domains represented across the clients. Additionally, the paper does not discuss potential privacy or fairness implications of the proposed approach.

It would also be worthwhile for future research to explore ways of further reducing the annotation burden on local clients, perhaps by investigating semi-supervised or self-supervised learning techniques in the federated setting. Uncertainty-Aware Evidential Fusion-Based Learning for Semi-Supervised Medical Image Classification is an example of related work in this direction.

Overall, the FEAL method represents a solid contribution to the field of cross-silo federated learning across divergent domains, and the code release will likely be valuable for further research and development in this area.

Conclusion

This paper proposes a novel Federated Evidential Active Learning (FEAL) method to address the challenge of expensive data annotation in federated learning scenarios with domain shifts across medical institutions. By modeling epistemic and aleatoric uncertainties using a Dirichlet distribution and employing a diversity relaxation strategy, FEAL can effectively identify informative local data samples for annotation, even when the local data comes from diverse domains.

The experimental results demonstrate the superiority of FEAL over state-of-the-art federated active learning approaches, particularly in realistic medical settings with domain shifts. This work represents an important step forward in distributed federated learning-based deep learning model development, with the potential to enable more effective collaborative learning among medical institutions without compromising patient data privacy.