FPL+: Filtered Pseudo Label-based Unsupervised Cross-Modality Adaptation for 3D Medical Image Segmentation
2404.04971
0
0
Abstract
Adapting a medical image segmentation model to a new domain is important for improving its cross-domain transferability, and due to the expensive annotation process, Unsupervised Domain Adaptation (UDA) is appealing where only unlabeled images are needed for the adaptation. Existing UDA methods are mainly based on image or feature alignment with adversarial training for regularization, and they are limited by insufficient supervision in the target domain. In this paper, we propose an enhanced Filtered Pseudo Label (FPL+)-based UDA method for 3D medical image segmentation. It first uses cross-domain data augmentation to translate labeled images in the source domain to a dual-domain training set consisting of a pseudo source-domain set and a pseudo target-domain set. To leverage the dual-domain augmented images to train a pseudo label generator, domain-specific batch normalization layers are used to deal with the domain shift while learning the domain-invariant structure features, generating high-quality pseudo labels for target-domain images. We then combine labeled source-domain images and target-domain images with pseudo labels to train a final segmentor, where image-level weighting based on uncertainty estimation and pixel-level weighting based on dual-domain consensus are proposed to mitigate the adverse effect of noisy pseudo labels. Experiments on three public multi-modal datasets for Vestibular Schwannoma, brain tumor and whole heart segmentation show that our method surpassed ten state-of-the-art UDA methods, and it even achieved better results than fully supervised learning in the target domain in some cases.
Create account to get full access
Overview
- This paper proposes a novel unsupervised cross-modality adaptation method called FPL+ (Filtered Pseudo Label-based Unsupervised Cross-Modality Adaptation) for 3D medical image segmentation.
- The method leverages pseudo-labels generated from a source-trained model to adapt the model to a target domain without any annotated target data.
- The key idea is to filter the pseudo-labels using an uncertainty-aware module to ensure only high-confidence pseudo-labels are used for adaptation.
Plain English Explanation
In the field of medical imaging, there is often a lack of annotated data, especially for rarer diseases or new imaging modalities. To address this challenge, FPL+: Filtered Pseudo Label-based Unsupervised Cross-Modality Adaptation for 3D Medical Image Segmentation proposes a technique called FPL+ that can adapt a segmentation model trained on one type of medical images (the source domain) to work well on a different type of medical images (the target domain) without any labeled data from the target domain.
The key insight is to use the predictions made by the source-trained model on the target domain images as "pseudo-labels" to fine-tune the model. However, these pseudo-labels may not always be accurate, so the FPL+ method includes a filtering step to only use the pseudo-labels that the model is highly confident about. This helps ensure the model is only updated using high-quality pseudo-labels, leading to better adaptation to the target domain.
The FPL+ method has the potential to significantly improve medical image analysis by enabling the reuse of models trained on one type of data for other related tasks, without the need for costly manual annotation of the target data.
Technical Explanation
The FPL+: Filtered Pseudo Label-based Unsupervised Cross-Modality Adaptation for 3D Medical Image Segmentation paper presents a novel unsupervised cross-modality adaptation method for 3D medical image segmentation.
The key steps of the FPL+ method are:
- Train a segmentation model on the source domain: The authors first train a segmentation model using annotated data from the source domain (e.g., CT scans).
- Generate pseudo-labels on the target domain: The source-trained model is then used to generate segmentation predictions (pseudo-labels) on the unlabeled target domain data (e.g., MRI scans).
- Filter the pseudo-labels: An uncertainty-aware module is used to assess the confidence of the pseudo-labels and only the high-confidence pseudo-labels are retained for the next step.
- Adapt the model to the target domain: The model is then fine-tuned using the filtered pseudo-labels from the target domain, effectively adapting it to the target modality.
The authors evaluate the FPL+ method on several 3D medical image segmentation tasks, including brain, prostate, and pancreas segmentation. The results demonstrate that FPL+ can significantly improve segmentation performance on the target domain compared to the source-trained model, without requiring any annotated target data.
Critical Analysis
The FPL+: Filtered Pseudo Label-based Unsupervised Cross-Modality Adaptation for 3D Medical Image Segmentation paper presents a well-designed and thorough study on unsupervised cross-modality adaptation for 3D medical image segmentation. The key strength of the FPL+ method is its ability to effectively filter the pseudo-labels, ensuring that only high-confidence predictions are used for adaptation.
However, one potential limitation is that the method still relies on the availability of a well-performing source-trained model. If the source model has poor performance, the quality of the pseudo-labels may be low, which could limit the effectiveness of the adaptation. Additionally, the paper does not explore the sensitivity of the method to the choice of the uncertainty-aware module used for pseudo-label filtering.
Another area for further research could be to investigate the applicability of FPL+ to a wider range of medical imaging tasks and modalities, as the evaluation in the paper is limited to a few specific use cases. How Useful Is Continued Pre-Training, Generative models could also be explored as an alternative approach for unsupervised cross-modality adaptation.
Conclusion
The FPL+: Filtered Pseudo Label-based Unsupervised Cross-Modality Adaptation for 3D Medical Image Segmentation paper presents a promising technique for addressing the challenge of limited annotated data in medical imaging. The FPL+ method leverages pseudo-labels and an uncertainty-aware filtering mechanism to effectively adapt a segmentation model from one modality to another, without requiring any labeled target domain data.
This work has the potential to significantly improve the applicability and accessibility of medical image analysis tools, as it allows models trained on one type of data to be reused for related tasks, reducing the need for costly data annotation. As the authors note, further research is needed to explore the versatility of the FPL+ method and its sensitivity to various design choices, but the overall approach represents an important step forward in the field of CODA: Instructive Chain Domain Adaptation for Severity-Aware medical image segmentation.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
Style Adaptation for Domain-adaptive Semantic Segmentation
Ting Li, Jianshu Chao, Deyu An
0
0
Unsupervised Domain Adaptation (UDA) refers to the method that utilizes annotated source domain data and unlabeled target domain data to train a model capable of generalizing to the target domain data. Domain discrepancy leads to a significant decrease in the performance of general network models trained on the source domain data when applied to the target domain. We introduce a straightforward approach to mitigate the domain discrepancy, which necessitates no additional parameter calculations and seamlessly integrates with self-training-based UDA methods. Through the transfer of the target domain style to the source domain in the latent feature space, the model is trained to prioritize the target domain style during the decision-making process. We tackle the problem at both the image-level and shallow feature map level by transferring the style information from the target domain to the source domain data. As a result, we obtain a model that exhibits superior performance on the target domain. Our method yields remarkable enhancements in the state-of-the-art performance for synthetic-to-real UDA tasks. For example, our proposed method attains a noteworthy UDA performance of 76.93 mIoU on the GTA->Cityscapes dataset, representing a notable improvement of +1.03 percentage points over the previous state-of-the-art results.
4/26/2024
Achieving Reliable and Fair Skin Lesion Diagnosis via Unsupervised Domain Adaptation
Janet Wang, Yunbei Zhang, Zhengming Ding, Jihun Hamm
0
0
The development of reliable and fair diagnostic systems is often constrained by the scarcity of labeled data. To address this challenge, our work explores the feasibility of unsupervised domain adaptation (UDA) to integrate large external datasets for developing reliable classifiers. The adoption of UDA with multiple sources can simultaneously enrich the training set and bridge the domain gap between different skin lesion datasets, which vary due to distinct acquisition protocols. Particularly, UDA shows practical promise for improving diagnostic reliability when training with a custom skin lesion dataset, where only limited labeled data are available from the target domain. In this study, we investigate three UDA training schemes based on source data utilization: single-source, combined-source, and multi-source UDA. Our findings demonstrate the effectiveness of applying UDA on multiple sources for binary and multi-class classification. A strong correlation between test error and label shift in multi-class tasks has been observed in the experiment. Crucially, our study shows that UDA can effectively mitigate bias against minority groups and enhance fairness in diagnostic systems, while maintaining superior classification performance. This is achieved even without directly implementing fairness-focused techniques. This success is potentially attributed to the increased and well-adapted demographic information obtained from multiple sources.
4/17/2024
New!STAL3D: Unsupervised Domain Adaptation for 3D Object Detection via Collaborating Self-Training and Adversarial Learning
Yanan Zhang, Chao Zhou, Di Huang
0
0
Existing 3D object detection suffers from expensive annotation costs and poor transferability to unknown data due to the domain gap, Unsupervised Domain Adaptation (UDA) aims to generalize detection models trained in labeled source domains to perform robustly on unexplored target domains, providing a promising solution for cross-domain 3D object detection. Although Self-Training (ST) based cross-domain 3D detection methods with the assistance of pseudo-labeling techniques have achieved remarkable progress, they still face the issue of low-quality pseudo-labels when there are significant domain disparities due to the absence of a process for feature distribution alignment. While Adversarial Learning (AL) based methods can effectively align the feature distributions of the source and target domains, the inability to obtain labels in the target domain forces the adoption of asymmetric optimization losses, resulting in a challenging issue of source domain bias. To overcome these limitations, we propose a novel unsupervised domain adaptation framework for 3D object detection via collaborating ST and AL, dubbed as STAL3D, unleashing the complementary advantages of pseudo labels and feature distribution alignment. Additionally, a Background Suppression Adversarial Learning (BS-AL) module and a Scale Filtering Module (SFM) are designed tailored for 3D cross-domain scenes, effectively alleviating the issues of the large proportion of background interference and source domain size bias. Our STAL3D achieves state-of-the-art performance on multiple cross-domain tasks and even surpasses the Oracle results on Waymo $rightarrow$ KITTI and Waymo $rightarrow$ KITTI-rain.
6/28/2024
Rethinking Barely-Supervised Segmentation from an Unsupervised Domain Adaptation Perspective
Zhiqiang Shen, Peng Cao, Junming Su, Jinzhu Yang, Osmar R. Zaiane
0
0
This paper investigates an extremely challenging problem, barely-supervised medical image segmentation (BSS), where the training dataset comprises limited labeled data with only single-slice annotations and numerous unlabeled images. Currently, state-of-the-art (SOTA) BSS methods utilize a registration-based paradigm, depending on image registration to propagate single-slice annotations into volumetric pseudo labels for constructing a complete labeled set. However, this paradigm has a critical limitation: the pseudo labels generated by image registration are unreliable and noisy. Motivated by this, we propose a new perspective: training a model using only single-annotated slices as the labeled set without relying on image registration. To this end, we formulate BSS as an unsupervised domain adaptation (UDA) problem. Specifically, we first design a novel noise-free labeled data construction algorithm (NFC) for slice-to-volume labeled data synthesis, which may result in a side effect: domain shifts between the synthesized images and the original images. Then, a frequency and spatial mix-up strategy (FSX) is further introduced to mitigate the domain shifts for UDA. Extensive experiments demonstrate that our method provides a promising alternative for BSS. Remarkably, the proposed method with only one labeled slice achieves an 80.77% dice score on left atrial segmentation, outperforming the SOTA by 61.28%. The code will be released upon the publication of this paper.
5/17/2024