DG-TTA: Out-of-domain medical image segmentation through Domain Generalization and Test-Time Adaptation
2312.06275
0
0
Abstract
Applying pre-trained medical segmentation models on out-of-domain images often yields predictions of insufficient quality. Several strategies have been proposed to maintain model performance, such as finetuning or unsupervised- and source-free domain adaptation. These strategies set restrictive requirements for data availability. In this study, we propose to combine domain generalization and test-time adaptation to create a highly effective approach for reusing pre-trained models in unseen target domains. Domain-generalized pre-training on source data is used to obtain the best initial performance in the target domain. We introduce the MIND descriptor previously used in image registration tasks as a further technique to achieve generalization and present superior performance for small-scale datasets compared to existing approaches. At test-time, high-quality segmentation for every single unseen scan is ensured by optimizing the model weights for consistency given different image augmentations. That way, our method enables separate use of source and target data and thus removes current data availability barriers. Moreover, the presented method is highly modular as it does not require specific model architectures or prior knowledge of involved domains and labels. We demonstrate this by integrating it into the nnUNet, which is currently the most popular and accurate framework for medical image segmentation. We employ multiple datasets covering abdominal, cardiac, and lumbar spine scans and compose several out-of-domain scenarios in this study. We demonstrate that our method, combined with pre-trained whole-body CT models, can effectively segment MR images with high accuracy in all of the aforementioned scenarios. Open-source code can be found here: https://github.com/multimodallearning/DG-TTA
Create account to get full access
Overview
ā¢ This paper proposes a new approach called DG-TTA (Domain Generalization and Test-Time Adaptation) to improve out-of-domain medical image segmentation.
ā¢ The key ideas are to use domain generalization techniques to train a model that can perform well on a wide range of medical imaging domains, and then further adapt the model at test time to the specific target domain.
ā¢ The paper evaluates DG-TTA on several medical image segmentation tasks, demonstrating improved performance compared to previous methods.
Plain English Explanation
Medical image segmentation is the process of automatically identifying and outlining different structures or regions within medical images, such as organs, tumors, or blood vessels. This is an important task for various clinical applications, like disease diagnosis and treatment planning.
However, a major challenge is that medical imaging data can come from many different sources and devices, leading to significant differences in image characteristics. This makes it difficult for machine learning models trained on one type of medical data to perform well on data from other domains.
The DG-TTA approach proposed in this paper aims to address this challenge. The key idea is to train a model that can generalize well to diverse medical imaging domains, rather than optimizing it for a single data source. This "domain generalization" step teaches the model to recognize underlying patterns that are consistent across different medical imaging modalities.
Then, at test time, the model is further adapted to the specific target domain using a technique called "test-time adaptation." This fine-tunes the model's performance on the particular data it will be applied to, without requiring any additional labeled training examples from that domain.
By combining these two strategies of domain generalization and test-time adaptation, the DG-TTA method is able to achieve improved medical image segmentation performance on data that is outside the original training distribution. This makes the model more robust and practical for real-world clinical applications, where the input data can vary widely.
Technical Explanation
The DG-TTA approach consists of two main components:
-
Domain Generalization: The model is trained on a diverse set of medical imaging domains, using techniques like domain mixup and domain-specific batch normalization to learn representations that are robust to domain shift. This helps the model generalize to unseen target domains.
-
Test-Time Adaptation: At inference time, the trained model is further adapted to the specific target domain using a small amount of unlabeled target data. This is done by aligning the model's feature distributions to match the target domain, without requiring any additional labeled data.
The authors evaluate DG-TTA on several medical image segmentation tasks, including segmentation of brain MRI, cardiac MRI, and abdominal CT scans. They compare the performance of DG-TTA to previous state-of-the-art methods for out-of-domain medical image segmentation, such as Language-Guided Domain Generalized Medical Image Segmentation, Adaptive Affinity-Based Generalization for MRI Imaging Segmentation, and FPL: Filtered Pseudo-Label-based Unsupervised Cross-Domain Medical Image Segmentation.
The results show that DG-TTA outperforms these previous methods, demonstrating the effectiveness of combining domain generalization and test-time adaptation for improving out-of-domain medical image segmentation performance.
Critical Analysis
The paper provides a comprehensive evaluation of the DG-TTA approach, considering various medical imaging domains and comparing to multiple state-of-the-art baselines. However, the authors acknowledge some limitations:
- The test-time adaptation step requires a small amount of unlabeled target domain data, which may not always be available in real-world scenarios.
- The performance improvements, while substantial, may not be sufficient for some high-stakes clinical applications where very high segmentation accuracy is required.
- The paper does not explore the computational efficiency of the DG-TTA approach, which could be an important factor for practical deployment.
Additionally, one could question whether the proposed domain generalization techniques, such as domain mixup, are truly learning representations that capture the underlying commonalities across medical imaging modalities, or if they are simply exploiting superficial correlations in the data.
Further research could investigate more principled ways of learning domain-invariant features, as well as exploring the transferability of the DG-TTA approach to other medical imaging tasks beyond segmentation.
Conclusion
This paper presents a novel DG-TTA method that combines domain generalization and test-time adaptation to improve out-of-domain medical image segmentation. By training models to be robust to domain shift and then further adapting them to specific target domains, the approach demonstrates significant performance gains over previous state-of-the-art methods.
The research highlights the importance of developing machine learning models that can generalize well to the diverse data encountered in real-world clinical settings, rather than being overly specialized to a single data source. The DG-TTA framework provides a promising direction for making medical image analysis systems more practical and impactful for a wide range of healthcare applications.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
Unsupervised Domain Adaptation for Pediatric Brain Tumor Segmentation
Jingru Fu, Simone Bendazzoli, Orjan Smedby, Rodrigo Moreno
0
0
Significant advances have been made toward building accurate automatic segmentation models for adult gliomas. However, the performance of these models often degrades when applied to pediatric glioma due to their imaging and clinical differences (domain shift). Obtaining sufficient annotated data for pediatric glioma is typically difficult because of its rare nature. Also, manual annotations are scarce and expensive. In this work, we propose Domain-Adapted nnU-Net (DA-nnUNet) to perform unsupervised domain adaptation from adult glioma (source domain) to pediatric glioma (target domain). Specifically, we add a domain classifier connected with a gradient reversal layer (GRL) to a backbone nnU-Net. Once the classifier reaches a very high accuracy, the GRL is activated with the goal of transferring domain-invariant features from the classifier to the segmentation model while preserving segmentation accuracy on the source domain. The accuracy of the classifier slowly degrades to chance levels. No annotations are used in the target domain. The method is compared to 8 different supervised models using BraTS-Adult glioma (N=1251) and BraTS-PED glioma data (N=99). The proposed method shows notable performance enhancements in the tumor core (TC) region compared to the model that only uses adult data: ~32% better Dice scores and ~20 better 95th percentile Hausdorff distances. Moreover, our unsupervised approach shows no statistically significant difference compared to the practical upper bound model using manual annotations from both datasets in TC region. The code is shared at https://github.com/Fjr9516/DA_nnUNet.
6/26/2024
New!An Uncertainty-guided Tiered Self-training Framework for Active Source-free Domain Adaptation in Prostate Segmentation
Zihao Luo, Xiangde Luo, Zijun Gao, Guotai Wang
0
0
Deep learning models have exhibited remarkable efficacy in accurately delineating the prostate for diagnosis and treatment of prostate diseases, but challenges persist in achieving robust generalization across different medical centers. Source-free Domain Adaptation (SFDA) is a promising technique to adapt deep segmentation models to address privacy and security concerns while reducing domain shifts between source and target domains. However, recent literature indicates that the performance of SFDA remains far from satisfactory due to unpredictable domain gaps. Annotating a few target domain samples is acceptable, as it can lead to significant performance improvement with a low annotation cost. Nevertheless, due to extremely limited annotation budgets, careful consideration is needed in selecting samples for annotation. Inspired by this, our goal is to develop Active Source-free Domain Adaptation (ASFDA) for medical image segmentation. Specifically, we propose a novel Uncertainty-guided Tiered Self-training (UGTST) framework, consisting of efficient active sample selection via entropy-based primary local peak filtering to aggregate global uncertainty and diversity-aware redundancy filter, coupled with a tiered self-learning strategy, achieves stable domain adaptation. Experimental results on cross-center prostate MRI segmentation datasets revealed that our method yielded marked advancements, with a mere 5% annotation, exhibiting an average Dice score enhancement of 9.78% and 7.58% in two target domains compared with state-of-the-art methods, on par with fully supervised learning. Code is available at:https://github.com/HiLab-git/UGTST
7/4/2024
š¼ļø
Towards Clinician-Preferred Segmentation: Leveraging Human-in-the-Loop for Test Time Adaptation in Medical Image Segmentation
Shishuai Hu, Zehui Liao, Zeyou Liu, Yong Xia
0
0
Deep learning-based medical image segmentation models often face performance degradation when deployed across various medical centers, largely due to the discrepancies in data distribution. Test Time Adaptation (TTA) methods, which adapt pre-trained models to test data, have been employed to mitigate such discrepancies. However, existing TTA methods primarily focus on manipulating Batch Normalization (BN) layers or employing prompt and adversarial learning, which may not effectively rectify the inconsistencies arising from divergent data distributions. In this paper, we propose a novel Human-in-the-loop TTA (HiTTA) framework that stands out in two significant ways. First, it capitalizes on the largely overlooked potential of clinician-corrected predictions, integrating these corrections into the TTA process to steer the model towards predictions that coincide more closely with clinical annotation preferences. Second, our framework conceives a divergence loss, designed specifically to diminish the prediction divergence instigated by domain disparities, through the careful calibration of BN parameters. Our HiTTA is distinguished by its dual-faceted capability to acclimatize to the distribution of test data whilst ensuring the model's predictions align with clinical expectations, thereby enhancing its relevance in a medical context. Extensive experiments on a public dataset underscore the superiority of our HiTTA over existing TTA methods, emphasizing the advantages of integrating human feedback and our divergence loss in enhancing the model's performance and adaptability across diverse medical centers.
5/15/2024
Language Guided Domain Generalized Medical Image Segmentation
Shahina Kunhimon, Muzammal Naseer, Salman Khan, Fahad Shahbaz Khan
0
0
Single source domain generalization (SDG) holds promise for more reliable and consistent image segmentation across real-world clinical settings particularly in the medical domain, where data privacy and acquisition cost constraints often limit the availability of diverse datasets. Depending solely on visual features hampers the model's capacity to adapt effectively to various domains, primarily because of the presence of spurious correlations and domain-specific characteristics embedded within the image features. Incorporating text features alongside visual features is a potential solution to enhance the model's understanding of the data, as it goes beyond pixel-level information to provide valuable context. Textual cues describing the anatomical structures, their appearances, and variations across various imaging modalities can guide the model in domain adaptation, ultimately contributing to more robust and consistent segmentation. In this paper, we propose an approach that explicitly leverages textual information by incorporating a contrastive learning mechanism guided by the text encoder features to learn a more robust feature representation. We assess the effectiveness of our text-guided contrastive feature alignment technique in various scenarios, including cross-modality, cross-sequence, and cross-site settings for different segmentation tasks. Our approach achieves favorable performance against existing methods in literature. Our code and model weights are available at https://github.com/ShahinaKK/LG_SDG.git.
4/4/2024