Cross-Modality Translation with Generative Adversarial Networks to Unveil Alzheimer's Disease Biomarkers
2405.05462
0
0
Abstract
Generative approaches for cross-modality transformation have recently gained significant attention in neuroimaging. While most previous work has focused on case-control data, the application of generative models to disorder-specific datasets and their ability to preserve diagnostic patterns remain relatively unexplored. Hence, in this study, we investigated the use of a generative adversarial network (GAN) in the context of Alzheimer's disease (AD) to generate functional network connectivity (FNC) and T1-weighted structural magnetic resonance imaging data from each other. We employed a cycle-GAN to synthesize data in an unpaired data transition and enhanced the transition by integrating weak supervision in cases where paired data were available. Our findings revealed that our model could offer remarkable capability, achieving a structural similarity index measure (SSIM) of $0.89 pm 0.003$ for T1s and a correlation of $0.71 pm 0.004$ for FNCs. Moreover, our qualitative analysis revealed similar patterns between generated and actual data when comparing AD to cognitively normal (CN) individuals. In particular, we observed significantly increased functional connectivity in cerebellar-sensory motor and cerebellar-visual networks and reduced connectivity in cerebellar-subcortical, auditory-sensory motor, sensory motor-visual, and cerebellar-cognitive control networks. Additionally, the T1 images generated by our model showed a similar pattern of atrophy in the hippocampal and other temporal regions of Alzheimer's patients.
Create account to get full access
Overview
- This paper presents a method using Generative Adversarial Networks (GANs) to translate between different medical imaging modalities, specifically to generate 3D amyloid-beta PET images from structural MRI scans.
- The goal is to unveil potential Alzheimer's disease biomarkers by leveraging the complementary information in these different imaging modalities.
- The proposed cross-modality translation approach could enable the synthesis of missing PET scans, which are more expensive and less widely available than MRI.
Plain English Explanation
Alzheimer's disease is a devastating neurological condition that affects memory and cognition. Researchers are working to develop better ways to detect and understand the disease early on, before symptoms become severe. One promising approach is to analyze medical images of the brain, such as MRI and PET scans, which can reveal changes associated with Alzheimer's.
However, PET scans, which measure specific molecules like amyloid-beta that are linked to Alzheimer's, are more expensive and less widely available than MRI scans. This paper proposes a way to use machine learning to "translate" between these two imaging modalities. The idea is to train a neural network, called a Generative Adversarial Network (GAN), to generate realistic-looking PET images from MRI scans.
By being able to produce synthetic PET scans, researchers could potentially uncover new biomarkers - measurable indicators of the disease - that are hidden in the complementary information captured by the two imaging techniques. This could lead to earlier and more accurate Alzheimer's diagnosis, as well as better understanding of the underlying disease processes.
The GAN approach proposed in this paper allows for the generation of 3D PET images from 3D MRI scans, which provides more detailed spatial information compared to 2D images. This could be particularly useful for localizing and characterizing Alzheimer's-related changes in the brain.
Technical Explanation
The authors develop a Conditional GAN (cGAN) architecture to translate between 3D MRI and 3D amyloid-beta PET images. The generator network in the cGAN is tasked with generating synthetic PET images from input MRI scans, while the discriminator network tries to distinguish the generated PET images from real ones.
The model is trained on a dataset of paired MRI and PET scans from Alzheimer's patients and healthy controls. The authors use various loss functions to optimize the generator to produce PET images that are not only visually realistic, but also capture the underlying disease-related patterns present in the real PET data.
Experiments show that the proposed cross-modality translation approach is able to generate high-quality 3D PET images from MRI scans. The synthetic PET images demonstrate strong spatial correspondence with the ground truth PET scans, suggesting that the model has learned to capture the salient disease-related information present in the two imaging modalities.
Critical Analysis
The paper presents a promising approach for leveraging the complementary information in MRI and PET scans to uncover Alzheimer's disease biomarkers. By enabling the synthesis of PET images from more widely available MRI data, the proposed method could significantly expand the pool of imaging data available for research and clinical applications.
However, the authors acknowledge several limitations of the current work. First, the dataset used for training and evaluation is relatively small, which may limit the generalizability of the results. Secondly, the authors do not provide extensive validation of the clinical relevance of the generated PET images, such as their ability to accurately detect Alzheimer's-related changes or predict disease progression.
Additionally, while the spatial correspondence between synthetic and real PET images is encouraging, more work is needed to ensure the generated images faithfully capture the underlying disease biomarkers. Artifacts or distortions introduced by the GAN model could potentially lead to misinterpretation of the synthetic PET data.
Further research is also needed to understand the limitations of this cross-modality translation approach and to explore ways to make the generated images more clinically useful. Integrating the synthetic PET data with other modalities, such as cognitive assessments or genetic data, could also help provide a more comprehensive picture of Alzheimer's disease pathology.
Conclusion
This paper presents a novel GAN-based approach for cross-modality translation between MRI and amyloid-beta PET scans, with the goal of uncovering potential Alzheimer's disease biomarkers. The ability to generate high-quality synthetic PET images from more widely available MRI data could significantly expand the pool of imaging data available for Alzheimer's research and clinical applications.
While the preliminary results are promising, more work is needed to ensure the clinical relevance and reliability of the generated PET images. Integrating the synthetic data with other modalities and validating its ability to detect and predict Alzheimer's-related changes will be crucial next steps. Overall, this research represents an important step towards leveraging the complementary information in multimodal medical imaging to advance our understanding and early detection of this devastating neurodegenerative disease.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
An interpretable generative multimodal neuroimaging-genomics framework for decoding Alzheimer's disease
Giorgio Dolci (Department of Engineering for Innovation Medicine, University of Verona, Verona, Italy, Tri-Institutional Center for Translational Research in Neuroimaging and Data Science), Federica Cruciani (Department of Engineering for Innovation Medicine, University of Verona, Verona, Italy), Md Abdur Rahaman (Tri-Institutional Center for Translational Research in Neuroimaging and Data Science), Anees Abrol (Tri-Institutional Center for Translational Research in Neuroimaging and Data Science), Jiayu Chen (Tri-Institutional Center for Translational Research in Neuroimaging and Data Science), Zening Fu (Tri-Institutional Center for Translational Research in Neuroimaging and Data Science), Ilaria Boscolo Galazzo (Department of Engineering for Innovation Medicine, University of Verona, Verona, Italy), Gloria Menegaz (Department of Engineering for Innovation Medicine, University of Verona, Verona, Italy), Vince D. Calhoun (Tri-Institutional Center for Translational Research in Neuroimaging and Data Science)
0
0
Alzheimer's disease (AD) is the most prevalent form of dementia with a progressive decline in cognitive abilities. The AD continuum encompasses a prodormal stage known as Mild Cognitive Impairment (MCI), where patients may either progress to AD or remain stable. In this study, we leveraged structural and functional MRI to investigate the disease-induced grey matter and functional network connectivity changes. Moreover, considering AD's strong genetic component, we introduce SNPs as a third channel. Given such diverse inputs, missing one or more modalities is a typical concern of multimodal methods. We hence propose a novel deep learning-based classification framework where generative module employing Cycle GANs was adopted to impute missing data within the latent space. Additionally, we adopted an Explainable AI method, Integrated Gradients, to extract input features relevance, enhancing our understanding of the learned representations. Two critical tasks were addressed: AD detection and MCI conversion prediction. Experimental results showed that our model was able to reach the SOA in the classification of CN/AD reaching an average test accuracy of $0.926pm0.02$. For the MCI task, we achieved an average prediction accuracy of $0.711pm0.01$ using the pre-trained model for CN/AD. The interpretability analysis revealed significant grey matter modulations in cortical and subcortical brain areas well known for their association with AD. Moreover, impairments in sensory-motor and visual resting state network connectivity along the disease continuum, as well as mutations in SNPs defining biological processes linked to amyloid-beta and cholesterol formation clearance and regulation, were identified as contributors to the achieved performance. Overall, our integrative deep learning approach shows promise for AD detection and MCI prediction, while shading light on important biological insights.
6/21/2024
🧠
SNeurodCNN: Structure-focused Neurodegeneration Convolutional Neural Network for Modelling and Classification of Alzheimer's Disease
Simisola Odimayo, Chollette C. Olisah, Khadija Mohammed
0
0
Alzheimer's disease (AD), the predominant form of dementia, is a growing global challenge, emphasizing the urgent need for accurate and early diagnosis. Current clinical diagnoses rely on radiologist expert interpretation, which is prone to human error. Deep learning has thus far shown promise for early AD diagnosis. However, existing methods often overlook focal structural atrophy critical for enhanced understanding of the cerebral cortex neurodegeneration. This paper proposes a deep learning framework that includes a novel structure-focused neurodegeneration CNN architecture named SNeurodCNN and an image brightness enhancement preprocessor using gamma correction. The SNeurodCNN architecture takes as input the focal structural atrophy features resulting from segmentation of brain structures captured through magnetic resonance imaging (MRI). As a result, the architecture considers only necessary CNN components, which comprises of two downsampling convolutional blocks and two fully connected layers, for achieving the desired classification task, and utilises regularisation techniques to regularise learnable parameters. Leveraging mid-sagittal and para-sagittal brain image viewpoints from the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset, our framework demonstrated exceptional performance. The para-sagittal viewpoint achieved 97.8% accuracy, 97.0% specificity, and 98.5% sensitivity, while the mid-sagittal viewpoint offered deeper insights with 98.1% accuracy, 97.2% specificity, and 99.0% sensitivity. Model analysis revealed the ability of SNeurodCNN to capture the structural dynamics of mild cognitive impairment (MCI) and AD in the frontal lobe, occipital lobe, cerebellum, temporal, and parietal lobe, suggesting its potential as a brain structural change digi-biomarker for early AD diagnosis. This work can be reproduced using code we made available on GitHub.
6/3/2024
Generative artificial intelligence in ophthalmology: multimodal retinal images for the diagnosis of Alzheimer's disease with convolutional neural networks
I. R. Slootweg, M. Thach, K. R. Curro-Tafili, F. D. Verbraak, F. H. Bouwman, Y. A. L. Pijnenburg, J. F. Boer, J. H. P. de Kwisthout, L. Bagheriye, P. J. Gonz'alez
0
0
Background/Aim. This study aims to predict Amyloid Positron Emission Tomography (AmyloidPET) status with multimodal retinal imaging and convolutional neural networks (CNNs) and to improve the performance through pretraining with synthetic data. Methods. Fundus autofluorescence, optical coherence tomography (OCT), and OCT angiography images from 328 eyes of 59 AmyloidPET positive subjects and 108 AmyloidPET negative subjects were used for classification. Denoising Diffusion Probabilistic Models (DDPMs) were trained to generate synthetic images and unimodal CNNs were pretrained on synthetic data and finetuned on real data or trained solely on real data. Multimodal classifiers were developed to combine predictions of the four unimodal CNNs with patient metadata. Class activation maps of the unimodal classifiers provided insight into the network's attention to inputs. Results. DDPMs generated diverse, realistic images without memorization. Pretraining unimodal CNNs with synthetic data improved AUPR at most from 0.350 to 0.579. Integration of metadata in multimodal CNNs improved AUPR from 0.486 to 0.634, which was the best overall best classifier. Class activation maps highlighted relevant retinal regions which correlated with AD. Conclusion. Our method for generating and leveraging synthetic data has the potential to improve AmyloidPET prediction from multimodal retinal imaging. A DDPM can generate realistic and unique multimodal synthetic retinal images. Our best performing unimodal and multimodal classifiers were not pretrained on synthetic data, however pretraining with synthetic data slightly improved classification performance for two out of the four modalities.
6/27/2024
Trustworthy Enhanced Multi-view Multi-modal Alzheimer's Disease Prediction with Brain-wide Imaging Transcriptomics Data
Shan Cong, Zhoujie Fan, Hongwei Liu, Yinghan Zhang, Xin Wang, Haoran Luo, Xiaohui Yao
0
0
Brain transcriptomics provides insights into the molecular mechanisms by which the brain coordinates its functions and processes. However, existing multimodal methods for predicting Alzheimer's disease (AD) primarily rely on imaging and sometimes genetic data, often neglecting the transcriptomic basis of brain. Furthermore, while striving to integrate complementary information between modalities, most studies overlook the informativeness disparities between modalities. Here, we propose TMM, a trusted multiview multimodal graph attention framework for AD diagnosis, using extensive brain-wide transcriptomics and imaging data. First, we construct view-specific brain regional co-function networks (RRIs) from transcriptomics and multimodal radiomics data to incorporate interaction information from both biomolecular and imaging perspectives. Next, we apply graph attention (GAT) processing to each RRI network to produce graph embeddings and employ cross-modal attention to fuse transcriptomics-derived embedding with each imagingderived embedding. Finally, a novel true-false-harmonized class probability (TFCP) strategy is designed to assess and adaptively adjust the prediction confidence of each modality for AD diagnosis. We evaluate TMM using the AHBA database with brain-wide transcriptomics data and the ADNI database with three imaging modalities (AV45-PET, FDG-PET, and VBM-MRI). The results demonstrate the superiority of our method in identifying AD, EMCI, and LMCI compared to state-of-the-arts. Code and data are available at https://github.com/Yaolab-fantastic/TMM.
6/24/2024