Detecting Heart Disease from Multi-View Ultrasound Images via Supervised Attention Multiple Instance Learning

Read original: arXiv:2306.00003 - Published 4/8/2024 by Zhe Huang, Benjamin S. Wessler, Michael C. Hughes

👨‍🏫

Overview

Aortic stenosis (AS) is a heart valve condition that causes significant health problems and death
AS is often underdiagnosed and undertreated
Diagnosing AS typically involves expert review of echocardiography images, which can be time-consuming
Automating AS detection using deep learning could help improve diagnosis, but previous approaches have lacked accuracy

Plain English Explanation

Aortic stenosis is a problem with the aortic valve in the heart. This valve can become narrowed and stiff, making it harder for the heart to pump blood through. Aortic stenosis can be very serious, leading to lots of health issues and even death. But the condition is often missed or not treated properly.

Doctors usually diagnose aortic stenosis by looking at images of the heart taken with an ultrasound machine. An expert has to carefully review many different views of the heart to identify the aortic valve and assess if it is working correctly. This process can be time-consuming.

Researchers have tried to use artificial intelligence (AI) deep learning to automate the diagnosis of aortic stenosis from these heart ultrasound images. But the AI systems developed so far have not been accurate enough. One problem is that they just take an average of all the images, when in fact some views of the heart are more important than others for diagnosing aortic stenosis.

The researchers in this paper developed a new AI approach to address these limitations. Their key innovations are:

Using a "supervised attention" technique to guide the AI to focus on the most relevant views of the aortic valve.
Applying a novel "self-supervised pretraining" strategy to help the AI learn more useful representations of the whole heart ultrasound study, not just individual images.

These advances allowed the researchers to build an AI system that is more accurate at detecting aortic stenosis, while also being smaller in size. This could make the technology more practical for real-world clinical use.

Technical Explanation

The researchers developed a new end-to-end multiple instance learning (MIL) approach for automating the detection of aortic stenosis from transthoracic echocardiography (TTE) studies.

Previous attempts at aortic stenosis detection using deep learning have faced accuracy issues due to relying on inflexible averaging across all images in a study. The researchers also found that off-the-shelf attention-based MIL methods performed poorly.

To address these limitations, the researchers introduced two key methodological innovations:

Supervised Attention: A supervised attention technique was used to guide the learned attention mechanism to focus on the TTE views most relevant for diagnosing aortic stenosis. This helps the model identify the critical visual cues.
Self-Supervised Pretraining: Instead of pretraining on individual TTE images, the researchers applied a novel self-supervised contrastive learning strategy to the representation of the entire TTE study. This allowed the model to learn more holistic and generalizable representations.

The researchers evaluated their approach on an open-access TTE dataset as well as an external validation set. Their method outperformed previous deep learning approaches for aortic stenosis detection, while also reducing the overall model size.

Critical Analysis

The researchers acknowledge several limitations and areas for further work in their paper:

The dataset used was relatively small, so further validation on larger, more diverse datasets is needed.
The model was trained and evaluated on resting TTE studies, but aortic stenosis can manifest differently during physical activity. Incorporating stress echocardiography data could improve real-world performance.
The researchers did not compare their approach to human expert performance on the task. Evaluating the model against clinical experts would help quantify its potential impact.

Additionally, while the supervised attention and self-supervised pretraining innovations are promising, it's unclear how generalizable these techniques are to other medical imaging tasks beyond aortic stenosis detection. Further research exploring the broader applicability of these methods would be valuable.

Overall, the researchers have made a meaningful contribution to automating aortic stenosis diagnosis from echocardiography. However, as with any AI system in healthcare, careful evaluation, monitoring, and integration into clinical workflows will be crucial to ensuring safe and effective deployment.

Conclusion

This paper presents a new deep learning approach for automated detection of aortic stenosis from echocardiography images. By incorporating supervised attention and self-supervised pretraining innovations, the researchers were able to develop a more accurate and efficient model compared to previous methods.

If validated further, this technology could help improve the diagnosis and management of aortic stenosis, which is a common and serious heart valve condition that is often underdiagnosed. Automating aspects of the diagnostic process has the potential to save time for clinicians and increase access to care.

However, as with any AI system in healthcare, ongoing evaluation, monitoring, and responsible integration into clinical workflows will be critical to realizing the full benefits while mitigating risks. Continued research in this area could yield valuable advances in cardiovascular imaging and diagnosis.

This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!

Follow @aimodelsfyi on 𝕏 →

Related Papers

👨‍🏫

Detecting Heart Disease from Multi-View Ultrasound Images via Supervised Attention Multiple Instance Learning

Zhe Huang, Benjamin S. Wessler, Michael C. Hughes

Aortic stenosis (AS) is a degenerative valve condition that causes substantial morbidity and mortality. This condition is under-diagnosed and under-treated. In clinical practice, AS is diagnosed with expert review of transthoracic echocardiography, which produces dozens of ultrasound images of the heart. Only some of these views show the aortic valve. To automate screening for AS, deep networks must learn to mimic a human expert's ability to identify views of the aortic valve then aggregate across these relevant images to produce a study-level diagnosis. We find previous approaches to AS detection yield insufficient accuracy due to relying on inflexible averages across images. We further find that off-the-shelf attention-based multiple instance learning (MIL) performs poorly. We contribute a new end-to-end MIL approach with two key methodological innovations. First, a supervised attention technique guides the learned attention mechanism to favor relevant views. Second, a novel self-supervised pretraining strategy applies contrastive learning on the representation of the whole study instead of individual images as commonly done in prior literature. Experiments on an open-access dataset and an external validation set show that our approach yields higher accuracy while reducing model size.

4/8/2024

🔮

New!Reliable Multi-View Learning with Conformal Prediction for Aortic Stenosis Classification in Echocardiography

Ang Nan Gu, Michael Tsang, Hooman Vaseli, Teresa Tsang, Purang Abolmaesumi

The fundamental problem with ultrasound-guided diagnosis is that the acquired images are often 2-D cross-sections of a 3-D anatomy, potentially missing important anatomical details. This limitation leads to challenges in ultrasound echocardiography, such as poor visualization of heart valves or foreshortening of ventricles. Clinicians must interpret these images with inherent uncertainty, a nuance absent in machine learning's one-hot labels. We propose Re-Training for Uncertainty (RT4U), a data-centric method to introduce uncertainty to weakly informative inputs in the training set. This simple approach can be incorporated to existing state-of-the-art aortic stenosis classification methods to further improve their accuracy. When combined with conformal prediction techniques, RT4U can yield adaptively sized prediction sets which are guaranteed to contain the ground truth class to a high accuracy. We validate the effectiveness of RT4U on three diverse datasets: a public (TMED-2) and a private AS dataset, along with a CIFAR-10-derived toy dataset. Results show improvement on all the datasets.

9/17/2024

AI-powered multimodal modeling of personalized hemodynamics in aortic stenosis

Caglar Ozturk, Daniel H. Pak, Luca Rosalia, Debkalpa Goswami, Mary E. Robakowski, Raymond McKay, Christopher T. Nguyen, James S. Duncan, Ellen T. Roche

Aortic stenosis (AS) is the most common valvular heart disease in developed countries. High-fidelity preclinical models can improve AS management by enabling therapeutic innovation, early diagnosis, and tailored treatment planning. However, their use is currently limited by complex workflows necessitating lengthy expert-driven manual operations. Here, we propose an AI-powered computational framework for accelerated and democratized patient-specific modeling of AS hemodynamics from computed tomography. First, we demonstrate that our automated meshing algorithms can generate task-ready geometries for both computational and benchtop simulations with higher accuracy and 100 times faster than existing approaches. Then, we show that our approach can be integrated with fluid-structure interaction and soft robotics models to accurately recapitulate a broad spectrum of clinical hemodynamic measurements of diverse AS patients. The efficiency and reliability of these algorithms make them an ideal complementary tool for personalized high-fidelity modeling of AS biomechanics, hemodynamics, and treatment planning.

7/2/2024

Multimodal Fusion of Echocardiography and Electronic Health Records for the Detection of Cardiac Amyloidosis

Zishun Feng, Joseph A. Sivak, Ashok K. Krishnamurthy

Cardiac amyloidosis, a rare and highly morbid condition, presents significant challenges for detection through echocardiography. Recently, there has been a surge in proposing machine-learning algorithms to identify cardiac amyloidosis, with the majority being imaging-based deep-learning approaches that require extensive data. In this study, we introduce a novel transformer-based multimodal fusion algorithm that leverages information from both imaging and electronic health records. Specifically, our approach utilizes echocardiography videos from both the parasternal long-axis (PLAX) view and the apical 4-chamber (A4C) view along with patients' demographic data, laboratory tests, and cardiac metrics to predict the probability of cardiac amyloidosis. We evaluated our method using 5-fold cross-validation on a dataset comprising 41 patients and achieved an Area Under the Receiver Operating Characteristic curve (AUROC) of 0.94. The experimental results demonstrate that our approach can achieve competitive results with a significantly smaller dataset compared to prior imaging-based methods that required data from thousands of patients. This underscores the potential of leveraging multimodal data to enhance diagnostic accuracy in the identification of complex cardiac conditions such as cardiac amyloidosis.

6/10/2024