Pulmonary Embolism Mortality Prediction Using Multimodal Learning Based on Computed Tomography Angiography and Clinical Data

Overview

• This study aimed to develop deep learning models using Computed Tomography Pulmonary Angiography (CTPA) images, clinical data, and Pulmonary Embolism Severity Index (PESI) scores to predict mortality in patients with pulmonary embolism (PE).
• The researchers tested four different models: (1) using CTPA imaging features only, (2) using clinical variables only, (3) a multimodal model integrating both CTPA and clinical variables, and (4) a multimodal model fused with the calculated PESI score.
• The performance of these models was evaluated and compared to the PESI score alone using concordance index (c-index) and Net Reclassification Improvement metrics.
• Kaplan-Meier analysis was used to stratify patients into high- and low-risk groups based on the model predictions, and the impact of right ventricular (RV) dysfunction was also assessed.

Plain English Explanation

Pulmonary embolism (PE) is a serious condition where a blood clot blocks the arteries in the lungs, which can be fatal. Researchers in this study wanted to see if they could use deep learning models and different types of data to better predict which patients with PE are more likely to die.

They tested four different deep learning models:

1. One that used only the information from the CT scans of the patients' lungs (CTPA).
2. One that used only the patients' clinical data, like their age and medical history.
3. A "multimodal" model that combined the CTPA and clinical data.
4. A multimodal model that also included the Pulmonary Embolism Severity Index (PESI) score, which is a tool doctors use to estimate a patient's risk of dying from PE.

The researchers found that the multimodal models that used both CTPA and clinical data performed better at predicting which patients were more likely to die than using the PESI score alone. They also discovered that patients who were classified as "high risk" by the multimodal models were more likely to have a problem with the right side of their heart (right ventricular dysfunction), which can be a serious complication of PE.

Technical Explanation

This study utilized deep learning models to predict mortality in patients with pulmonary embolism (PE) by incorporating Computed Tomography Pulmonary Angiography (CTPA) imaging features, clinical data, and Pulmonary Embolism Severity Index (PESI) scores.

The researchers retrospectively analyzed data from 918 patients with 3,978 CTPA scans across three institutions. They developed four deep learning models:

1. A model using only CTPA imaging features
2. A model using only clinical variables
3. A multimodal model integrating both CTPA and clinical variables
4. A multimodal model fused with the calculated PESI score

The performance of these models was evaluated using concordance index (c-index) and Net Reclassification Improvement (NRI) metrics, and compared to PESI predictions using the Wilcoxon signed-rank test. Kaplan-Meier analysis was then used to stratify patients into high- and low-risk groups based on the model outputs, and the impact of right ventricular (RV) dysfunction was also assessed.

Critical Analysis

The study provides a comprehensive evaluation of deep learning models for predicting mortality in pulmonary embolism patients, incorporating multimodal data including CTPA imaging, clinical variables, and the established PESI score. The authors demonstrate that the multimodal models outperform the PESI score alone, suggesting that the integration of imaging and clinical data can lead to more accurate risk stratification.

However, the study does not provide detailed information about the specific deep learning architectures used, making it difficult to fully evaluate the technical aspects of the models. Additionally, the retrospective nature of the data and the single-center design may limit the generalizability of the findings. Future research could focus on validating the models in larger, multicenter cohorts and exploring the incorporation of other relevant data modalities to further improve the predictive performance.

Conclusion

This study shows that deep learning models that combine CTPA imaging, clinical data, and the PESI score can more accurately predict mortality in patients with pulmonary embolism compared to the PESI score alone. The ability to stratify patients into high- and low-risk groups based on these models could help clinicians make more informed decisions about treatment and prognosis. Further research is needed to validate these findings and explore ways to enhance the predictive power of these multimodal approaches for improving patient outcomes in PE.