Feature importance to explain multimodal prediction models. A clinical use case

Overview

• This paper presents a multimodal deep learning model for predicting post-operative mortality in elderly hip fracture patients.
• The model uses pre-operative (patient data, hip and chest images) and per-operative (vital signals, medications) data to predict the risk of mortality.
• The authors use Shapley values to explain the model's predictions, allowing for interpretable and clinically applicable outcomes.

Plain English Explanation

Hip fractures are a serious health issue, especially for elderly patients. Surgery to treat these fractures can sometimes lead to complications that increase the risk of early death. An early warning system that identifies high-risk patients could help doctors monitor these patients more closely and address potential issues quickly, or inform the patient about their risks.

The researchers in this study developed a machine learning model that can predict the risk of a patient dying after hip fracture surgery. Their model uses different types of data, including the patient's medical history, images of the patient's hip and chest before surgery, and measurements of the patient's vital signs and medications during the surgery. The model combines these different data sources using advanced deep learning techniques.

Importantly, the researchers also developed a way to explain the model's predictions, using a technique called Shapley values. This allows doctors to understand why the model is making particular predictions, which is crucial for using the model in a real clinical setting. The researchers found that Shapley values can be used to estimate the relative importance of each data source in the model's predictions, both globally and for specific patients.

Technical Explanation

The researchers developed a multimodal deep learning model to predict post-operative mortality in elderly hip fracture patients. The model takes in both pre-operative data (static patient information, hip and chest X-ray images) and per-operative data (vital signs, medications) to make its predictions.

For the image data, the researchers used a pre-trained ResNet model to extract relevant features. The vital sign data was processed using an LSTM neural network. These different data streams were then combined using a multimodal fusion approach.

To make the model's predictions interpretable, the researchers computed Shapley values for each input feature. Shapley values estimate the relative contribution of each feature to the model's overall prediction. The researchers found that Shapley values could be used to explain the model's predictions both globally (across all patients) and locally (for individual patients).

Furthermore, the researchers developed a modified version of the chain rule to propagate Shapley values through the sequence of models in their multimodal architecture. This allowed them to provide interpretable explanations for the model's predictions at the local level.

Critical Analysis

The researchers acknowledge several limitations in their work. First, the study was conducted on a single-center dataset, so the generalizability of the results to other populations is unclear. Second, the model was trained on historical data, which may not reflect current clinical practices and outcomes.

Additionally, while the Shapley value approach provides interpretable explanations, it does not necessarily reveal the underlying causal relationships between the input features and the predicted outcome. Further research is needed to better understand the mechanisms driving the model's predictions.

Another potential concern is the ethical implications of using an AI system to predict mortality risk. While the model could help identify high-risk patients, there are concerns about the potential for bias, discrimination, and the psychological impact on patients. Careful consideration of these issues is crucial before deploying such a system in a clinical setting.

Conclusion

This study presents a promising approach for using multimodal deep learning to predict post-operative mortality risk in elderly hip fracture patients. The researchers' use of Shapley values to explain the model's predictions is a particularly notable contribution, as it enhances the model's clinical applicability and transparency.

However, further research is needed to validate the model's performance on more diverse datasets and to better understand the causal relationships underlying the predictions. Additionally, careful consideration of the ethical implications of such a system is crucial before it can be deployed in real-world clinical settings.

Overall, this work represents an important step towards developing explainable and clinically useful AI systems for healthcare applications.