Evaluating Explanatory Capabilities of Machine Learning Models in Medical Diagnostics: A Human-in-the-Loop Approach

Introduction

This section introduces Explainable AI (XAI) and discusses its importance in making AI systems more understandable to humans. It highlights the advantages of XAI, such as fostering confidence in model predictions, promoting responsible AI development, aiding in debugging, and allowing for auditing of AI models.

The section describes the evolution of AI systems from symbolic AI models, which were inherently explainable, to machine learning models that can learn from data but often lack interpretability. It explains the trade-off between model interpretability and performance, with more interpretable models tending to perform less well than less interpretable models.

The authors mention the development of "agnostic" techniques to include explainability in opaque models but point out their limitations in terms of reliability and computational efficiency. They emphasize the need to validate the explanatory capabilities of these models, especially in complex domains like medical environments.

The paper focuses on solving a problem related to pancreatic cancer treatment using classical machine learning models such as Decision Trees, Random Forest, and XGBoost. The authors aim to validate the explainability of these models by comparing different explainability methods and assessing their medical significance through collaboration with human experts and the use of medical guidelines.

The contribution of the paper lies in the development of different ways to evaluate explainability models based on human expert collaboration and domain-specific guidelines, applied to the pancreatic cancer treatment problem.

State of the art

The provided text discusses explainability in AI, focusing on its importance in the healthcare domain, particularly for pancreatic cancer diagnosis and treatment. Key points:

• Explainable AI techniques help humans understand, trust, and manage machine learning systems. Explainability goes beyond interpretability.
• Explainability methods can be classified by scope (local or global) and generality (model-agnostic or model-specific).
• Building trust is crucial when creating AI systems for collaboration with humans, especially in critical domains like healthcare.
• Examples of explainability techniques applied to medical ML models are provided, with SHAP and GradCAM being the most popular.
• Problems with explainability methods are discussed, including inconsistency, instability, computational inefficiency, and the need for domain experts to evaluate explanations.
• Pancreatic cancer staging is explained, using the TNM system and classification into stages 0, I, II, III, and IV.
• The NCCN Clinical Practice Guidelines for Pancreatic Adenocarcinoma are introduced, organized into processes (PANCs) for different stages of diagnosis and treatment.
• The guidelines establish four main diagnostic groups: resectable, borderline resectable, locally advanced, and metastatic disease, which guide treatment decisions.

Data and methodology

The dataset used in this work was obtained from The Cancer Genome Atlas Program and consists of 181 pancreatic cancer cases, of which 117 received chemotherapy treatment. Each case has 158 features ranging from family history data to treatment and follow-up information.

A meticulous feature selection process was performed, involving data pruning, eliminating redundant information, and asking a panel of oncologists to select relevant features for treatment selection. This resulted in 27 features and the target therapy type variable.

Three different feature sets were created:

1. Recommended set: 14 features rated as highly relevant or relevant by medical experts
2. Maximum set: All 27 features, including those considered barely relevant
3. Minimum set: 5 features (Age, T, N, M, Stage) achieved through dimensionality reduction

The medical guidelines from "Pancreatic Adenocarcinoma - NCCN Clinical Practice Guidelines in Oncology" were analyzed to evaluate feature importance regarding chemotherapy treatment. A simplified graph of the diagnostic decisions involving chemotherapy was created.

Three types of machine learning models were chosen for comparison: Decision Tree, Random Forest, and XGBoost. Their advantages and disadvantages in terms of accuracy and understandability are discussed.

To enhance the interpretability of the ML models, explainability techniques were employed:

• Model-specific methods: Mean Decrease in Impurity (MDI) and Mean Decrease Accuracy (MDA) for tree-based models
• Model-agnostic methods: SHapley Additive exPlanations (SHAP) and Locally Interpretable Model-agnostic Explanations (LIME)

Results

The results of the different machine learning models are presented in terms of accuracy and interpretability, using the three feature sets discussed in section 3.2. The models are compared with medical guidelines and expert criteria.

In terms of accuracy, the models built with the minimum set of features performed best, suggesting that adding more features does not always improve accuracy and can make models more complex. Decision trees were generally more accurate, with results close to XGBoost. The small training dataset size may explain why the more complex XGBoost did not outperform simpler models.

For interpretability, various explainable AI methods were used to extract feature importance and compare it to medical guidelines and expert opinion. With the minimum feature set, pathologic stage, age and pathologic N were consistently identified as the most important features across methods.

Weighted Jaccard similarity was used to quantitatively compare feature importance rankings between XAI methods, guidelines and experts. Similarity was high between guidelines and experts. For the recommended feature set, decision trees considered fewer features than experts and guidelines, while XGBoost spread importance more evenly, leading to higher similarity with expert opinion.

Overall, the results show that accurate and interpretable models can be built with a small set of key clinical features. The interpretations produced by XAI methods generally align with expert knowledge, though some differences exist. Simpler models like decision trees may be preferable for easier interpretability.