Unmasking Bias in AI: A Systematic Review of Bias Detection and Mitigation Strategies in Electronic Health Record-based Models

Overview

• This study reviews methods to detect and mitigate diverse forms of bias in AI models developed using electronic health records (EHRs).
• The researchers conducted a systematic review following the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines.
• The review identified key biases, outlined strategies for detecting and mitigating bias throughout the AI model development process, and analyzed metrics for bias assessment.

Plain English Explanation

Healthcare systems are increasingly using artificial intelligence (AI) models developed with electronic health records (EHRs) to improve decision-making and patient outcomes. However, these AI models can sometimes exhibit biases that can worsen healthcare disparities. This study examines different types of biases that can arise in AI models built using EHR data and explores methods to detect and address these biases.

The researchers systematically reviewed scientific literature to identify the key biases, strategies for detecting and mitigating them, and metrics for assessing bias. They found that AI models in healthcare can be affected by various types of biases, including algorithmic bias, confounding bias, implicit bias, measurement bias, selection bias, and temporal bias.

To address these biases, the researchers found that data collection and preprocessing techniques like resampling, reweighting, and transformation can be effective. They also identified the need for standardized, generalizable, and interpretable methodologies to foster the creation of ethical AI systems that promote fairness and equity in healthcare.

Technical Explanation

The researchers conducted a systematic review following the PRISMA guidelines to identify and analyze methods for detecting and mitigating bias in AI models developed using EHR data. They searched PubMed, Web of Science, and IEEE databases for relevant articles published between January 1, 2010, and December 17, 2023, and identified 20 articles that met their criteria.

The review revealed six major types of bias that can affect AI models in healthcare settings: algorithmic bias, confounding bias, implicit bias, measurement bias, selection bias, and temporal bias. The AI models were primarily developed for predictive tasks, such as disease diagnosis or risk prediction.

The researchers found that four of the studies focused on the detection of implicit and algorithmic biases, employing fairness metrics like statistical parity, equal opportunity, and predictive equity. Additionally, 60 of the studies proposed various strategies for mitigating biases, especially targeting implicit and selection biases. These strategies were evaluated through both performance (e.g., accuracy, AUROC) and fairness metrics, and predominantly involved data collection and preprocessing techniques like resampling, reweighting, and transformation.

Critical Analysis

The review highlights the varied and evolving nature of strategies to address bias in EHR-based AI models. However, the researchers note that there is an urgent need for the establishment of standardized, generalizable, and interpretable methodologies to foster the creation of ethical AI systems that promote fairness and equity in healthcare.

One potential limitation of the study is that it may not have captured all relevant research, as the review was limited to articles published between January 1, 2010, and December 17, 2023. Additionally, the review focused on a relatively small set of 20 articles, which may not fully represent the breadth of work in this rapidly evolving field.

Furthermore, the review did not deeply explore the role of interdisciplinary expertise in advancing equitable and explainable AI, which could be a crucial factor in developing effective bias mitigation strategies. Collaboration between AI researchers, healthcare professionals, ethicists, and other relevant stakeholders may be necessary to address the complex challenges of bias in healthcare AI.

Conclusion

This systematic review highlights the need for robust and systematic approaches to detecting and mitigating bias in AI models developed using EHR data. The researchers identified key types of bias and various strategies for addressing them, but emphasized the urgent need for standardized, generalizable, and interpretable methodologies to ensure the development of ethical AI systems that promote fairness and equity in healthcare. Continued research and cross-disciplinary collaboration will be crucial to realizing the transformative potential of AI in healthcare while safeguarding against the risks of bias and healthcare disparities.