End-To-End Clinical Trial Matching with Large Language Models

Read original: arXiv:2407.13463 - Published 7/19/2024 by Dyke Ferber, Lars Hilgers, Isabella C. Wiest, Marie-Elisabeth Le{ss}mann, Jan Clusmann, Peter Neidlinger, Jiefu Zhu, Georg Wolflein, Jacqueline Lammert, Maximilian Tschochohei and 6 others
Total Score

0

End-To-End Clinical Trial Matching with Large Language Models

Sign in to get full access

or

If you already have an account, we'll log you in

Overview

  • This paper explores the use of large language models (LLMs) for end-to-end clinical trial matching, which involves matching patients to suitable clinical trials based on their medical profiles.
  • The researchers developed a novel architecture that utilizes a pipeline of LLMs to accomplish this task, demonstrating significant improvements over traditional methods.
  • The paper presents a comprehensive evaluation of the proposed approach, highlighting its potential to streamline the clinical trial recruitment process and facilitate access to novel treatments for patients.

Plain English Explanation

The paper describes a new way to match patients with clinical trials using large language models (LLMs) - powerful AI systems that can understand and generate human-like text. The authors developed an approach called "End-To-End Clinical Trial Matching with Large Language Models" that uses a series of LLMs to automatically connect patients with the most relevant clinical trials for their medical conditions.

Typically, the process of finding suitable clinical trials for patients is time-consuming and complex. Doctors and researchers have to manually review a patient's medical history and compare it to the eligibility criteria for hundreds of available trials. This new AI-powered approach aims to streamline and automate this matching process, making it faster and more efficient.

The key idea is to use LLMs to understand the patient's medical profile and the details of each clinical trial, and then match them together. This involves breaking the task down into several steps, with different LLMs handling different parts of the process. For example, one model might extract relevant medical information from the patient's records, while another evaluates how well the patient fits the criteria for a given trial.

The researchers tested their approach on real-world data and found that it significantly outperformed traditional methods. This suggests that LLMs could be a powerful tool for helping more patients access potentially life-saving clinical trials, which are often underutilized due to the challenges of matching patients to the right opportunities.

Technical Explanation

The paper presents an end-to-end clinical trial matching approach that leverages a pipeline of large language models (LLMs) to automate the process of connecting patients with suitable clinical trials. The proposed architecture, dubbed "End-To-End Clinical Trial Matching with Large Language Models", consists of several interconnected LLM components that work together to extract relevant information from patient records, understand the eligibility criteria of clinical trials, and make accurate matches.

The first step involves using an LLM-powered medical information extraction model to parse patient medical records and identify key details such as diagnoses, treatments, and lab results. This information is then used by another LLM to evaluate how well the patient's profile aligns with the eligibility criteria of different clinical trials. A third LLM is responsible for encoding the textual descriptions of the clinical trials to enable efficient retrieval and comparison.

The researchers evaluated their approach on a large dataset of real-world patient records and clinical trial information, and found that it significantly outperformed traditional methods based on keyword matching and rule-based systems. The proposed pipeline demonstrated the ability to make accurate zero-shot predictions, matching patients to suitable trials even when there was no direct overlap in the terminology used.

[The authors also discussed the potential of their approach to be integrated into a larger AI-powered clinical decision support system that could assist healthcare providers in navigating the complex landscape of clinical trials and improving patient access to novel treatments](https://aimodels.fyi/papers/arxiv/cancerllm-large-language-model-cancer-domain).

Critical Analysis

The paper presents a promising approach to automating the clinical trial matching process, which could have significant implications for improving patient access to innovative therapies. By leveraging the power of large language models, the researchers have developed a solution that can extract relevant information from patient records and accurately match them to suitable clinical trials, even in cases where there is limited overlap in the terminology used.

However, the authors acknowledge several limitations and areas for further research. For instance, the performance of the proposed pipeline is heavily dependent on the quality and completeness of the patient data, which may not always be available or standardized in real-world settings. Additionally, the ethical and legal implications of automating such a sensitive medical decision-making process warrant further investigation and consideration.

Moreover, the paper does not explore the potential biases that may arise from the language models used, which could lead to unintended discrimination or exclusion of certain patient populations. Addressing these challenges and ensuring the fairness and transparency of the system will be crucial for its successful deployment in clinical practice.

Despite these caveats, the work presented in this paper demonstrates the potential of large language models to transform the clinical trial matching process and improve patient access to novel treatments. As the field of AI-powered healthcare continues to evolve, it will be important for researchers and practitioners to carefully evaluate the tradeoffs and consider the broader implications of these technologies.

Conclusion

This paper introduces a novel end-to-end approach for matching patients to suitable clinical trials using a pipeline of large language models (LLMs). The proposed architecture leverages the powerful text understanding capabilities of LLMs to extract relevant information from patient records, evaluate eligibility criteria, and make accurate matches, outperforming traditional methods.

The demonstrated ability of the system to make zero-shot predictions and connect patients with trials even when there is no direct overlap in terminology suggests that this approach could significantly streamline the clinical trial recruitment process and improve patient access to innovative therapies. As the healthcare industry continues to embrace the potential of AI, this work highlights the transformative possibilities of LLMs in facilitating personalized and data-driven clinical decision-making.

However, the authors also acknowledge the need to address the ethical, legal, and technical challenges associated with the deployment of such systems in real-world settings. Continued research and collaboration between AI researchers, clinicians, and policymakers will be crucial to ensuring the responsible development and implementation of these technologies for the benefit of patients and the broader healthcare ecosystem.



This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!

Follow @aimodelsfyi on 𝕏 →

Related Papers

End-To-End Clinical Trial Matching with Large Language Models
Total Score

0

End-To-End Clinical Trial Matching with Large Language Models

Dyke Ferber, Lars Hilgers, Isabella C. Wiest, Marie-Elisabeth Le{ss}mann, Jan Clusmann, Peter Neidlinger, Jiefu Zhu, Georg Wolflein, Jacqueline Lammert, Maximilian Tschochohei, Heiko Bohme, Dirk Jager, Mihaela Aldea, Daniel Truhn, Christiane Hoper, Jakob Nikolas Kather

Matching cancer patients to clinical trials is essential for advancing treatment and patient care. However, the inconsistent format of medical free text documents and complex trial eligibility criteria make this process extremely challenging and time-consuming for physicians. We investigated whether the entire trial matching process - from identifying relevant trials among 105,600 oncology-related clinical trials on clinicaltrials.gov to generating criterion-level eligibility matches - could be automated using Large Language Models (LLMs). Using GPT-4o and a set of 51 synthetic Electronic Health Records (EHRs), we demonstrate that our approach identifies relevant candidate trials in 93.3% of cases and achieves a preliminary accuracy of 88.0% when matching patient-level information at the criterion level against a baseline defined by human experts. Utilizing LLM feedback reveals that 39.3% criteria that were initially considered incorrect are either ambiguous or inaccurately annotated, leading to a total model accuracy of 92.7% after refining our human baseline. In summary, we present an end-to-end pipeline for clinical trial matching using LLMs, demonstrating high precision in screening and matching trials to individual patients, even outperforming the performance of qualified medical doctors. Our fully end-to-end pipeline can operate autonomously or with human supervision and is not restricted to oncology, offering a scalable solution for enhancing patient-trial matching in real-world settings.

Read more

7/19/2024

Zero-Shot Clinical Trial Patient Matching with LLMs
Total Score

0

Zero-Shot Clinical Trial Patient Matching with LLMs

Michael Wornow, Alejandro Lozano, Dev Dash, Jenelle Jindal, Kenneth W. Mahaffey, Nigam H. Shah

Matching patients to clinical trials is a key unsolved challenge in bringing new drugs to market. Today, identifying patients who meet a trial's eligibility criteria is highly manual, taking up to 1 hour per patient. Automated screening is challenging, however, as it requires understanding unstructured clinical text. Large language models (LLMs) offer a promising solution. In this work, we explore their application to trial matching. First, we design an LLM-based system which, given a patient's medical history as unstructured clinical text, evaluates whether that patient meets a set of inclusion criteria (also specified as free text). Our zero-shot system achieves state-of-the-art scores on the n2c2 2018 cohort selection benchmark. Second, we improve the data and cost efficiency of our method by identifying a prompting strategy which matches patients an order of magnitude faster and more cheaply than the status quo, and develop a two-stage retrieval pipeline that reduces the number of tokens processed by up to a third while retaining high performance. Third, we evaluate the interpretability of our system by having clinicians evaluate the natural language justifications generated by the LLM for each eligibility decision, and show that it can output coherent explanations for 97% of its correct decisions and 75% of its incorrect ones. Our results establish the feasibility of using LLMs to accelerate clinical trial operations.

Read more

4/11/2024

💬

Total Score

0

PRISM: Patient Records Interpretation for Semantic Clinical Trial Matching using Large Language Models

Shashi Kant Gupta, Aditya Basu, Mauro Nievas, Jerrin Thomas, Nathan Wolfrath, Adhitya Ramamurthi, Bradley Taylor, Anai N. Kothari, Regina Schwind, Therica M. Miller, Sorena Nadaf-Rahrov, Yanshan Wang, Hrituraj Singh

Clinical trial matching is the task of identifying trials for which patients may be potentially eligible. Typically, this task is labor-intensive and requires detailed verification of patient electronic health records (EHRs) against the stringent inclusion and exclusion criteria of clinical trials. This process is manual, time-intensive, and challenging to scale up, resulting in many patients missing out on potential therapeutic options. Recent advancements in Large Language Models (LLMs) have made automating patient-trial matching possible, as shown in multiple concurrent research studies. However, the current approaches are confined to constrained, often synthetic datasets that do not adequately mirror the complexities encountered in real-world medical data. In this study, we present the first, end-to-end large-scale empirical evaluation of clinical trial matching using real-world EHRs. Our study showcases the capability of LLMs to accurately match patients with appropriate clinical trials. We perform experiments with proprietary LLMs, including GPT-4 and GPT-3.5, as well as our custom fine-tuned model called OncoLLM and show that OncoLLM, despite its significantly smaller size, not only outperforms GPT-3.5 but also matches the performance of qualified medical doctors. All experiments were carried out on real-world EHRs that include clinical notes and available clinical trials from a single cancer center in the United States.

Read more

4/30/2024

💬

Total Score

0

Matching Patients to Clinical Trials with Large Language Models

Qiao Jin, Zifeng Wang, Charalampos S. Floudas, Fangyuan Chen, Changlin Gong, Dara Bracken-Clarke, Elisabetta Xue, Yifan Yang, Jimeng Sun, Zhiyong Lu

Clinical trials are often hindered by the challenge of patient recruitment. In this work, we introduce TrialGPT, a first-of-its-kind large language model (LLM) framework to assist patient-to-trial matching. Given a patient note, TrialGPT predicts the patient's eligibility on a criterion-by-criterion basis and then consolidates these predictions to assess the patient's eligibility for the target trial. We evaluate the trial-level prediction performance of TrialGPT on three publicly available cohorts of 184 patients with over 18,000 trial annotations. We also engaged three physicians to label over 1,000 patient-criterion pairs to assess its criterion-level prediction accuracy. Experimental results show that TrialGPT achieves a criterion-level accuracy of 87.3% with faithful explanations, close to the expert performance (88.7%-90.0%). The aggregated TrialGPT scores are highly correlated with human eligibility judgments, and they outperform the best-competing models by 32.6% to 57.2% in ranking and excluding clinical trials. Furthermore, our user study reveals that TrialGPT can significantly reduce the screening time (by 42.6%) in a real-life clinical trial matching task. These results and analyses have demonstrated promising opportunities for clinical trial matching with LLMs such as TrialGPT.

Read more

4/30/2024