The Literature Review Network: An Explainable Artificial Intelligence for Systematic Literature Reviews, Meta-analyses, and Method Development

Overview

• Systematic literature reviews are considered the highest quality of evidence in research.
• However, the review process is hindered by significant resource and data constraints.
• The Literature Review Network (LRN) is an explainable AI platform designed to automate the entire literature review process and adhere to PRISMA 2020 standards.

Plain English Explanation

The paper discusses Literature Review Network (LRN), which is an artificial intelligence (AI) system that can automatically conduct literature reviews. Literature reviews are important in research as they summarize the current state of knowledge on a particular topic. However, conducting these reviews manually is a time-consuming and resource-intensive process.

LRN aims to address this problem by using AI to automate the entire literature review process. The researchers evaluated LRN's performance in the domain of surgical glove practices, using search queries developed by experts. They found that LRN was able to achieve high classification accuracy without any expert training, and was able to identify key themes and insights that were nearly identical to those found by the clinical researchers. LRN was also able to complete the entire literature review process much faster than the manual review, reducing the time from 11 months to just 5 days.

The researchers conclude that explainable AI can accurately and efficiently conduct PRISMA-compliant systematic literature reviews, without the need for expert training. This could potentially revolutionize healthcare research by helping researchers quickly and accurately understand the current state of knowledge on a particular topic.

Technical Explanation

The researchers evaluated the performance of the Literature Review Network (LRN) in conducting a systematic literature review on surgical glove practices. They used three search strings developed by experts to query the PubMed database, and trained all LRN models using a non-expert.

The researchers benchmarked LRN's performance against an expert manual review, measuring concordance using the Jaccard index and confusion matrices. They also assessed LRN's explainability by linking key terms to the classification decisions.

The results show that LRN models demonstrated superior classification accuracy without expert training, achieving 84.78% and 85.71% accuracy. The highest-performing model achieved high interrater reliability (k = 0.4953) and strong explainability metrics, linking terms like 'reduce', 'accident', and 'sharp' with 'double-gloving'. Another LRN model covered 91.51% of the relevant literature despite diverging from the non-expert's judgments (k = 0.2174), with terms like 'latex', 'double' (gloves), and 'indication'.

Importantly, LRN outperformed the manual review, reducing the entire process from 19,920 minutes over 11 months to just 288.6 minutes over 5 days.

Critical Analysis

The paper provides a compelling demonstration of how explainable AI can be used to automate the systematic literature review process, without the need for expert training.

One potential limitation is that the evaluation was conducted in a single domain (surgical glove practices), and it's unclear how well the LRN system would perform in other domains. Additionally, the researchers noted that the non-expert training data may have introduced some biases or inconsistencies, which could impact the system's performance.

Further research is needed to explore the generalizability of the LRN approach, as well as to address any potential ethical or privacy concerns associated with automating the literature review process. It will also be important to continue exploring ways to enhance the explainability and transparency of the LRN system, to build trust and confidence in its outputs.

Conclusion

This study demonstrates that explainable AI can accurately and efficiently conduct PRISMA-compliant systematic literature reviews, without the need for expert training. The Literature Review Network (LRN) was able to summarize the results of surgical glove studies and identify key themes that were nearly identical to the findings of the clinical researchers, while completing the entire process much faster than a manual review.

This has the potential to revolutionize healthcare research by helping researchers quickly and accurately understand the current state of knowledge on a particular topic, allowing them to focus their efforts on advancing the field rather than being bogged down by the resource-intensive literature review process.