Artificial Intelligence (AI) Readiness to Support Evidence Synthesis by Workflow: Findings From a Review of Reviews.

内容摘要

Evidence synthesis is crucial for informing evidence-based practice across various fields. However, the traditional methodology is resource-intensive, and its findings can be outdated before publication. There is a growing trend toward integrating automation and artificial intelligence (AI) approaches into evidence synthesis to enhance efficiency, but standardized adoption is still pending. The goal of this study is to identify peer-reviewed evidence documenting AI readiness for evidence synthesis. We searched MEDLINE, Embase, and Global Index Medicus in May 2025 to identify review articles that evaluated evidence synthesis tools. Relevant study reviews and tool reviews published in English between January 2020 and May 2025 were included in our review of reviews. Tool features and performance metrics were extracted according to stages of the evidence synthesis workflow, including search, screening, appraisal, extraction, and synthesis. We included 21 studies in our review of reviews and identified 46 evidence synthesis tools. Nine tools supported all five stages of the evidence synthesis workflow, among which DistillerSR covered the most workflow-supporting features (19 out of 21). Ten of the identified tools reported sensitivity rates for AI-powered title/abstract screening, all of which achieved ≥ 95 % sensitivity in at least one configuration. Reported sensitivity rates of EPPI-Reviewer, Research Screener and SWIFT-Active Screener consistently reached the 95% threshold with varying degrees of automation. This review found peer-reviewed evidence supporting AI readiness for human-supervised automation of title/abstract screening. However, evidence documenting AI readiness for other evidence synthesis tasks remains limited. DistillerSR and EPPI-Reviewer demonstrated the broadest feature support and strong evidence for AI-powered title/abstract screening. Our study highlights the potential of AI to improve efficiency while maintaining high sensitivity in the screening stage. AI-powered screening may serve as a critical first step toward scaling rapid reviews into living evidence syntheses. Why was the study done: Reviewing large numbers of studies is essential for making informed decisions in healthcare and other fields. This process called evidence synthesis brings together findings from many studies to answer important questions. However, it can take a long time and requires a lot of effort from researchers. By the time a review is completed, new studies may already have been published, making the results less up to date. Artificial intelligence (AI) has been suggested as a way to speed up this process. AI tools can help with tasks such as searching for studies, screening which studies are relevant, extracting data from studies, and combining results. Despite growing interest, it is not yet clear how peer-reviewed literature documents the readiness of these tools. What did the researchers do: In this study, we looked at existing research to understand how well AI is currently supporting evidence synthesis. We searched major health databases for published review articles from 2020 to 2025 that evaluated tools used in evidence synthesis. We then collected evidence on what these tools can do and how well they perform at different stages of the review process. These stages include searching for studies, screening them for relevance, assessing their quality, extracting key information, and combining findings. What did the researchers find: We included 21 review articles and identified 46 different tools designed to support evidence synthesis. A few of these tools could support all stages of the review process. Among them, some tools, such as DistillerSR and EPPI-Reviewer, offered the widest range of features. We found the strongest evidence for the use of AI in screening studies by title and abstract. In this task, AI systems help researchers quickly decide which studies are likely to be relevant. Several tools reported high sensitivity, meaning they were able to correctly identify at least 95% of relevant studies in some settings. However, for other stages of evidence synthesis, such as assessing study quality or combining results, there is still limited evidence on how well AI performs. While AI shows promise, its use beyond screening is not yet fully supported by strong research. What do the findings mean: Overall, the evidence we found suggests that AI is ready to assist with some parts of evidence synthesis, especially the early screening stage. Using AI in this way could make reviews faster while still maintaining quality. This may also help support “living” reviews, which are updated regularly as new evidence becomes available. More research is needed to understand how AI can reliably support the full evidence synthesis process.