Artificial intelligence software to help detect fractures on X-rays in urgent care: An Early Value Assessment.

内容摘要

Artificial intelligence algorithms have been developed to support clinicians in diagnosing fractures, with the intention to improve the diagnostic accuracy of clinicians reviewing X-rays. The purpose of this rapid early value assessment was to identify the existing evidence base for the technology and to assess whether there was a prima facie case for the technology to represent positive outcomes for patients and a value-for-money investment for people in the National Health Service. This early value assessment assessed the potential value of the use of artificial intelligence to aid clinician diagnosis of fractures in emergency care settings as compared to clinician-diagnosis alone. A rapid evidence review was conducted followed by 'light touch' early economic modelling to explore whether a plausible case could be made for cost-effectiveness at the prices charged by the companies. Evidence searches were conducted in June and July 2024 to identify clinical, diagnostic and service outcomes associated with the technology. A simple decision model incorporating prevalence, sensitivity, specificity and cost per scan for each of the technologies was developed to evaluate plausible cost-effectiveness for detecting ankle and foot, wrist and hand, and hip fractures, selected based on the availability of evidence and their downstream costs and consequences. Sixteen studies identified evaluated the diagnostic accuracy of the technology. None of the included studies were conducted in the United Kingdom and all were associated with limitations. While the studies were not considered to be able to provide reliable estimates of diagnostic accuracy, there was a trend for the technology to improve sensitivity for detecting fractures. The technology had no discernible impact on the rate of false-positive diagnoses. Overall, most of the evaluated technologies were associated with a positive incremental net health benefit at willingness-to-pay thresholds of £20,000 and £30,000 per quality-adjusted life-year gained. Due to data limitations, it was not possible to compare technologies against each other. The results were mostly robust to scenario analyses. The evidence base for the technology is currently limited to studies evaluating diagnostic accuracy and it is unclear whether increases in fracture detection would translate into meaningful benefits for patients and services. While there are some fractures that, if missed, can result in significant harm to patients, it is plausible that the technology would improve diagnosis of more subtle fractures that may not require a change in management. Use of the technology would not eradicate the risk of missed fractures, meaning that health services would need to continue to take precautions to avoid the risk of a missed fracture in clinical practice. A simple decision tree analysis suggested that the technology was plausibly cost-effective at conventional National Institute for Health and Care Excellence thresholds. There are significant limitations in the available evidence leading to uncertainties about the diagnostic accuracy of the technology within NHS settings. Due to the pragmatic nature of the early value assessment and the available evidence base, the economic analysis included many gross assumptions and was unable to produce a definitive estimate of cost-effectiveness. The appraisal resulted in a number of research recommendations for evaluating the technology further. More detailed modelling in a full formal diagnostic assessment review is required to consider the longer-term costs and consequences of false negatives and positives, and how they are likely to impact the estimates of cost-effectiveness. This study is registered as PROSPERO CRD42024574393. This award was funded by the National Institute for Health and Care Research (NIHR) Evidence Synthesis programme (NIHR award ref: NIHR136024) and is published in full in Health Technology Assessment; Vol. 30, No. 33. See the NIHR Funding and Awards website for further award information. X-rays are the usual method for diagnosing broken bones (fractures) in urgent care settings, including accident and emergency, urgent treatment centres, and minor injuries units. Artificial intelligence technologies have been developed to help identify fractures on X-rays. Peninsula Technology Assessment Group was commissioned to conduct an early value assessment to explore whether licensed artificial intelligence technologies could support fracture detection in urgent care while further evidence is developed. A search was conducted to identify all relevant evidence evaluating artificial intelligence to assist fracture detection using X-rays, including published evidence and confidential data from artificial intelligence companies. Sixteen studies evaluated how accurate artificial intelligence was to assist diagnosis and five studies reported how artificial intelligence changed the time needed to interpret X-rays. None of the studies were based within the artificial intelligence and most involved staff different to those who would normally read X-rays in the NHS. This meant that it was not possible to reliably estimate how accurate artificial intelligence would be in UK urgent care. There were early indications that artificial intelligence could help reduce missed fractures, but further research is needed to confirm this. No studies evaluated how artificial intelligence affected patient outcomes (such as health and mobility) or its impact on NHS resources (e.g. repeat appointments or overall costs). To assess whether artificial intelligence could be cost-effective for the NHS, we developed an economic model that combined information on artificial intelligence accuracy, costs, and what happens to patients once their fracture is correctly – and incorrectly – diagnosed (including quality of life and NHS costs). Our findings suggest most artificial intelligence technologies are reasonably priced for the estimated benefits. We explored uncertainties related to the data and assumptions in our analysis and found that our conclusions did not change most of the time.