搜索结果：AMIA ... Annual Symposium proceedings. AMIA Symposium

Subject: This study aims to evaluate the effectiveness of Complementary and Integrative Health (CIH) therapies in reducing the incidence and severity of Postpartum Depression (PPD) using real-world data and target trial emulation. Methods: Using electronic health records (EHR) from a large healthcare system, we emulated target trials for CIH approaches including acupuncture, chiropractic, aromatherapy, and omega-3 fatty acids. CIH usage was identified and extracted from clinical notes using natural language processing (NLP) techniques. Logistic regression-based propensity score matching was employed to address confounding factors. The primary outcome was the incidence of PPD within 12 months postpartum, defined by diagnostic codes or antidepressant initiation. Secondary outcomes included changes in PHQ-9 scores and subgroup analyses by treatment type. Results: For the primary outcome, none of the treatments significantly reduced PPD risk intervals (CIs). However, omega-3 fatty acids and chiropractic care significantly reduced PHQ-9 scores in the treatment groups (omega-3 fatty acids: p<0.001, chiropractic care: p = 0.021), with no comparable improvements in controls. Aromatherapy showed mixed results, with reduced severe depression in the treatment group but increased severity in controls. Acupuncture had no significant effect (p > 0.05). These findings suggest that omega-3 fatty acids and chiropractic care may alleviate PPD symptoms, while the effects of aromatherapy, acupuncture and chiropractic remain inconclusive and warrant further investigation. Conclusion: This study provides approach to evaluating CIH interventions in real-world settings. These findings underscore the importance of integrating non-traditional treatment options into clinical practice to improve outcomes for individuals affected by PPD.

This study explores cancer vaccine adjuvant name recognition using Large Language Models (LLMs), specifically Generative Pretrained Transformers (GPT), Large Language Model Meta AI (Llama), and Gemma. The models were tested in zero- and few-shot learning paradigms using AdjuvareDB and Vaccine Adjuvant Compendium (VAC) datasets. Prompts were designed to extract adjuvant names and assess the impact of contextual details. Notably, Llama-3.2 3B achieved a Recall of up to 68.7% (72.5% with manual validation) on the VAC dataset with four-shots, although its Precision and F1-score were lower. In contrast, GPT-4o, with additional contextual interventions, achieved a Precision of 65.9%, Recall of 79.7%, and F1-score of 69.8% on the AdjuvareDB dataset. Gemma-2 9B also demonstrated moderate few-shot gains, peaking at 63.6% F1-score. These LLMs outperformed BioBERT, a model widely used for biomedical text mining, highlighting the potential of general-purpose LLMs for automatic vaccine adjuvant name extraction and contributing to advancements in vaccine research.

Large language models (LLMs) excel in natural language processing (NLP) but struggle with domain-specific complexities in electronic health records (EHRs). We demonstrate that retrieval-augmented generation (RAG) enhances LLMs for dietary supplement (DS) information extraction. By testing models like Llama-3 with diverse retrievers on tasks including entity recognition and usage classification, task-aligned retrieval outperforms reliance on model size or specialization. Smaller general models paired with optimized retrievers match or exceed specialized counterparts-structured retrieval aids complex tasks (e.g., triple extraction), while semantic retrieval improves classification. Results challenge assumptions that larger or domain-specific models are superior, emphasizing dynamic knowledge integration over brute-force scaling. This approach offers practical strategies for clinical NLP, enabling efficient EHR analysis without massive resources. Prioritizing retrieval strategies over model size advances tools for evidence-based healthcare, highlighting adaptability and cost-effectiveness in real-world medical applications.

Reference errors, such as citation and quotation errors, are common in scientific papers. Such errors can result in the propagation of inaccurate information, but are difficult and time-consuming to detect, posing a significant threat to the integrity of scientific literature. To support automatic detection of reference errors, we evaluated the ability of large language models in OpenAI's GPT family to detect quotation errors. Specifically, we prepared an expert-annotated, general-domain dataset of statement-reference pairs from journal articles, one-third of which is in biomedicine. Large language models were evaluated in different settings with varying amounts of reference information provided by retrieval augmentation. Results showed that large language models are able to detect erroneous citations with limited context and without fine-tuning. This study contributes to the growing literature that seeks to utilize artificial intelligence to assist in the writing, reviewing, and publishing of scientific papers as well as grounding of language model responses.

This work introduces the Sequential Multiple Instance Learning (SMIL) framework, addressing the challenge of interpreting sequential, variable-length sequences of medical images with a single diagnostic label. Diverging from traditional MIL approaches that treat image sequences as unordered sets, SMIL systematically integrates the sequential nature of clinical imaging. We develop a bidirectional Transformer architecture, BiSMIL, that optimizes for both early and final prediction accuracies through a novel training procedure to balance diagnostic accuracy with operational efficiency. We evaluated BiSMIL on three medical image datasets to demonstrate that it simultaneously achieves state-of-the-art final accuracy and superior performance in early prediction accuracy, requiring 30-50% fewer images for a similar level of performance compared to existing models. Additionally, we introduce SMILU, an interpretable uncertainty metric that outperforms traditional metrics in identifying challenging instances.

In vivo confocal microscopy (IVCM) assesses corneal innervation in the sub-basal nerve plexus but is typically quantified manually from a single z-scan, limiting biomarker extrapolation for diagnosing limbal stem cell deficiency (LSCD). We developed an automated 3D reconstruction method of IVCM image volumes to improve sub-basal nerve density quantification. Our dataset comprised 99 IVCM stacks from 63 LSCD eyes (51 patients) and 23 stacks from 15 normal eyes. We designed an image registration algorithm combining phase correlation and homography transformation, which achieved a pairwise image correlation of 0.69 and mutual information of 0.60, significantly outperforming manual registration (0.60 and 0.43, respectively; p<0.001). Validation on an independent dataset of 325 volume scans from 24 eyes of 12 unilateral, severe LSCD patients yielded a correlation of 0.75 and MI of 0.76. This method enhances sequential IVCM scan alignment and supports more accurate, reproducible 3D evaluation of LSC biomarkers.

Ensuring the completeness of IS-A relations in SNOMED CT is crucial for maintaining its accuracy in clinical applications. In this study, we propose a hybrid approach leveraging non-lattice subgraphs and pre-trained language models (PLMs) to identify missing IS-A relations in SNOMED CT. We fine-tuned four BERT-based models: BERT, DistillBERT, DeBERTa, and BioClinicalBERT, and four generative large language models (LLMs): BioMistral, Llama3, Gemma2, and Phi-4. Missing IS-A relations were identified through consensus predictions by all eight models. De-BERTa achieved the best performance (precision: 0.96, recall: 0.97, F1-score: 0.965) for IS-A relation prediction. Our approach identified 678 potential missing IS-A relations in SNOMED CT (March 2023 US Edition), of which 100 randomly selected cases were manually reviewed by a domain expert, confirming 93 as valid (93% precision). These results demonstrate the effectiveness of fine-tuned PLMs in detecting missing IS-A relations within non-lattice subgraphs, offering a promising avenue for improving SNOMED CT's quality.

Reliable drug safety reference databases are essential for pharmacovigilance, yet existing resources like SIDER are outdated and static. We introduce PVLens, an automated system that extracts labeled safety information from FDA Structured Product Labels (SPLs) and maps terms to MedDRA. PVLens integrates automation with expert oversight through a web-based review tool. In validation against 97 drug labels, PVLens achieved an F1 score of 0.882, with high recall (0.983) and moderate precision (0.799). By offering a scalable, more accurate and continuously updated alternative to SIDER, PVLens enhances real-time pharamcovigilance with improved accuracy and contemporaneous insights.

Artificial intelligence and machine learning are transforming healthcare by improving clinical risk predictions and diagnostic precision. However, their performance can be compromised by data drifts due to changes in patient populations and evolving clinical practices. This study investigated performance drift in models predicting Acute Kidney Injury (AKI) using electronic health records from 249,749 inpatient encounters over ten years, analyzing performance across both the overall population and nine subgroups with unique health profiles. To mitigate the performance drift, we implemented two model updating strategies: an Overall Population Update (OPU) and a Specific Subgroup Update (SSU). Our results demonstrated significant reductions in drift, with OPU increasing the average area-under-the-precision-recall-curve (AUPRC) by 0.14 in the overall population and 0.11 across subgroups, and SSU improving the average AUPRC by 0.10 among subgroups. These findings highlight the importance of continuous model surveillance and adaptive updates to maintain reliable predictive performance in dynamic clinical environments.

Cardiovascular event adjudication is essential in clinical trials but relies on manual chart review that is slow, variable, and expensive. We present a two-stage framework that automates adjudication of cardiovascular deaths using large language models (LLMs). First, a few-shot LLM extracts structured evidence (event, span, negation, date) from unstructured clinical documents. Second, a Tree-of-Thoughts adjudicator aligns its reasoning with clinical endpoint committee (CEC) guidelines to classify deaths as cardiovascular or non-cardiovascular and produce an auditable rationale. On Lilly clinical-trial data, extraction achieved precision 0.96, recall 0.71 (F1 0.82), and adjudication attained 0.68 accuracy (GPT-4 ToT), outperforming a summarizer-plus-adjudicator baseline. We introduce CLEART, a rubric-based automated score that quantifies rationale quality across clarity, consistency, detail, guideline adherence, relevance, and timeline accuracy (overall 0.67), highlighting temporal reasoning and relevance as key areas for improvement. This approach can reduce adjudication time and variability while increasing transparency.

Accurately identifying disease diagnoses from electronic health records (EHRs) is crucial for clinical/biomedical research; however, this is challenging when diagnoses are complex and require data from several sources, e.g., multiple myeloma (MM) and its precursor condition, MGUS. Leveraging the national Veterans Health Administration EHRs, we developed and validated a large language model (LLM)-based pipeline that utilizes only clinical notes from randomly selected patients identified via ICD codes for MGUS/MM. Among the evaluated LLMs and alternative approaches, Llama-3-8B-based pipeline with prompt engineering achieved the best performance. This pipeline not only saved the preprocessing steps and shortened the overall processing time but also outperformed rule-based or machine learning-based methods for identifying MGUS and achieved comparable performance for MM, solely relying on clinical notes. Our work demonstrates that the developed LLM-based pipeline can efficiently and effectively identify MGUS/MM diagnoses to replace manual chart abstraction and rule- or machine learning-based natural language processing methods.

Counterfactual simulation-exploring hypothetical consequences under alternative clinical scenarios-holds promise for transformative applications such as personalized medicine and in-silico trials. However, it remains challenging due to methodological limitations. Here, we show that an autoregressive generative model, trained on real-world data from over 300,000 patients and 400 million patient timeline entries, can generate clinically plausible counterfactual trajectories. As a validation task, we applied the model to patients hospitalized with COVID-19 in 2023, modifying age, serum C-reactive protein (CRP), and serum creatinine to simulate 7-day outcomes. Increased in-hospital mortality was observed in counterfactual simulations with older age, elevated CRP, and elevated serum creatinine. Remdesivir prescriptions increased in simulations with higher CRP values and decreased in those with impaired kidney function. These counterfactual trajectories reproduced known clinical patterns. These findings suggest that autoregressive generative models trained on real-world data in a self-supervised manner can establish a foundation for counterfactual clinical simulation.

This study is part of the OsteoPorotic fracTure preventION System (OPTIONS) project which aims to develop an evidence-based mobile application for older adults transitioning from skilled nursing facilities (SNFs) back to the community after lower limb fractures. The app promotes exercise, nutrition, and bone health medications to prevent future fractures. Using a Design science framework, app requirements were identified by synthesizing scientific knowledge, clinical expertise, and end-user needs. An initial mockup was developed based on these specifications and iteratively refined through design sessions incorporating end-user feedback. The final OPTIONS app features four core functions: (1) task-based self-management support, (2) personalized exercises and education, (3) motivational messaging, and (4) progress tracking allowing users to monitor their progress through visualizations. By addressing usability challenges for older adults, the app provides a personalized, engaging experience for continuous health management.

Identifying immune checkpoint inhibitor (ICI) studies in genomic repositories like Gene Expression Omnibus (GEO) is vital for cancer research yet remains challenging due to semantic ambiguity, extreme class imbalance, and limited labeled data in low-resource settings. We present ProtoBERT-LoRA, a hybrid framework that combines PubMedBERT with prototypical networks and Low-Rank Adaptation (LoRA) for efficient fine-tuning. The model enforces class-separable embeddings via episodic prototype training while preserving biomedical domain knowledge. Our dataset was divided as: Training (20 positive, 20 negative), Prototype Set (10 positive, 10 negative), Validation (20 positive, 200 negative), and Test (71 positive, 765 negative). Evaluated on test dataset, ProtoBERT-LoRA achieved F1-score of 0.624 (precision: 0.481, recall: 0.887), outperforming the rule-based system, machine learning baselines and finetuned PubMedBERT. Application to 44,287 unlabeled studies reduced manual review efforts by 82%. Ablation studies confirmed that combining prototypes with LoRA improved performance by 29% over stand-alone LoRA.

Clinical calculators are widely used, and large language models (LLMs) make it possible to engage them using natural language. We demonstrate a purpose-built chatbot that leverages (1) software implementations of verifiable clinical calculators via LLM tools, and (2) metadata about these calculators via retrieval augmented generation (RAG). We compare its accuracy to an unassisted LLM on four natural language conversation workloads. Our chatbot achieves 100% accuracy on queries interrogating calculator metadata content and shows a significant increase in clinical calculation accuracy vs. the off-the-shelf LLM when prompted with complete sentences (86.4% vs. 61.8%) or with medical shorthand (79.2% vs. 62.0%). It eliminates calculation errors when prompted with complete sentences (0% vs. 16.8%) and greatly reduces them when prompted with medical shorthand (2.4% vs. 18%). While our chatbot is not yet ready for clinical use, these results show progress in minimizing incorrect calculation results.

Recent advances in vision-language models (VLMs) have enabled automatic radiology report generation, yet current evaluation methods remain limited to general-purpose NLP metrics or coarse classification-based clinical scores. In this study, we propose a clinically informed evaluation framework for VLM-generated radiology reports that goes beyond traditional performance measures. We define a taxonomy of 12 radiology-specific error types, each annotated with clinical risk levels (low, medium, high) in collaboration with physicians. Using this framework, we conduct a comprehensive error analysis of three representative VLMs, i.e., DeepSeek VL2, CXR-LLaVA, and CheXagent, on 685 gold-standard, expert-annotated MIMIC-CXR cases. We further introduce a risk-aware evaluation metric, the Clinical Risk-weighted Error Score for Text-generation (CREST), to quantify safety impact. Our findings reveal critical model vulnerabilities, common error patterns, and condition-specific risk profiles, offering actionable insights for model development and deployment. This work establishes a safety-centric foundation for evaluating and improving medical report generation models. The source code of our evaluation framework, including CREST computation and error taxonomy analysis, is available at https://github.com/guanharry/VLM-CREST.

Implicit bias impacts the quality of patient-clinician interactions, influencing patient outcomes and trust in healthcare. Most interventions to mitigate bias rely solely on expensive human assessments, rather than leveraging AI technology with clinician input. To explore clinician-envisioned interventions, we conducted interviews with 16 primary care clinicians using provocative design methods to facilitate innovative ideation on using technology to address implicit bias. Themes from interviews included: patient communication monitoring, clinician self-awareness, systemic solutions, optimizing workflow, clinician education, and patient feedback. These envisioned interventions provide design considerations for technology-based implicit bias feedback tools. The broad range of innovative solutions generated by clinicians at various career stages reflects the utility of provocative design methods in unlocking creative thinking among a population that is not often encouraged to think beyond structured real-world constraints.

Despite significant advancements in Generative Artificial Intelligence (GenAI), practical adoption in healthcare, particularly patient safety, remains challenging due to concerns regarding data privacy, model transparency, clinical relevance and user engagement. We present LLaMPS (Large Language Model for Patient Safety), a locally deployed GenAI platform designed to enhance patient safety event management and reporting. LLaMPS integrates automated incident classification, harm-level prediction, intelligent search, and an interactive chatbot. The system employs a Retrieval-Augmented Generation (RAG) approach, leveraging secure, institutionally hosted large language models (LLMs) and a vector database to ensure data privacy and regulatory compliance. Developed iteratively with direct input from clinicians and patient safety experts, LLaMPS demonstrates high classification accuracy and improved user satisfaction, underscoring the potential of locally controlled AI solutions to enhance patient safety workflows.

Researchers have developed pharmacogenomics datasets for various purposes, such as biomarker identification, yet drug response prediction models often underperform due to dataset inconsistencies. These variations arise from inter-tumoral heterogeneity, experimental conditions, and cell subtype complexity, limiting model generalizability. To address this, we propose a computational model based on Aggregated Learning (AL) to enhance drug response prediction by learning from inconsistencies across multiple datasets. Our model minimizes discrepancies by training on overlapping inconsistent data points from three pharmacogenomic datasets-CCLE, GDSC2, and gCSI. Compared to four baseline methods-Selecting Better (SB), Result Average (RA), Combining Data (CD), and Model Average (MA)-our approach achieved superior performance with lower Mean Absolute Error (MAE) scores: 0.090 (CCLE-GDSC), 0.096 (CCLE-gCSI), and 0.081 (GDSC-gCSI). These results demonstrate that addressing inconsistencies enhances prediction accuracy and generalizability, making our model a promising solution for robust drug response predictions.