共找到 20 条结果
[This corrects the article DOI: 10.1093/jamiaopen/ooaf134.].
This study aims to compare the effectiveness of 2 ambient AI scribe technologies in reducing physician burnout, improving workflow satisfaction, and enhancing documentation efficiency through a randomized crossover trial. An open-label randomized crossover trial involving 160 outpatient clinicians was conducted at a tertiary academic medical center. Volunteers were randomized to 2 groups of 80 with 2 crossover periods. We assessed workflow satisfaction (1-7 scale), burnout (Copenhagen Burnout Index), and efficiency metrics (eg, electronic health record time outside scheduled hours, documentation time, etc.). Data was analyzed using Wilcoxon signed-rank tests and generalized linear mixed models. Surveys from 136 respondents were analyzed. Clinicians reported greater improvements in satisfaction with product B (2.51 points on a 7-point scale) compared to product A (1.91 points; mean difference: 0.60, 95% CI: 0.32-0.90). Both tools reduced personal and work burnout scores, but differences between tools were not meaningful. Product B demonstrated greater reductions in average minutes-in-notes per day compared to product A (B - A = -3.19 minutes; 95% CI -4.87 to -1.50). No meaningful differences were observed in pajama time or patient-related burnout. Both tools improved workflow satisfaction and reduced burnout, with product B showing superior performance in satisfaction and documentation time. However, efficiency metrics like pajama time were largely unaffected, potentially due to participant selection bias and the study period's timing. Product B yielded greater satisfaction and time savings compared to product A, though both tools effectively reduced physician burnout and improved workflow satisfaction.
Rare-earth high-entropy oxides (RE-HEOs) represent a distinct class of entropy-stabilized ceramics in which multiple lanthanide cations occupy a common crystallographic sublattice, generating strong chemical disorder, lattice distortion, and complex defect landscapes. Unlike transition-metal-based high-entropy oxides, RE-HEOs are governed by localized 4f electronic states, weak crystal-field coupling, and variable redox chemistry, leading to emergent structural, electronic, magnetic, and optical phenomena that challenge conventional solid-state descriptions. This review provides a physics-oriented analysis of RE-HEOs, focusing on the thermodynamic foundations of configurational entropy stabilization, the interplay between enthalpy, entropy, and kinetic trapping, and the consequences of severe chemical disorder for crystal structure and phase stability. We review how lattice distortion, oxygen vacancy disorder, and cation randomness modify phonon spectra, ionic transport pathways, and electronic structures, with particular emphasis on the role of localized 4f states, defect-induced in-gap levels, and disorder-broadened excitation spectra. Spectroscopic manifestations of disorder including crystal-field relaxation, line broadening, lifetime modification, and energy transfer processes are discussed within a unified framework linking local symmetry breaking to macroscopic response. We further discuss the optoelectronic properties of RE-HEOs, including photoluminescence from intra-4f transitions, upconversion mechanisms, and disorder-induced modifications of radiative lifetimes and quantum efficiency. The application landscape spans both energy conversion (electrocatalysis, solid oxide fuel cells, thermal barrier coatings) and optoelectronic technologies (phosphors, scintillators, optical thermometry, and anti-counterfeiting). Likewise, we assess theoretical and computational approaches, including density functional theory with strong correlation corrections, statistical thermodynamics, and emerging machine-learning models, highlighting their ability and current limitations in capturing disorder-driven physics in multi-component oxides. Finally, we identify open questions central to condensed-matter physics, including the nature of entropy-stabilized metastability, the limits of band theoretical descriptions in highly disordered 4f systems, and the role of configurational entropy in tuning electron-phonon and defect interactions. By consolidating experimental and theoretical insights, this review establishes RE-HEOs as a platform for exploring disorder-dominated solid-state physics beyond conventional crystalline oxides.
Echocardiography and cardiac catheterization reports capture important clinical assessment information of cardiac function and disease severity. This study explores using open-source transformer-based language models (LMs) that are run locally within an institutional environment as a privacy-preserving alternative to external API-based large LM to systematically extract clinical data from unstructured echocardiography and cardiac catheterization reports, aiming to improve data accessibility for research and patient care. Two transformer-based LMs, BioclinicalBERT and BART-Large-CNN, were fine-tuned in a secure local environment using a question-answering approach. The dataset included 3286 echocardiography and 1884 cardiac catheterization reports from Kaiser Permanente Southern California's electronic health records, annotated for 25 and 47 predefined categories, respectively. Three hundred reports from each type were randomly selected and used for validation, with the remainder for training. Model performance was assessed using accuracy, precision, recall, and F1-score at 2 probability thresholds. The effect of training set size on model performance was also evaluated. Both models achieved consistent and high accuracy, precision, and recall (all >90%) across the 5 seed runs for both report types. For echocardiography, BioclinicalBERT reached mean accuracy of 95.7%, precision of 97.6%, recall of 97.4%, and F1-score of 0.98 at the probability threshold of 0.1; BART-Large-CNN had similar results. For cardiac catheterization, BART-Large-CNN slightly outperformed BioclinicalBERT with mean accuracy 94.9% vs 94.3%; precision 96.7% vs 96.3%; recall 96.1% vs 95.7%, and F1-score 0.96 vs 0.96 at the probability threshold of 0.1. Most individual categories showed strong performance, though a few (eg, prosthetic mitral valve, right atrial pressure) had lower scores. Performance improved with more training data, but plateauing around 1000 reports. Fine-tuned transformer-based LMs can effectively extract structured data from unstructured cardiac reports, supporting automated information extraction to enhance research and clinical applications.
Endothelial nitric oxide synthase (eNOS) produces nitric oxide (NO), a key molecule for maintaining vascular health. While phosphorylation is a well-established regulatory mechanism of eNOS activity, the functional contribution of conserved lysine residues to electron transfer and catalytic coupling remains less clearly defined. In this study, we examined two conserved lysines in eNOS, Lys609 located within the autoinhibitory (AI) region and Lys733 positioned within the FMN-FNR hinge, by substituting them with arginine to preserve positive charge while altering side-chain geometry. Biochemical and spectroscopic analysis revealed that both substitutions significantly impaired enzyme function. Cytochrome c reductase activity was reduced by 3-6-fold, and NO synthesis decreased by approximately 37% for K609R and 25% for K733R relative to wild-type (WT) eNOS. Elevated NADPH/NO ratios indicated impaired catalytic coupling and increased diversion of electrons away from productive NO synthesis. Flavin fluorescence and auto-oxidation measurements showed that both mutations favored a closed, FMN-shielded conformation and reduced the Ca2+/calmodulin-induced transition to the open, catalytically competent state. Structural analyses and Molecular Dynamics simulations show that substitutions at Lys609 and Lys733 alter FMN-domain dynamics through distinct mechanisms. K609R induces increased flexibility and global expansion of the reductase domain, and K733R restricts hinge motion, maintaining overall compactness. Despite these defects, ferricyanide reductase activity was unchanged, showing that FAD-mediated hydride transfer remains unaffected. Electron flux through the heme correlated strongly with NO production, identifying heme-directed electron transfer as the principal step affected. Together, these findings suggest Lys609 and Lys733 as regulators of eNOS conformational dynamics, interdomain electron transfer, and catalytic efficiency.
Stigmatizing language in clinical documentation can contribute to healthcare disparities and affect patient-provider relationships. Given their strong capacity for contextual language understanding, large language models (LLMs) offer potential for detecting and reducing such language. This study evaluates the accuracy of LLMs in detecting stigmatizing language, focusing on model size, temperature settings, and the inclusion of examples. We evaluated multiple configurations of 2 local Llama-based large language models, Llama 3.2 (3B) and Llama 3.1 (8B) with varying temperature (0.25, 0.5, 0.75) and the inclusion of exemple prompts. The models were evaluated on 3643 de-identified clinical notes obtained from a tertiary care teaching hospital. Performance was assessed using accuracy, True Positive Rate (TPR), and True Negative Rate (TNR), with human annotator performance used as a benchmark. The 8B model with a temperature of 0.25 and examples achieved the highest overall accuracy (70.2%), with the best TPR (94.1%), but the lowest TNR (47.4%). The 3B model without examples achieved the highest TNR (99.7%) but a very low TPR (2%). The inclusion of examples improved model accuracy across all configurations, while temperature settings had a variable impact, with smaller models benefiting from higher temperatures and larger models performing better at lower temperatures. ED provider notes showed higher accuracy (69.4%) and the plan of care was the lowest (55.8%). Model size, temperature, and the inclusion of examples play a critical role in optimizing open-source LLM performance. Tailoring these parameters to note types enhances effectiveness. Further research should refine these models for broader clinical application and assess their potential to reduce bias in healthcare documentation.
Medical coding structures health-care data for research, quality monitoring, and policy. This study assesses the potential of large language models (LLMs) to assign International Classification of Primary Care, 2nd edition (ICPC-2) codes using the output of a domain-specific search engine. A dataset of 437 Brazilian Portuguese clinical expressions, each annotated with ICPC-2 codes, was used. A semantic search engine (OpenAI's text-embedding-3-large) retrieved candidates from 73 563 labeled concepts. Thirty-three LLMs were prompted with each query and retrieved results to select the best-matching ICPC-2 code. Performance was evaluated using F1-score, along with token usage, cost, response time, and format adherence. Twenty-eight models achieved F1-score>0.8; 10 exceeded 0.85. Top performers included gpt-4.5-preview, o3, and gemini-2.5-pro. Retriever optimization can improve performance by up to 4 points. Most models returned valid codes in the expected format, with reduced hallucinations. Smaller models (<3B parameters) struggled with formatting and input length. Large language models show strong potential for automating ICPC-2 coding, even without fine-tuning. This work offers a benchmark and highlights challenges, but findings are limited by dataset scope and setup. Broader, multilingual, end-to-end evaluations are needed for clinical validation.
Sharing behavioral health and wearable data poses privacy challenges, as traditional de-identification remains vulnerable to re-identification. Differential privacy (DP) provides mathematical guarantees through a tunable privacy budget, ϵ . This study evaluates the feasibility of generating and releasing DP synthetic behavioral health data with high analytical utility, identifying practical ϵ values for public data sharing. We analyzed physiological data from wearable devices and self-reported data from Phase 1 of the Lived Experiences Measured Using Rings Study (LEMURS), which tracked sleep, stress, and well-being among first-year college students. Three DP synthetic data generators: AIM, MST, and PATECTGAN, were evaluated across privacy budgets ranging from ϵ = 1 to 100. Utility was assessed using L1/L2 errors, correlation, regression, UMAP, and assessed vulnerability via privacy attacks. AIM outperformed MST and PATECTGAN in preserving both statistical and analytical properties of the original data. For the Survey dataset, the lowest marginal errors occurred at ϵ = 5 and 10. Correlation, regression, and UMAP analyses confirmed that AIM-generated data closely replicated original relationships at moderate ϵ values. Choice of privacy budget is still an open question, and it is task-agnostic and dataset-specific. Moderate privacy budgets ( 5 ≤ ϵ ≤ 10 ) maintained key associations between physiological and psychological measures while ensuring privacy. AIM's workload-aware design effectively allocated noise toward relevant features, enhancing performance. A privacy budget of ϵ = 5 offers a practical balance between data utility and participant privacy for LEMURS behavioral health data sharing.
This study compares multiple LLMs, including ChatGPT, DeepSeek, and Llama, to generate meaningful, audience-adapted labels for the existing latent classes among patients with chronic low back pain (cLBP). Phenotypes were derived from baseline data from two cohorts within the NIH HEAL BACPAC consortium: BACKHOME, a large nationwide e-cohort (train set: N = 3025), and COMEBACK, a deep phenotyping cohort (test set: N = 450). The analysis included pain characteristics, psychosocial factors, lifestyle habits, and social determinants of health. ChatGPT-4o (OpenAI), DeepSeek-R1, and Llama 3.3 (Meta) were applied to generate class labels for each combination of audience (clinician, patient, and caregiver), tone (formal, empathetic, and informal), and technicality (high, medium, and low). Latent Class Model (LCM) identified four distinct behavioral phenotypes in patients with cLBP: High Distress and Maladaptive Behaviors, Resilient and Adaptive Coping, Intermediate Maladaptive Patterns, and Emotionally Regulated with High Pain Burden. Previously validated by domain experts, these profiles served as the basis for automated labeling using three LLMs (ChatGPT-4o, DeepSeek-R1, and Llama 3.3). Using different tones and complexity levels, each model produced class labels specific to clinicians, patients, and caregivers. The generated class names for all LLMs closely matched expert-defined traits like emotional regulation, resilience, and high distress, indicating strong conceptual alignment and the capacity of LLMs to generate precise, audience-specific labels for intricate behavioral and psychological profiles. These results highlight the possibility of integrating LLM-driven labeling into research and clinical practice, helping to achieve more transparent knowledge translation, improved decision-making, and personalized care.
Global immunization efforts still face major inequities and declining vaccine confidence, leaving millions of children in low- and middle-income countries unvaccinated or under-vaccinated. This article aims to discuss "digital vaccines," including SMS reminders, mobile apps, electronic immunization registries, gamification, and virtual reality education, as practical complements to routine immunization services. Using an organizing framework focused on access, equity, and trust, we highlight how digital tools can reduce missed appointments, strengthen follow-up for zero-dose children, improve data quality for planning, and support transparent and culturally responsive communication to counter misinformation. We also outline the barriers that limit equitable impact, including digital divides, gender gaps in phone access, fragmented information systems, limited financing, and concerns about data governance. Many children in poorer countries still do not get the vaccines they need. Some families live too far from clinics. Others do not trust vaccines or the health system. This article looks at how digital tools can help more children get vaccinated. These tools include text message reminders, phone apps, online health records, digital games, and virtual reality lessons. Text reminders help parents remember vaccine dates. Online records help health workers find children who missed their vaccines. Digital games teach people why vaccines are safe. These tools can also help planners know how many vaccines are needed and where to send them. They can share clear, respectful health messages and fight false claims about vaccines. But not everyone can use these tools. Some people do not have smartphones or internet access. Women, who often care for children, may not have their own phones. There are also worries about keeping personal data safe and paying for these systems. We propose implementation principles that emphasize inclusive design, interoperability, privacy safeguards, and hybrid online and offline delivery models. We suggest that digital tools should be easy to use for all, keep private data safe, and work well with other health systems. Where there is no internet, non-digital options should also be offered. With the right support, these tools can help make sure all children get their vaccines.
Ohio has been severely impacted by the opioid crisis, with opioid overdose (OD) death rates exceeding national averages. Accurate OD death prediction supports proactive prevention and treatment allocation. Existing methods often focus on ZIP Code Tabulation Area (ZCTA)-level prediction for small-area resource allocation; however, performance at this resolution is poor due to substantial fluctuations in OD death counts, which introduce noise. This raises a critical methodological question: what is the optimal population threshold for OD death prediction that balances predictive accuracy with geographic resolution? We perform a theoretical analysis of variance and error bounds to establish the minimal population required for robust prediction. Building on this analysis, we propose an Area-specific Autoencoder Spatiotemporal Graph Neural Network (AAE-STGNN) framework for OD death count prediction using urine drug test (UDT) data as dynamic features and Social Determinants of Health (SDoH) as static features. The framework consists of two key components: (1) an Area-specific Autoencoder (AAE), which learns latent spatial representations while incorporating the minimal population threshold, and (2) a Spatiotemporal Graph Neural Network (STGNN), which models geographic adjacency between areas and dynamic features across time. Empirical evaluations demonstrate that AAE-STGNN outperforms state-of-the-art (SOTA) approaches, achieving improved accuracy and robustness. We also provide the OD death count trend estimation to support public health decision-making. These findings underscore the importance of selecting an optimal spatial granularity and leveraging spatiotemporal modeling techniques for data-driven public health surveillance and targeted intervention in the opioid crisis.
We report on using electronic health records (EHRs) and other health information technology (IT) (eg, REDCap, Excel, and population-health tools) for tracking patients and managing interventions to improve colorectal screening (CRC) among primary care practices who participated in the National Cancer Institute's Accelerating Colorectal Cancer Screening and Follow-up through Implementation Science (ACCSIS) program. We conducted semi-structured, recorded interviews with staff from 7 ACCSIS Research Projects (RPs). Using the interview notes, we conducted content analysis to report on the characteristics of the EHR systems and health IT, and thematic analysis to identify key concepts related to the ability to capture and monitor data for CRC screening. RPs used different data capture models to support EHRs and health IT: (1) centralized data capture models from projects or third-party services; or (2) direct data capture models, relying on features and functions within commercial EHRs. Respondents reported challenges to using EHRs and health IT, including generating patient reports to track interventions, working across EHR and research platforms because of lack of interoperability, and training for clinic staff on EHR and research platforms. RPs would benefit from more streamlined data capture and reporting for managing CRC screening in primary care. Efforts reportedly fell onto staff who could have benefited from training around data handling and EHR-specific navigation. RPs experienced challenges in leveraging data capture models for EHR and health IT data management. Our research calls for technical capabilities that promote more efficient data capture and reporting, as well as greater capacity building among clinic staff.
Traditional clinical trial enrollment relies on manual screening and coordinator-led recruitment, creating scalability barriers in high-volume perioperative environments. This study evaluated whether a fully automated, electronic health record (EHR)-integrated clinical decision support (CDS) system could identify eligible patients and engage clinicians in real time without manual screening or dedicated research staff. In this prospective implementation study, predefined respiratory-risk criteria were computed within the UCLA Perioperative Data Warehouse and transmitted to the EHR via Healthcare Level Seven interfaces. Patients meeting inclusion criteria automatically triggered Best Practice Advisories (BPAs) recommending an intervention. Outcomes included system accuracy in eligibility identification, provider adherence to BPA recommendations, and technical performance metrics. The automated system processed 10 592 eligible patients and achieved 51.2% provider adherence (5424 patients) to CDS prompts without coordinator involvement. BPA allocation accuracy was 69.7% among patients recovering in the post-anesthesia care unit and 59.4% when including unanticipated ICU transfers. Adherence varied significantly by care team composition, with full teams (attending + CRNA + resident) achieving 57.4% adherence compared with 42.2% for solo attendings. Workflow factors were stronger predictors of adherence than patient clinical characteristics, indicating minimal selection bias. Fully automated, EHR-integrated CDS can enable large-scale, workflow-embedded enrollment into implementation-focused studies. While not a substitute for research designs requiring consent or randomization, this framework demonstrates a scalable approach for automated prescreening and CDS-driven prompting that reduces reliance on coordinator-dependent processes and supports real-world implementation science.
To test the impact of visual summaries of blood pressure (BP) data (eg, stoplight and gradient displays), within the context of a patient-facing digital application connected to the EHR, on patient judgments about hypertension control. Participants (N = 117; Internet sample of patients with hypertension) viewed graphs depicting BP data for fictitious patients. For each graph, participants rated perceived hypertension control, risk of heart attack and stroke, urgency, worry, and perceived understanding of health implications on a 0-100 slider bar and indicated the preferred action to take in response this BP data (eg, talk to doctor at next appointment, go to hospital immediately). Using a within-subjects design, all participants evaluated 12 graphs with data that varied in systolic BP mean (controlled or uncontrolled) and standard deviation (moderate or high) and included three different types of visual summaries: (1) control (average BP only), (2) stoplight, (3) gradient. Participants also completed the Graph Literacy-Short Form and the Electronic Health Literacy Scales (eHEALS). Measures of perceived risk of heart attack and stroke, urgency, and worry were significantly greater and perceived hypertension control was significantly lower for cases where hypertension was uncontrolled P < 0.05. However, there were no significant differences between visual summary methods on the primary outcomes. Graph literacy and electronic health literacy were globally related to judgments of hypertension control but did not interact with any of the study factors. The verbal summary, stoplight, and gradient displays performed similarly despite the addition of more precise risk information.
To develop and validate machine learning (ML) models that predict probable cause of death (CoD) using structured electronic health record (EHR) data, unstructured clinical notes, and publicly available sources. This multi-institutional retrospective study was conducted across Vanderbilt University Medical Center (VUMC) and Massachusetts General Brigham (MGB), including deceased patients with encounters between October 1, 2015, and January 1, 2021, and confirmed death records. The cohort included 13 708 patients from VUMC and 34 839 from MGB.The primary outcome was underlying CoD categorized into the top 15 National Center for Health Statistics rankable causes, with others grouped as "Other." Performance was assessed using weighted area under the receiver operating characteristic curve (AUC) and F-measure. The XGBoost model using structured EHR data alone achieved weighted AUCs of 0.86 (95% CI, 0.84-0.88) at VUMC and 0.80 (95% CI, 0.79-0.80) at MGB. Adding unstructured notes improved performance, with weighted AUCs of 0.90 (95% CI, 0.88-0.93) at VUMC and 0.92 (95% CI, 0.91-0.92) at MGB. Adding publicly available data did not further improve performance. Cross-institutional validation revealed significant performance degradation. Models integrating structured and unstructured EHR data show strong within-institution performance but limited generalizability across healthcare systems, highlighting challenges related to institutional data heterogeneity. Machine learning models combining structured and unstructured EHR data accurately predict CoD within institutions but perform poorly across sites. Health-care institutions may benefit from adopting robust processes for locally tailored models, and future research should focus on enhancing model generalizability while addressing unique institutional data environments.
This scoping review aimed to (1) map current applications of transformers and large language models (LLMs) for extracting social drivers of health (SDOH) from clinical text, (2) benchmark model performance across SDOH domains, and (3) evaluate methodological rigor to identify research gaps and inform clinical deployment. We searched PubMed, Web of Science, Embase, Scopus, and IEEE Xplore for studies applying transformers or LLMs to detect SDOH in clinical narratives. We developed a novel methodological framework integrating (1) hierarchical classification of SDOH domains and transformer/LLM architectures, (2) systematic synthesis of performance metrics, and (3) a 7-domain instrument assessing internal validity, external validity, and reporting transparency. Forty-two studies met inclusion criteria. Performance varied substantially across SDOH domains. Behavioral Factors achieved the highest median F1-score (0.87), while Health Care Access and Quality showed the lowest performance and greatest variability (median F1 = 0.59). Research concentrated in the United States (85.7%), relied predominantly on private institutional datasets (69%), and focused primarily on critical care populations (45.2%). Methodological assessment revealed critical gaps; only 29% of studies provided annotation guidelines, 24% assessed fairness across demographic groups, and 21% performed external validation. Smaller open-source transformer models show promise for democratizing SDOH detection by achieving competitive performance at lower costs while enabling secure local deployment in resource-limited settings. Advancing clinical readiness requires standardized reporting practices, diverse benchmark datasets across care settings, and systematic equity evaluation to prevent perpetuating health disparities. Transformer and LLM performance for SDOH detection varied substantially across domains, with encoder-based models excelling at structured tasks and decoder-only models at linguistically complex tasks. Critical gaps in fairness assessment, external validation, and dataset diversity restrict generalizability and readiness for widespread clinical deployment.
This study aims to understand how inpatient nurses determine and prioritize necessary documentation within the context of the Excessive Documentation Burden (ExDocBurden) in Electronic Health Records (EHRs). A phenomenological approach was used to explore inpatient nurses' lived experiences of prioritizing EHR documentation. Interpretive phenomenology guided the study design, focusing on how nurses prioritize documentation. Purposive sampling recruited 14 registered nurses (RNs) from acute and critical care settings. Data was collected through semi-structured interviews and analyzed using Colaizzi's 7-step and Smith's Interpretive Phenomenology Analysis. Five themes emerged: (1) Advocating for Quality Patient Care Environment and Patient Needs, (2) What to Document in Near-Real Time Versus What Can Wait, (3) EHR-Driven Documentation and the Erosion of Nurse Autonomy, (4) Unnecessary (Frequent and Redundant) Documentation, and (5) Fear, Frustration, and Punitive Pressure in Charting. Nurses prioritized patient care over EHR documentation and frequently encountered unnecessary and redundant documentation tasks that did not contribute to patient needs. Defensive charting practices driven by fear of litigation further compounded nurses' emotional strain. The study emphasizes the importance of empowering nurses by minimizing non-value-added documentation and enabling them to exercise their clinical judgment. Streamlining documentation processes can help alleviate the emotional and mental strain on nurses, enabling a more patient-centered approach to care. Understanding how experienced nurses prioritize documentation in the context of ExDocBurden provides valuable insights to ameliorate EHR Burden. Nurses drive quality of patient care; consequently, supporting nurse-driven documentation enhances both patient care quality and organizational needs.
Federated research networks, like Evolve to Next-Gen Accrual of patients to Clinical Trials (ENACT), aim to facilitate medical research by exchanging electronic health record (EHR) data. However, poor data quality can hinder this goal. While networks typically set guidelines and standards to address this problem, we developed an organically evolving, data-centric method using patient counts to identify data quality issues, applicable even to sites not yet in the network. We distribute high-performance patient counting scripts as part of Integrating Biology at the Bedside (i2b2), which all ENACT sites operate. They produce counts of patients associated with ENACT ontology terms for each site. At the ENACT Hub, our pipeline aggregates site-contributed counts to produce network statistics, which our self-service web application, Data Quality Explorer (DQE), ingests to help sites conduct data quality investigation relative to the network. Thirteen ENACT sites have contributed their patient counts, and currently ten sites have signed up to use DQE to analyze data quality issues. We announced a call to all ENACT sites to contribute additional patient counts. Identifying site data quality problems relative to the network is novel. Using a metric based on evolving network statistics complements rigid data quality checks. It is adaptable to any network and has low barriers of entry, with patient counting being the sole requirement. We implemented a metric for conducting data quality investigation in ENACT using patient counting and network statistics. Our end-to-end pipeline is privacy-preserving and the underlying design is generalizable.
Electronic health record (EHR) order preference lists and order sets potentially improve efficiency but have limited utility in complex primary care settings. We assessed adoption, impact on ordering efficiency, and clinician perceptions of a comprehensive set of nested order panels (xOrders) for adult primary care. In Phase 1 (gradual implementation), 404 xOrders were released (November 29, 2020-September 25, 2021). Beginning of Phase 2 (rapid implementation), 630 xOrders were released with an additional 253 xOrders added (September 26, 2021-June 24, 2023). Three outcomes captured adoption: xOrders used per week; number of clinician users per week; and percent of xOrders of all orders. Impact of xOrders on times in orders per encounter per clinician was evaluated with mixed effects interrupted time series. t-Tests evaluated differences between low, moderate, and high utilizers. A survey captured clinicians' perceptions in November 2022. xOrders were used 536 (SD, 245) times/week and by 57(15) clinicians/week in Phase 2. xOrders as a percent of all orders ranged from 0% to 31% across clinicians. Time spent in orders per encounter decreased by 14 ± 5 s (P =.01) from Phase 1 to 2 for high utilizers, decreased by 7(3) s (P=.05) for moderate utilizers, and increased by 1(3) s for low utilizers (P=.81); low and high utilizers were significantly different (P=.02). Most (77%) survey respondents agreed that xOrders improved ordering efficiency. Despite yielding time savings and positive clinician feedback, the xOrder intervention showed limited adoption and impact, suggesting the need for expanded content and increased adoption to realize larger efficiency gains.
Medical product safety surveillance efforts, whether using electronic health record (EHR) or claims data, typically rely on structured codes. Utilizing unstructured EHR data, particularly information extracted from clinical text through natural language processing (NLP), enriches information available for data mining, phenotyping, and surveillance. To assess overlapping and distinct information across structured and unstructured EHR data, we mapped both to a common vocabulary (Medical Dictionary for Regulatory Activities, MedDRA). We assess the feasibility of implementing such a mapping and explored similarities and differences at multiple levels of the concept hierarchy. We randomly sampled 15,000 encounters (5000 each from ambulatory, emergency, and inpatient encounters). For each encounter, we extracted MedDRA concepts from clinical notes using MetaMap and mapped structured ICD-10-CM diagnoses to MedDRA. We evaluated corroboration between data sources across the MedDRA hierarchy, as well as the unique information contributed by each source. We processed 119,492 clinical notes and mapped 163,254 ICD-10-CM codes to MedDRA. Most encounters (73-98%) had some overlap between MedDRA preferred terms identified from structured and unstructured data. Among MedDRA concepts found in unstructured text, 80-95% were not found in the encounter's associated ICD-10-CM coded data. While MedDRA concepts from structured data were mostly corroborated by those extracted from unstructured clinical text, the majority of MedDRA concepts recognized in each encounter were only mentioned in text. Leveraging MedDRA-encoded unstructured text can provide a more comprehensive clinical picture of patients and complement the structured data traditionally used in epidemiological and pharmacovigilance studies.