Neuroscience studies entail the generation of massive collections of heterogeneous data (e.g. demographics, clinical records, medical images). Integration and analysis of such data in research centers is pivotal for elucidating disease mechanisms and improving clinical outcomes. However, data collection in clinics often relies on non-standardized methods, such as paper-based documentation. Moreover, diverse data types are collected in different departments hindering efficient data organization, secure sharing and compliance to the FAIR (Findable, Accessible, Interoperable, Reusable) principles. Henceforth, in this manuscript we present a specialized data management system designed to enhance research workflows in Deep Brain Stimulation (DBS), a state-of-the-art neurosurgical procedure employed to treat symptoms of movement and psychiatric disorders. The system leverages REDCap to promote accurate data capture in hospital settings and secure sharing with research institutes, Brain Imaging Data Structure (BIDS) as image storing standard and a DBS-specific SQLite database as comprehensive data store and unified interface to all data types. A self-developed Python tool automates the data flow between these three components, ensuring their full interoperability. The proposed framework has already been successfully employed for capturing and analyzing data of 107 patients from 2 medical institutions. It effectively addresses the challenges of managing, sharing and retrieving diverse data types, fostering advancements in data quality, organization, analysis, and collaboration among medical and research institutions.
Functional enrichment analysis is essential for understanding the biological significance of differentially expressed genes. Commonly used tools such as g:Profiler, DAVID, and GOrilla are effective when applied to well-annotated model organisms. However, for non-model organisms, particularly for bacteria and other microorganisms, curated functional annotations are often scarce. In such cases, researchers often rely on homology-based approaches, using tools like BLAST to transfer annotations from closely related species. Although this strategy can yield some insights, it often introduces annotation errors and overlooks unique species-specific functions. To address this limitation, we present a user-friendly and adaptable method for creating custom annotation R packages using genomic data retrieved from NCBI. These packages can be directly imported as libraries into the R environment and are compatible with the clusterProfiler package, enabling effective gene ontology and pathway enrichment analysis. We demonstrate this approach by constructing an R annotation package for Mycobacterium tuberculosis H37Rv, as an example. The annotation package is then utilized to analyze differentially expressed genes from a subset of RNA-seq dataset (GSE292409), which investigates the transcriptional response of M. tuberculosis H37Rv to rifampicin treatment. The chosen dataset includes six samples, with three serving as untreated controls and three exposed to rifampicin for 1 h. Further, enrichment analysis was performed on genes to demonstrate changes in response to the treatment. This workflow provides a reliable and scalable solution for functional enrichment analysis in organisms with limited annotation resources. It also enhances the accuracy and biological relevance of gene expression interpretation in microbial genomics research. Key features • Comprehensive SQLite database with gene information and detailed annotation of all organisms in NCBI. • Customized R annotation package built for Mycobacterium tuberculosis H37Rv by extracting species-specific records from the SQLite database using the taxonomic identifier. • Gene ontology and KEGG enrichment analysis on significantly expressed genes from the RNA-seq dataset GSE292409 by importing the customized annotation package as an R library.
Genomic variant data are useful in detecting and treating antibiotic-resistant bacteria. However, there are no bacterial genomic variant databases that catalogue the variations in the different genes across strains. In this work a Nextflow- and Docker-based end-to-end pipeline, BVbase, that can automate the creation of databases from raw high-throughput sequences has been created to fill this lacuna with Pseudomonas aeruginosa as a case study. Pseudomonas aeruginosa is a Gram-negative adaptable pathogen with multiple antibiotic resistances that causes various types of infections, including respiratory, urinary, and bloodstream infections. The pipeline can take multistrain genomic files, detect missense variants, and save results in a database with the help of Python and SQLite (https://github.com/bic-sastra/BVbase). Using the generated database for P. aeruginosa, a web application interface has been made using Flask and HTML that runs in a server with MySQL backend (https://bic.sastra.edu/pavardb). The web application provides supports for different types of queries to select variants by gene, geographical group, isolation country, antibiotics, and resistance phenotype. This web interface generates results as variant tables, plots, and statistics for the selected data. By enabling interactive visualizations and advanced selection, the platform supports research and clinical use through the exploration of genomic variations associated with antimicrobial resistance.
Phenome-wide association studies (PheWAS) using polygenic risk scores (PRS) offer a powerful framework for exploring the shared genetic architecture of complex traits across diverse phenotypic domains. However, no standardized, portable pipeline exists to facilitate their systematic execution and visualization in arbitrary population cohorts. We present PolyGenie, an open-source Nextflow pipeline that takes precomputed PRS and cohort phenotype data as input and performs scalable PheWAS analysis across binary and continuous outcomes. The pipeline produces regression results and percentile-based prevalence estimates, which are stored in a SQLite database and visualized through an interactive Dash web application. To demonstrate its utility and reproducibility, we provide a fully worked example using the GCAT cohort, applying 135 PRS to a broad range of clinical, molecular, and lifestyle phenotypes. PolyGenie is designed to be deployed on any cohort with minimal configuration, enabling standardized cross-trait analyses and interactive exploration of genetic risk.
Accurate real-time soil moisture monitoring is critical for optimizing water use and ensuring crop health and food security. This study aims to calibrate and integrate low-cost capacitive soil moisture sensors into a solar-powered sensor node for real-time soil moisture monitoring in a loamy sand soil. Three capacitive soil moisture sensors were calibrated in the laboratory under controlled volumetric water content conditions (0-40%) using a constrained linear regression approach. The system was tested in a limited pilot-scale in a drip-irrigated onion field at the IWAD farm, Yagaba (North-East Region, Ghana). The results showed good agreement of the sensor readings with the soil moisture obtained using the gravimetric method (R2 of 0.92-0.94, RMSE of 0.40-0.52%, and MAE of 0.35-0.39%) demonstrating the successful transfer of the calibration functions to field conditions. Soil moisture data was successfully monitored and transmitted from the nodes to a LoRa gateway via LoRaWAN (433 MHz) and from the gateway to a Raspberry Pi edge server via Wi-Fi. Data was stored both locally in SQLite on the Raspberry Pi and on the InfluxDB cloud. These results suggest that the developed system, when extensively validated under field conditions, can be used to support decision-making for data-driven IoT-based irrigation scheduling.
Liquid chromatography-mass spectrometry (LC-MS/MS) data analysis requires adaptable software solutions to meet diverse analytical needs. We present eMZed 3, a modern Python framework for flexible and interactive analysis of LC-MS/MS data. eMZed 3 enables users to develop scalable workflows tailored to their specific requirements while leveraging Python's extensive ecosystem of libraries. Building on its predecessor, eMZed 3 is now Python 3-based and includes substantial enhancements, including support for chromatogram-based LC-MS data, a new SQLite-based backend supporting optional out-of-memory processing, and rich interactive visualization tools. Compared to the previous version, eMZed 3 is now split into three packages: emzed (core functionalities), emzed-gui (interactive data visualization), and emzed-spyder (an integrated development environment). This modular architecture allows straightforward integration of the emzed core library into headless Python environments, including computational notebooks (such as Jupyter) or high-performance computing clusters. eMZed 3 incorporates well-established libraries such as OpenMS, and is suited for both targeted and untargeted metabolomics. Overall, eMZed 3 supports the efficient development of scalable and reproducible LC-MS data analysis and is accessible to both novice and advanced programmers. eMZed 3 and its documentation are freely available at https://emzed.ethz.ch, the source code is hosted at https://gitlab.com/groups/emzed3.An online-executable example workflow is available on Binder at: https://mybinder.org/v2/gl/emzed3%2Femzed-example-workflow/HEAD?_labpath=example.ipynb.
The Clinical Practice Research Datalink (CPRD) is a large and widely used resource of electronic health records from the UK, linking primary care data to hospital data, death registration data, cancer registry data, deprivation data and mental health services data. Extraction and management of CPRD data is a computationally demanding process and requires a significant amount of work, in particular when using R. The rcprd package simplifies the process of extracting and processing CPRD data in order to build datasets ready for statistical analysis. Raw CPRD data is provided in thousands of.txt files, making querying this data cumbersome and inefficient. rcprd saves the relevant information into an SQLite database stored on the hard drive which can then be queried efficiently to extract required information about individuals. rcprd follows a four-stage process: 1) Definition of a cohort, 2) Read in medical/prescription data and save into an SQLite database, 3) Query this SQLite database for specific codes and tests to create variables for each individual in the cohort, 4) Combine extracted variables into a dataset ready for statistical analysis. Functions are available to extract common variable types (e.g., history of a condition, or time until an event occurs, relative to an index date), and more general functions for database queries, allowing users to define their own variables for extraction. The entire process can be done from within R, with no knowledge of SQL required. This manuscript showcases the functionality of rcprd by running through an example using simulated CPRD Aurum data. rcprd will reduce the duplication of time and effort among those using CPRD data for research, allowing more time to be focused on other aspects of research projects.
The continuous and automated monitoring of animal activity with sensors can help to refine and improve the quality of animal experiments. The aim of this study was to develop a system for real-time automated multi-species monitoring of individual animal activity in research trials ("RAMSMART"), to test this system in terms of battery life and data completeness, and to demonstrate its added value. We used lightweight commercially available accelerometers with edge computing and uploaded self-written firmware to these accelerometers to calculate the vectorial dynamic body acceleration (VeDBA). The accelerometer is attached, for example, to an animal's ear tag. It then collects raw acceleration data (at a user-defined sampling frequency), calculates the mean VeDBA across a certain period (based on a user-defined updating interval) and creates a data packet with the VeDBA, which is then broadcasted (at a user-defined broadcasting frequency) through Bluetooth Low Energy to a nearby receiving device, e.g., a laptop. On this receiving device, data are stored in a SQLite database and regularly copied to a shared drive to enable (near) real-time visualization of results from outside animal facilities. Battery life depended on the sampling frequency, the updating interval and the broadcasting frequency. For example, when the sampling frequency was kept constant at 12.5 Hz, and the broadcasting frequency at 4 Hz, an updating interval of 1 second or 1 minute resulted in a battery life of 15.9 days (± 1.0 days) or 47.7 days (± 1.9 days), respectively. To avoid missing data, a sufficiently high broadcasting frequency was needed and the distance between sensor and receiver should be limited (preferably to < 20 m), or a multiple receiver set-up could be used. We showcased RAMSMART in three infectious disease trials in pigs, sheep and calves, connecting the VeDBA with other clinical symptoms, and demonstrating the added value of this system for early detection of deviations in activity. RAMSMART provides a low-invasive, affordable, re-usable, generalizable and plug-and-play approach for routine and long-term real-time automated activity monitoring based on acceleration data of (particularly livestock) animals in research trials. The source code and documentation of RAMSMART are openly available.
Dry eye is a common ocular surface disorder resulting from tear deficiency or excess tear evaporation. This protocol will be implemented for a study designed to determine the effectiveness of a mobile self-care application on objective clinical test results and subjective symptoms in patients with dry eye disease. The first stage will include the development of a mobile self-care application using the Python framework of Flask technology and SQLite database in the backend, and the Flutter framework of Dart for the bot technology in the front end. The second stage will be conducted using a two-arm blinded randomized clinical trial. The sample size calculation initially targeted 104 patients with dry eye disease, intended for division into intervention and control groups via an online random number generator. However, unforeseen challenges, including COVID-19 disruptions, led to randomization of 181 patients (91 intervention, 90 control).This resulted in 104 completers after a 42.5% dropout rate. Patients in both groups will receive usual medical care, but those in the intervention group will also use a mobile-based application for a period of 12 weeks. Study outcomes including the Tear Break-Up Time clinical test as the primary outcome, and the validated Persian version of the Ocular Surface Disease Index questionnaire as a secondary outcome measure in patients with dry eye disease, will be assessed at 6- and 12-weeks post-baseline. It is expected that delivering customized training via a mobile application during clinical visits would promote improvement and reduction of disease severity, as well as strengthen the physician-patient relationship. This study was developed by an interdisciplinary research team in accordance with current dry eye disease guidelines and taking into account the medical history and classification of dry eye disease. Considering the lack of financial resources for traditional self-care methods and the development of technologies based on mobile health, it seems that one can hope for the acceptance of such systems in the patients with dry eyes. This protocol is registered in the Iranian registration of clinical trial (IRCT) with the code IRCT20200721048162N1. Registered 31 August 2020, URL: https://irct.behdasht.gov.ir/trial/49779.
Assigning area-level socioeconomic measures to street addresses across historical and contemporary census years requires harmonising address, geography and deprivation datasets that change over time. This methods article describes a reproducible workflow and online implementation for assigning New Zealand Index of Socioeconomic Deprivation (NZDep) measures to street addresses for census years 1991-2023. The method integrates public Land Information New Zealand address data, openly licensed postcode boundary polygons, Stats NZ census geographies and University of Otago NZDep datasets into an indexed SQLite lookup database and Shiny address-lookup tool (https://bit.ly/nzdep). Address labels are standardised, address points are spatially linked to year-specific census geographies, and geography identifiers are joined to corresponding NZDep values. Raw NZDep scores are converted to census-year-specific national percentile ranks, allowing relative deprivation position to be expressed on a consistent 1-100 scale across census periods. The workflow supports unit-level address lookup where available and base street-address lookup only where deprivation assignment is unambiguous. The final database contained 2405,589 address records and achieved 99.65% completeness across 16,839,123 potential address-year NZDep assignments.
BRIDGE is a Shiny-based application that provides an accessible, modular platform for individual and integrative multi-omics analysis. Using an independent SQLite database backend, it offers a local, private, and user-friendly environment that requires no prior computational expertise. The application supports proteomics, phospho-proteomics, and RNA-seq analyses through a comprehensive suite of visualization and analytical modules, together with an integrated multi-omics analysis pipeline. Built-in caching and asynchronous processing improve responsiveness, enabling efficient exploration, analysis, and visualization of multi-omics datasets on moderate hardware. BRIDGE is implemented in R using Shiny and is freely available as a Docker container at https://ghcr.io/paulilab/bridge. A public demonstration server with example datasets is available at https://bridge.imp.ac.at. Code and datasets are also available at https://github.com/paulilab/BRIDGE under DOI: https://doi.org/10.5281/zenodo.20215824.
Existing pangenome file formats are designed for batch processing. Graphs must be loaded into memory, and alignment files must be read sequentially. Indexed file formats that can be used directly from disk would be more appropriate for interactive applications. We propose GBZ-base and GAF-base - SQLite-backed file formats comparable to GBZ and GAF. GBZ-base supports efficient extraction of local subgraphs, and GAF-base lets us extract all alignments to the subgraph. Additionally, GAF-base is smaller than any other file format for sequence-to-graph alignments. From https://github.com/jltsiren/gbz-base and https://crates.io/crates/gbz-base under the MIT license.
Biomedical imaging workflows generate large collections of histology, fluorescence microscopy, and high-throughput imaging assets that are difficult to organize, search, and reuse using conventional file systems. We present PixelDeck, an open-source, local-first browser application for management of large image and video libraries on commodity workstations. PixelDeck integrates recursive import, SHA-256-based duplicate detection, metadata extraction, thumbnail generation, and full-text search within a responsive interface. The system uses a transparent SQLite-backed architecture with managed filesystem storage and asynchronous processing. Benchmarks on fresh isolated libraries containing up to 1000 assets showed import throughput exceeding 500 files/min and representative median query latencies in the millisecond range.
This paper proposes DB-LIO (database-driven LiDAR-inertial odometry), a simultaneous localization and mapping (SLAM) system that addresses memory scalability challenges in extended autonomous operation. Existing LiDAR-SLAM systems accumulate keyframe history in memory, leading to O(N) growth and out-of-memory failures during extended operation. To overcome this limitation, DB-LIO introduces three core design elements. First, it proposes a spatially indexed keyframe management scheme that persistently stores keyframes in SQLite with R-Tree spatial indexing, enabling O(logN+k) spatial queries that tightly couple cache eviction with factor-graph optimization requirements-a design that ensures every keyframe potentially involved in the next optimization cycle resides in cache. Second, it presents a four-level memory bounding architecture-SLAM-engine keyframe trimming with transparent on-demand reloading, a DB-level least recently used (LRU) cache with a spatial active window, Scan Context descriptor pool bounding, and iSAM2 sliding window compaction with a sparse global anchor graph-that collectively bounds the dominant memory consumers to O(C). Third, the DB-based persistent storage enables a localization mode that can reload previously built maps-including full point clouds, six-degree-of-freedom poses, timestamps, and inter-keyframe relationships-and perform pose estimation using the stored map, which is particularly valuable for agricultural robots and other autonomous systems requiring map reuse. Experiments on a custom orchard dataset demonstrate an 81.9% reduction in memory usage compared with that of the in-memory baseline (2888 MB → 524 MB), while preserving equivalent trajectory accuracy (absolute trajectory error (ATE) root mean square error (RMSE) 0.305 ± 0.001 m vs. 0.296 m). Validation on the KITTI odometry benchmark confirms that the proposed localization mode generalizes across different LiDAR types (Livox Mid360, Velodyne HDL-64E) and environments (orchard, urban driving).
Horizon scanning (HS) is a methodology that aims to capture signals and trends that highlight future opportunities and challenges. The National Institute for Health and Care Research (NIHR) Innovation Observatory routinely scans for medical technologies and therapeutics to inform policy and practice for healthcare in the United Kingdom (UK). To date, there is no standardized terminology for horizon scanning in healthcare. Here, we discuss the development of a data glossary and the IOAtlas web app. We extracted data points from 4 years' worth of NIHR Innovation Observatory HS projects and collated them by technology type and descriptive family. A source repository was established by extracting a list of all sources used in NIHR Innovation Observatory briefing notes between 2017 and 2021. The repository was validated by external HS organizations and experts, and sources were then mapped to the appropriate time horizons. The glossary and repository were converted to an SQLite database format and connected to a free web app, IOAtlas. After de-duplication and consolidation, a total of 148 data points were included in the glossary. The source repository consists of 149 sources, with 99 percent being compliant with searching for two or more technology types. The final SQLite database contained 35 tables with 36 relationships. We present a data glossary to provide globalized standardization for the terminology used in HS projects. The glossary can be accessed through the IOAtlas web app. Furthermore, we provide users with an interface to generate downloadable data extraction templates within IOAtlas.
This research examines the evolution of health information systems for recording medical incidents at sea within the framework of Telemedical Maritime Assistance Services (T.M.A.S.). The article provides a comprehensive comparative analysis between a low-code implementation and two new architectures: a monolithic approach (Flask/SQLite) and a decoupled approach oriented towards microservices (React/Node.js/ Docker). It also attempts to compare the capabilities of Large Language Models (LLMs), which have evolved from being less-than-ideal code-writing assistants to becoming efficient Principal Full-Stack Engineers. The findings demonstrate the exponential leap in the capabilities of Generative Artificial Intelligence and highlight that Monolithic Flask is the ideal solution for a small research lab or a land-based hospital with a fast network, as its maintenance time is minimal. However, for the shipping industry, the React/Node.js architecture is undoubtedly superior.
Multi-omic integration involves the management of diverse omic datasets. Conducting an effective analysis of these datasets necessitates a data management system that meets a specific set of requirements, such as rapid storage and retrieval of data with varying numbers of features and mixed data-types, ensurance of reliable and secure database transactions, extension of stored data row and column-wise and facilitation of data distribution. SQLite and DuckDB are embedded databases that fulfil these requirements. However, they utilize the structured query language (SQL) that hinders their implementation by the uninitiated user, and complicates their use in repetitive tasks due to the necessity of writing SQL queries. This study offers Omilayers, a Python package that encapsulates these two databases and exposes a subset of their functionality that is geared towards frequent and repetitive analytical procedures. Synthetic data were used to demonstrate the use of Omilayers and compare the performance of SQLite and DuckDB.
Oaks (Quercus L.) are key components of Northern Hemisphere Forest ecosystems, yet the integration of their rapidly growing genomic resources remains challenging. Here, we present quercusTOA, a genomic and functional resource that integrates nine Quercus genome assemblies. By combining automated functional annotation (InterProScan, eggNOG-mapper) with comparative genomics via genomic lift-over, we have developed a relational database designed to link protein-centric annotations with positional genomic data. Our results demonstrate that this integration supports cross-species ortholog identification and synteny analysis. To facilitate data access and exploration, we provide the quercusTOA-app, a user-friendly interface that streamlines database management and specific bioinformatic tasks. These include functional annotation, sequence-based homology searches, and multiple sequence alignments. Furthermore, the application enables the construction of phylogenetic trees for individual orthologous genes and proteins, allowing for the study of specific sequence evolution across the included assemblies. quercusTOA provides a standardized and scalable framework for evolutionary and functional studies in Quercus, offering a consistent approach to maintain genomic coordinate synchronization across the genus.
Head and neck cancers represent a critical global health issue, contributing to substantial morbidity and mortality. Recent research has explored the role of microRNAs (miRNAs) in these cancers by constructing miRNA-associated disease networks using bipartite graphs. Graph attention networks (GATs) have emerged as a powerful tool for predicting disease associations within such biological networks, offering enhanced accuracy in identifying potential miRNA-disease relationships. This study employs GATs to uncover and predict potential miRNA contributors to head and neck cancers. Data on miRNA-disease associations were sourced from the HMDD v4.0 database, a platform based on SQLite and Django. The head and neck neoplasms dataset included miRNA, disease, causality, category, and PubMed ID (PMID). GATs were applied to analyze the network, leveraging their ability to capture the significance and interdependencies of nodes and edges. The model used a learnable weight matrix to compute attention coefficients, normalize them, and aggregate information from neighboring nodes for edge prediction. The GAT model, integrating graph neural networks with attention mechanisms, achieved an accuracy of 83% in predicting miRNA-disease associations for head and neck neoplasms. This study highlights the potential of graph-based deep learning models, particularly GATs, in accurately predicting miRNA-disease associations. A functional enrichment analysis revealed significant involvement of miRNAs in oral cancer pathways, notably highlighting the critical roles of the TGF-beta and PI3K-Akt signaling pathways in tumor progression and cell survival. These findings offer a pathway to better understanding the molecular mechanisms underlying head and neck cancers. Future improvements in dataset size, model evaluation, and interpretability could further enhance prediction accuracy, potentially advancing diagnostic and therapeutic strategies for these cancers.
Fragment-based quantum chemistry offers a means to circumvent the nonlinear computational scaling of conventional electronic structure calculations by partitioning a large calculation into smaller subsystems, then considering the many-body interactions between them. Variants of this approach have been used to parameterize classical force fields and machine learning potentials, applications that benefit from interoperability between quantum chemistry codes. However, there is a dearth of software that provides interoperability yet is purpose-built to handle the combinatorial complexity of fragment-based calculations. To fill this void we introduce "Fragme∩t", an open-source software application that provides a tool for community validation of fragment-based methods, a platform for developing new approximations, and a framework for analyzing many-body interactions. Fragme∩t includes algorithms for automatic fragment generation and structure modification and for distance- and energy-based screening of the requisite subsystems. Checkpointing, database management, and parallelization are handled internally and results are archived into a portable database format. Interfaces are provided to quantum chemistry engines including Q-Chem, PySCF, xTB, Orca, CP2K, MRCC, Psi4, NWChem, GAMESS, and MOPAC. Applications reported here demonstrate parallel efficiencies around 96% on more than 1,000 processors but also showcase that the code can handle large-scale protein fragmentation using only workstation hardware, all with a codebase that is designed to be usable by non-experts. Fragme∩t conforms to modern software development best practices and is built upon well established technologies including Python, SQLite, and Ray. The source code is available under the Apache 2.0 license.