Data is crucial for machine learning (ML) applications, yet acquiring large datasets can be costly and time-consuming, especially in complex, resource-intensive fields like biopharmaceuticals. A key process in this industry is upstream bioprocessing, where living cells are cultivated and optimised to produce therapeutic proteins and biologics. The intricate nature of these processes, combined with high resource demands, often limits data collection, resulting in smaller datasets. This comprehensive review explores ML methods designed to address the challenges posed by small data and classifies them into a taxonomy to guide practical applications. Furthermore, each method in the taxonomy was thoroughly analysed, with a detailed discussion of its core concepts and an evaluation of its effectiveness in tackling small data challenges, as demonstrated by application results in the upstream bioprocessing and other related domains. By analysing how these methods tackle small data challenges from different perspectives, this review provides actionable insights, identifies current research gaps, and offers guidance for leveraging ML in data-constrained environments.
Bioprocess mechanistic modeling is essential for advancing intelligent digital twin representation of biomanufacturing, yet challenges persist due to complex intracellular regulation, stochastic system behavior, and limited experimental data. This paper introduces a symbolic and statistical learning framework to identify key regulatory mechanisms and quantify model uncertainty. Bioprocess dynamics is formulated with stochastic differential equations characterizing intrinsic process variability, with a predefined set of candidate regulatory mechanisms constructed from biological knowledge. A Bayesian learning approach is developed, which is based on a joint learning of kinetic parameters and regulatory structure through a formulation of the mixture model. To enhance computational efficiency, a Metropolis-adjusted Langevin algorithm with adjoint sensitivity analysis is developed for posterior exploration. Compared to state-of-the-art Bayesian inference approaches, the proposed framework achieves improved sample efficiency and robust model selection. An empirical study demonstrates its ability to recover missing regulatory mechanisms and improve model fidelity under data-limited condi
Efficient implementation of nonlinear model predictive control (NMPC) for bioprocesses remains challenging because large nonlinear models are difficult to organize, simulate, and embed within optimization and control workflows. This difficulty is particularly pronounced for large-scale and multiscale systems that require hierarchical model construction and customized simulation strategies. To address this issue, we present GlycoPy, a CasADi-based Python framework for hierarchical modeling, optimization, and control of bioprocesses. GlycoPy combines an equation-oriented, object-oriented modeling architecture with CasADi's symbolic and differentiable computational capabilities, enabling hierarchical model composition, numerical and symbolic simulation, parameter estimation, dynamic optimization, and NMPC within a unified workflow. A key feature of the framework is its support for customized differentiable simulation algorithms that can be embedded directly in gradient-based optimization and control. GlycoPy is demonstrated on a multiscale monoclonal antibody glycosylation process in Chinese hamster ovary cell culture, where it is used for hierarchical model construction, quasi-steady
Parametric differential equations of the form du/dt = f(u, x, t, p) are fundamental in science and engineering. While deep learning frameworks such as the Fourier Neural Operator (FNO) can efficiently approximate solutions, they struggle with inverse problems, sensitivity estimation (du/dp), and concept drift. We address these limitations by introducing a sensitivity-based regularization strategy, called Sensitivity-Constrained Fourier Neural Operators (SC-FNO). SC-FNO achieves high accuracy in predicting solution paths and consistently outperforms standard FNO and FNO with physics-informed regularization. It improves performance in parameter inversion tasks, scales to high-dimensional parameter spaces (tested with up to 82 parameters), and reduces both data and training requirements. These gains are achieved with a modest increase in training time (30% to 130% per epoch) and generalize across various types of differential equations and neural operators. Code and selected experiments are available at: https://github.com/AMBehroozi/SC_Neural_Operators
In chemical processing and bioprocessing, conventional online sensors are limited to measure only basic process variables like pressure and temperature, pH, dissolved O and CO$_2$ and viable cell density (VCD). The concentration of other chemical species is more difficult to measure, as it usually requires an at-line or off-line approach. Such approaches are invasive and slow compared to on-line sensing. It is known that different molecules can be distinguished by their interaction with monochromatic light, producing different profiles for the resulting Raman spectrum, depending on the concentration. Given the availability of reference measurements for the target variable, regression methods can be used to model the relationship between the profile of the Raman spectra and the concentration of the analyte. This work focused on pretreatment methods of Raman spectra for the facilitation of the regression task using Machine Learning and Deep Learning methods, as well as the development of new regression models based on these methods. In the majority of cases, this allowed to outperform conventional Raman models in terms of prediction error and prediction robustness.
Efficient and robust bioprocess control is essential for maximizing performance and adaptability in advanced biotechnological systems. In this work, we present a reinforcement-learning framework for multi-setpoint and multi-trajectory tracking. Tracking multiple setpoints and time-varying trajectories in reinforcement learning is challenging due to the complexity of balancing multiple objectives, a difficulty further exacerbated by system uncertainties such as uncertain initial conditions and stochastic dynamics. This challenge is relevant, e.g., in bioprocesses involving microbial consortia, where precise control over population compositions is required. We introduce a novel return function based on multiplicative reciprocal saturation functions, which explicitly couples reward gains to the simultaneous satisfaction of multiple references. Through a case study involving light-mediated cybergenetic growth control in microbial consortia, we demonstrate via computational experiments that our approach achieves faster convergence, improved stability, and superior control compliance compared to conventional quadratic-cost-based return functions. Moreover, our method enables tuning of th
This paper introduces a novel fractional-order chemostat model (FOCM) incorporating Caputo fractional derivative with sliding memory (CFDS) to capture microbial memory effects in biological water treatment, addressing limitations of integer-order models that overlook time-dependent behaviors and fail to capture microbial memory such as delayed growth responses to past nutrient availability, history-dependent adaptation to inflow fluctuations, and persistent historical effects over hours to days, which are biologically critical in wastewater treatment. By optimizing periodic dilution rate control, we minimize the average pollutant output, constrained by treatment capacity and periodic boundaries. Key contributions include: (1) a rigorous fractional framework linking microbial kinetics to memory-driven control; (2) reduction to a 1D fractional-order differential equation (FDE) for computational efficiency; (3) proofs of optimal periodic solution (OPS) existence/uniqueness via Schauder's theorem and convexity; (4) bang-bang control derivation using fractional Pontryagin maximum principle (PMP) and Fourier-Gegenbauer pseudospectral (FG-PS) method with a specialized edge-detection techn
Provenance information are essential for the traceability of scientific studies or experiments and thus crucial for ensuring the credibility and reproducibility of research findings. This paper discusses a comprehensive provenance framework combining the two types 1. workflow provenance, and 2. data provenance as well as their dimensions and granularity, which enables the answering of W7+1 provenance questions. We demonstrate the applicability by employing a biomedical research use case, that can be easily transferred into other scientific fields. An integration of these concepts into a unified framework enables credibility and reproducibility of the research findings.
Automated Machine Learning (AutoML) technology can lower barriers in data work yet still requires human intervention to be functional. However, the complex and collaborative process resulting from humans and machines trading off work makes it difficult to trace what was done, by whom (or what), and when. In this research, we construct a taxonomy of data work artifacts that captures AutoML and human processes. We present a rigorous methodology for its creation and discuss its transferability to the visual design process. We operationalize the taxonomy through the development of AutoMLTrace, a visual interactive sketch showing both the context and temporality of human-ML/AI collaboration in data work. Finally, we demonstrate the utility of our approach via a usage scenario with an enterprise software development team. Collectively, our research process and findings explore challenges and fruitful avenues for developing data visualization tools that interrogate the sociotechnical relationships in automated data work.
Bioprocesses are central to modern biotechnology, enabling sustainable production in pharmaceuticals, specialty chemicals, cosmetics, and food. However, developing high-performing processes is costly and complex, requiring iterative, multi-scale experimentation from microtiter plates to pilot reactors. Conventional Design of Experiments (DoE) approaches often struggle to address process scale-up and the joint optimization of reaction conditions and biocatalyst selection. We propose a multi-fidelity batch Bayesian optimization framework to accelerate bioprocess development and reduce experimental costs. The method integrates Gaussian Processes tailored for multi-fidelity modeling and mixed-variable optimization, guiding experiment selection across scales and biocatalysts. A custom simulation of a Chinese Hamster Ovary bioprocess, capturing non-linear and coupled scale-up dynamics, is used for benchmarking against multiple simulated industrial DoE baselines. Multiple case studies show how the proposed workflow can achieve a reduction in experimental costs and increased yield. This work provides a data-efficient strategy for bioprocess optimization and highlights future opportunities
This study presents a broad perspective of hybrid process modeling and optimization combining the scientific knowledge and data analytics in bioprocessing and chemical engineering with a science-guided machine learning (SGML) approach. We divide the approach into two major categories. The first refers to the case where a data-based ML model compliments and makes the first-principle science-based model more accurate in prediction, and the second corresponds to the case where scientific knowledge helps make the ML model more scientifically consistent. We present a detailed review of scientific and engineering literature relating to the hybrid SGML approach, and propose a systematic classification of hybrid SGML models. For applying ML to improve science-based models, we present expositions of the sub-categories of direct serial and parallel hybrid modeling and their combinations, inverse modeling, reduced-order modeling, quantifying uncertainty in the process and even discovering governing equations of the process model. For applying scientific principles to improve ML models, we discuss the sub-categories of science-guided design, learning and refinement. For each sub-category, we i
Discotic colloids give rise to a paradigmatic family of liquid crystals with sound applications in Materials Science. In this paper, Monte Carlo simulations are employed to characterize the low-temperature liquid crystal phase diagram and the vapour-liquid coexistence of discotic colloids interacting via a Kihara potential. Discoidal particles with thickness-diameter aspect ratios $L^*\equiv L/D$=\,0.5, 0.3, 0.2 and 0.1 are considered. For the less anisotropic particles ($L^*$$\ge$0.2), coexistence of a vapour phase with the isotropic fluid and with the columnar liquid crystal phase is observed. As the particle anisotropy increases, the vapour-liquid coexistence shifts to lower temperatures and its density range diminishes, eventually merging with coexistences involving the liquid crystal phases. The $L^*=$\,0.1 fluid displays a rich sequence of mesophases, including a nematic phase and a novel lamellar phase in which particles arrange in layers perpendicular to the nematic director.
We propose a robust adaptive online synchronization method for leader-follower networks of nonlinear heterogeneous agents with system uncertainties and input magnitude saturation. Synchronization is achieved using a Distributed input Magnitude Saturation Adaptive Control with Reinforcement Learning (DMSAC-RL), which improves the empirical performance of policies trained on off-the-shelf models using Reinforcement Learning (RL) strategies. The leader observes the performance of a reference model, and followers observe the states and actions of the agents they are connected to, but not the reference model. The leader and followers may differ from the reference model in which the RL control policy was trained. DMSAC-RL uses an internal loop that adjusts the learned policy for the agents in the form of augmented input to solve the distributed control problem, including input-matched uncertainty parameters. We show that the synchronization error of the heterogeneous network is Uniformly Ultimately Bounded (UUB). Numerical analysis of a network of Multiple Input Multiple Output (MIMO) systems supports our theoretical findings.
Biopharmaceutical manufacturing is vital to public health but lacks the agility for rapid, on-demand production of biotherapeutics due to the complexity and variability of bioprocesses. To overcome this, we introduce RApid PosTeriOR GENerator (RAPTOR-GEN), a mechanism-informed Bayesian learning framework designed to accelerate intelligent digital twin development from sparse and heterogeneous experimental data. This framework is built on a multi-scale probabilistic knowledge graph (pKG), formulated as a stochastic differential equation (SDE)-based foundational model that captures the nonlinear dynamics of bioprocesses. RAPTOR-GEN consists of two ingredients: (i) an interpretable metamodel integrating linear noise approximation (LNA) that exploits the structural information of bioprocessing mechanisms and a sequential learning strategy to fuse heterogeneous and sparse data, enabling inference of latent state variables and explicit approximation of the intractable likelihood function; and (ii) an efficient Bayesian posterior sampling method that utilizes Langevin diffusion (LD) to accelerate posterior exploration by exploiting the gradients of the derived likelihood. It generalizes t
Driven by the critical needs of biomanufacturing 4.0, we introduce a probabilistic knowledge graph hybrid model characterizing the risk- and science-based understanding of bioprocess mechanisms. It can faithfully capture the important properties, including nonlinear reactions, partially observed state, and nonstationary dynamics. Given very limited real process observations, we derive a posterior distribution quantifying model estimation uncertainty. To avoid the evaluation of intractable likelihoods, Approximate Bayesian Computation sampling with Sequential Monte Carlo (ABC-SMC) is utilized to approximate the posterior distribution. Under high stochastic and model uncertainties, it is computationally expensive to match output trajectories. Therefore, we create a linear Gaussian dynamic Bayesian network (LG-DBN) auxiliary likelihood-based ABC-SMC approach. Through matching the summary statistics driven through LG-DBN likelihood that can capture critical interactions and variations, the proposed algorithm can accelerate hybrid model inference, support process monitoring, and facilitate mechanism learning and robust control.
Multiplicity and clustering of young pre-main sequence stars appear as critical clues to constrain the star formation process. Taurus is the archetypical example of the most quiescent star forming regions that may still retain primeval signatures of star formation. This work identifies local overdense stellar structures at the 99.8\% confidence level above random expectation using the DBSCAN algorithm, and setting its free parameters based on the one-point correlation function and the k-nearest neighbor statistics. Nearly half of the entire stellar population in Taurus is found to be concentrated in 20 dense, tiny and prolate regions called NESTs (for Nested Elementary STructures). They are regularly spaced ($\approx 2$ pc) and mainly oriented along the gas filaments axes. Each NEST contains between 4 and 23 stars. Inside NESTs, the surface density of stars may be as high as 2500 pc$^{-2}$. Nearly half (11) of these NESTs contain about 75\% of the class 0/I objects. The balance between Class I, II, and, III fraction within the NESTs suggests that they may be ordered as an evolutionary temporal scheme. We have inferred that only 20\% of stars in Taurus do not belong to any kind of s
Motivated by the pressing challenges in the digital twin development for biomanufacturing systems, we introduce an adjoint sensitivity analysis (SA) approach to expedite the learning of mechanistic model parameters. In this paper, we consider enzymatic stochastic reaction networks representing a multi-scale bioprocess mechanistic model that allows us to integrate disparate data from diverse production processes and leverage the information from existing macro-kinetic and genome-scale models. To support forward prediction and backward reasoning, we develop a convergent adjoint SA algorithm studying how the perturbations of model parameters and inputs (e.g., initial state) propagate through enzymatic reaction networks and impact on output trajectory predictions. This SA can provide a sample efficient and interpretable way to assess the sensitivities between inputs and outputs accounting for their causal dependencies. Our empirical study underscores the resilience of these sensitivities and illuminates a deeper comprehension of the regulatory mechanisms behind bioprocess through sensitivities.
In this paper, a differential MOEMS accelerometer based on the Fabry-Perot (FP) micro-cavities is presented. The optical system of the device consists of two FP cavities and the mechanical system is composed of a proof mass that is suspended by four springs. The applied acceleration tends to move the PM from its resting position. This mechanical displacement can be measured by the FP interferometer formed between the proof mass cross-section and the optical fiber end face. The proposed sensor is fabricated on a silicon on insulator (SOI) wafer using the bulk micromachining method. The results of the sensor characterization show that the accelerometer has a linear response in the range of 1g. Also, the optical sensitivity and resolution of the sensor in the static characterization are 6.52 nm/g and 153ug. The sensor sensitivity in the power measurement is 49.6 mV/g and its resonant is at 1372 Hz. Using the differential measurement method increases the sensitivity of the accelerometer. Based on experimental data, the sensor sensitivity is two times as high as that of a similar MOEMS accelerometer with one FP cavity.
Bayesian optimization has become widely popular across various experimental sciences due to its favorable attributes: it can handle noisy data, perform well with relatively small datasets, and provide adaptive suggestions for sequential experimentation. While still in its infancy, Bayesian optimization has recently gained traction in bioprocess engineering. However, experimentation with biological systems is highly complex and the resulting experimental uncertainty requires specific extensions to classical Bayesian optimization. Moreover, current literature often targets readers with a strong statistical background, limiting its accessibility for practitioners. In light of these developments, this review has two aims: first, to provide an intuitive and practical introduction to Bayesian optimization; and second, to outline promising application areas and open algorithmic challenges, thereby highlighting opportunities for future research in machine learning.
Data scarcity and sparsity in bio-manufacturing poses challenges for accurate model development, process monitoring, and optimization. We aim to replicate and capture the complex dynamics of industrial bioprocesses by proposing the use of a Quantum Wasserstein Generative Adversarial Network with Gradient Penalty (QWGAN-GP) to generate synthetic time series data for industrially relevant processes. The generator within our GAN is comprised of a Parameterized Quantum Circuit (PQC). This methodology offers potential advantages in process monitoring, modeling, forecasting, and optimization, enabling more efficient bioprocess management by reducing the dependence on scarce experimental data. Our results demonstrate acceptable performance in capturing the temporal dynamics of real bioprocess data. We focus on Optical Density, a key measurement for Dry Biomass estimation. The data generated showed high fidelity to the actual historical experimental data. This intersection of quantum computing and machine learning has opened new frontiers in data analysis and generation, particularly in computationally intensive fields, for use cases such as increasing prediction accuracy for soft sensor d