Severe heatwaves in urban areas significantly threaten public health, calling for establishing early warning strategies. Despite predicting occurrence of heatwaves and attributing historical mortality, predicting an incoming deadly heatwave remains a challenge due to the difficulty in defining and estimating heat-related mortality. Furthermore, establishing an early warning system imposes additional requirements, including data availability, spatial and temporal robustness, and decision costs. To address these challenges, we propose DeepTherm, a modular early warning system for deadly heatwave prediction without requiring heat-related mortality history. By highlighting the flexibility of deep learning, DeepTherm employs a dual-prediction pipeline, disentangling baseline mortality in the absence of heatwaves and other irregular events from all-cause mortality. We evaluated DeepTherm on real-world data across Spain. Results demonstrate consistent, robust, and accurate performance across diverse regions, time periods, and population groups while allowing trade-off between missed alarms and false alarms.
A Negative Pressure Room (NPR) is an essential requirement by the Bio-Safety Levels (BSLs) in biolabs or infectious-control hospitals to prevent deadly pathogens from being leaked from the facility. An NPR maintains a negative pressure inside with respect to the outside reference space so that microbes are contained inside of an NPR. Nowadays, differential pressure sensors (DPSs) are utilized by the Building Management Systems (BMSs) to control and monitor the negative pressure in an NPR. This paper demonstrates a non-invasive and stealthy attack on NPRs by spoofing a DPS at its resonant frequency. Our contributions are: (1) We show that DPSs used in NPRs typically have resonant frequencies in the audible range. (2) We use this finding to design malicious music to create resonance in DPSs, resulting in an overshooting in the DPS's normal pressure readings. (3) We show how the resonance in DPSs can fool the BMSs so that the NPR turns its negative pressure to a positive one, causing a potential \textit{leak} of deadly microbes from NPRs. We do experiments on 8 DPSs from 5 different manufacturers to evaluate their resonant frequencies considering the sampling tube length and find reso
We know from reinforcement learning theory that temporal difference learning can fail in certain cases. Sutton and Barto (2018) identify a deadly triad of function approximation, bootstrapping, and off-policy learning. When these three properties are combined, learning can diverge with the value estimates becoming unbounded. However, several algorithms successfully combine these three properties, which indicates that there is at least a partial gap in our understanding. In this work, we investigate the impact of the deadly triad in practice, in the context of a family of popular deep reinforcement learning models - deep Q-networks trained with experience replay - analysing how the components of this system play a role in the emergence of the deadly triad, and in the agent's performance
The deadly triad refers to the instability of a reinforcement learning algorithm when it employs off-policy learning, function approximation, and bootstrapping simultaneously. In this paper, we investigate the target network as a tool for breaking the deadly triad, providing theoretical support for the conventional wisdom that a target network stabilizes training. We first propose and analyze a novel target network update rule which augments the commonly used Polyak-averaging style update with two projections. We then apply the target network and ridge regularization in several divergent algorithms and show their convergence to regularized TD fixed points. Those algorithms are off-policy with linear function approximation and bootstrapping, spanning both policy evaluation and control, as well as both discounted and average-reward settings. In particular, we provide the first convergent linear $Q$-learning algorithms under nonrestrictive and changing behavior policies without bi-level optimization.
$Q$-learning with function approximation is one of the most empirically successful while theoretically mysterious reinforcement learning (RL) algorithms, and was identified in Sutton (1999) as one of the most important theoretical open problems in the RL community. Even in the basic linear function approximation setting, there are well-known divergent examples. In this work, we show that \textit{target network} and \textit{truncation} together are enough to provably stabilize $Q$-learning with linear function approximation, and we establish the finite-sample guarantees. The result implies an $O(ε^{-2})$ sample complexity up to a function approximation error. Moreover, our results do not require strong assumptions or modifying the problem parameters as in existing literature.
We consider a vaccination game that results with the introduction of premature and possibly scarce vaccines introduced in a desperate bid to combat the otherwise ravaging deadly pandemic. The response of unsure agents amid many uncertainties makes this game completely different from the previous studies. We construct a framework that combines SIS epidemic model with a variety of dynamic behavioral vaccination responses and demographic aspects. The response of each agent is influenced by the vaccination hesitancy and urgency, which arise due to their personal belief about efficacy and side-effects of the vaccine, disease characteristics, and relevant reported information (e.g., side-effects, disease statistics etc.). Based on such aspects, we identify the responses that are stable against static mutations. By analysing the attractors of the resulting ODEs, we observe interesting patterns in the limiting state of the system under evolutionary stable (ES) strategies, as a function of various defining parameters. There are responses for which the disease is eradicated completely (at limiting state), but none are stable against mutations. Also, vaccination abundance results in higher in
The formation of crystals has proven to be one of the most challenging phase transformations to quantitatively model - let alone to actually understand - be it by means of the latest experimental technique or the full arsenal of enhanced sampling approaches at our disposal. One of the most crucial quantities involved with the crystallization process is the nucleation rate, a single, elusive number that is supposed to quantify the average probability for a nucleus of critical size to occur within a certain volume and time span. A substantial amount of effort has been devoted to attempt a connection between the crystal nucleation rates computed by means of atomistic simulations and their experimentally measured counterparts. Sadly, this endeavour almost invariably fails to some extent, with the venerable classical nucleation theory typically blamed as the main culprit. Here, we review some of the recent advances in the field, focusing on a number of perhaps more subtle details that are sometimes overlooked when computing nucleation rates. We believe it is important for the community to be aware of the full impact of aspects such as finite size effects and slow dynamics, that often in
Background : The carnivorous plants of the genus Nepenthes, widely distributed in the Asian tropics, rely mostly on nutrients derived from arthropods trapped in their pitcher-shaped leaves and digested by their enzymatic fluid. The genus exhibits a great diversity of prey and pitcher forms and its mechanism of trapping has long intrigued scientists. The slippery inner surfaces of the pitchers, which can be waxy or highly wettable, have so far been considered as the key trapping devices. However, the occurrence of species lacking such epidermal specializations but still effective at trapping insects suggests the possible implication of other mechanisms. Methodology/Principal Findings : Using a combination of insect bioassays, high-speed video and rheological measurements, we show that the digestive fluid of Nepenthes rafflesiana is highly viscoelastic and that this physical property is crucial for the retention of insects in its traps. Trapping efficiency is shown to remain strong even when the fluid is highly diluted by water, as long as the elastic relaxation time of the fluid is higher than the typical time scale of insect movements. Conclusions/Significance : This finding challe
The recent detection of two faint and extended star clusters in the central regions of two Local Group dwarf galaxies, Eridanus II and Andromeda XXV, raises the question of whether clusters with such low densities can survive the tidal field of cold dark matter haloes with central density cusps. Using both analytic arguments and a suite of collisionless N-body simulations, I show that these clusters are extremely fragile and quickly disrupted in the presence of central cusps $ρ\sim r^{-α}$ with $α\gtrsim 0.2$. Furthermore, the scenario in which the clusters where originally more massive and sank to the center of the halo requires extreme fine tuning and does not naturally reproduce the observed systems. In turn, these clusters are long lived in cored haloes, whose central regions are safe shelters for $α\lesssim 0.2$. The only viable scenario for hosts that have preserved their primoridal cusp to the present time is that the clusters formed at rest at the bottom of the potential, which is easily tested by measurement of the clusters proper velocity within the host. This offers means to readily probe the central density profile of two dwarf galaxies as faint as $L_V\sim5\times 10^5
Elliptical galaxies have dynamically hot ($σ_{1D}$ ~ 100 -- 300 km s$^{-1}$) populations of stars, and presumably, smaller objects like comets. Because interstellar minor bodies are moving much faster, they hit planets harder and more often than in the local Galaxy. I estimate the rates for Chicxulub-scale impacts on an Earth-size planet in elliptical galaxies as a potential habitability constraint on intelligent life. Around most stars in a normal elliptical galaxy, these planets receive only ~0.01 -- 0.1 Gyr$^{-1}$, although hazardous rates may be common in certain compact early-type galaxies and red nuggets. About 5% of the stellar mass is in a region where the rate is >10 Gyr$^{-1}$, large enough to dominate the mass extinction rate. This suggests that elliptical galaxies have an exclusion zone of order one hundred parsecs in radius around their centers for the evolution of intelligent life.
This paper analyzes multi-step temporal difference (TD)-learning algorithms within the ``deadly triad'' scenario, characterized by linear function approximation, off-policy learning, and bootstrapping. In particular, we prove that $n$-step TD-learning algorithms converge to a solution as the sampling horizon $n$ increases sufficiently. The paper is divided into two parts. In the first part, we comprehensively examine the fundamental properties of their model-based deterministic counterparts, including projected value iteration, gradient descent algorithms, which can be viewed as prototype deterministic algorithms whose analysis plays a pivotal role in understanding and developing their model-free reinforcement learning counterparts. In particular, we prove that these algorithms converge to meaningful solutions when $n$ is sufficiently large. Based on these findings, in the second part, two $n$-step TD-learning algorithms are proposed and analyzed, which can be seen as the model-free reinforcement learning counterparts of the model-based deterministic algorithms.
According to the Pan American Health Organization, the number of cancer cases in Latin America was estimated at 4.2 million in 2022 and is projected to rise to 6.7 million by 2045. Osteosarcoma, one of the most common and deadly bone cancers affecting young people, is difficult to detect due to its unique texture and intensity. Surgical removal of osteosarcoma requires precise safety margins to ensure complete resection while preserving healthy tissue. Therefore, this study proposes a method for estimating the confidence interval of surgical safety margins in osteosarcoma surgery around the knee. The proposed approach uses MRI and X-ray data from open-source repositories, digital processing techniques, and unsupervised learning algorithms (such as k-means clustering) to define tumor boundaries. Experimental results highlight the potential for automated, patient-specific determination of safety margins.
This paper analyzes multi-step TD-learning algorithms within the `deadly triad' scenario, characterized by linear function approximation, off-policy learning, and bootstrapping. In particular, we prove that n-step TD-learning algorithms converge to a solution as the sampling horizon n increases sufficiently. The paper is divided into two parts. In the first part, we comprehensively examine the fundamental properties of their model-based deterministic counterparts, including projected value iteration, gradient descent algorithms, and the control theoretic approach, which can be viewed as prototype deterministic algorithms whose analysis plays a pivotal role in understanding and developing their model-free reinforcement learning counterparts. In particular, we prove that these algorithms converge to meaningful solutions when n is sufficiently large. Based on these findings, two n-step TD-learning algorithms are proposed and analyzed, which can be seen as the model-free reinforcement learning counterparts of the gradient and control theoretic algorithms.
The recent COVID-19 pandemic has highlighted the need of studying extreme, life-threatening phenomena in advance. In this article, a zombie epidemic in Uusimaa region in Finland is modeled. A stochastic agent based simulation model is proposed and extensive simulations are conducted for this purpose. The model utilizes knowledge on defensive human behavior during crises. Studying the effects of a hypothetical zombie attack resembles examining the spread of deadly diseases and of rumors. A zombie attack is simulated in the most densely populated region in Finland. The region's exact population densities over its rasterized geographical map are utilized. Furthermore, the simulations are used to study the effect of implementing a (strict or partial) quarantine area in the epicenter. Computationally efficient Scala codes and video animations of the simulated epidemics are provided. The main findings emphasize the importance of implementing very strict measures, without delay, to stop the outbreak.
Dengue fever is one of the most deadly mosquito-born tropical infectious diseases. Detailed long range forecast model is vital in controlling the spread of disease and making mitigation efforts. In this study, we examine methods used to forecast dengue cases for long range predictions. The dataset consists of local climate/weather in addition to global climate indicators of Singapore from 2000 to 2019. We utilize newly developed deep neural networks to learn the intricate relationship between the features. The baseline models in this study are in the class of recent transformers for long sequence forecasting tasks. We found that a Fourier mixed window attention (FWin) based transformer performed the best in terms of both the mean square error and the maximum absolute error on the long range dengue forecast up to 60 weeks.
Off-policy learning enables a reinforcement learning (RL) agent to reason counterfactually about policies that are not executed and is one of the most important ideas in RL. It, however, can lead to instability when combined with function approximation and bootstrapping, two arguably indispensable ingredients for large-scale reinforcement learning. This is the notorious deadly triad. The seminal work Sutton et al. (2008) pioneers Gradient Temporal Difference learning (GTD) as the first solution to the deadly triad, which has enjoyed massive success thereafter. During the derivation of GTD, some intermediate algorithm, called $A^\top$TD, was invented but soon deemed inferior. In this paper, we revisit this $A^\top$TD and prove that a variant of $A^\top$TD, called $A_t^\top$TD, is also an effective solution to the deadly triad. Furthermore, this $A_t^\top$TD only needs one set of parameters and one learning rate. By contrast, GTD has two sets of parameters and two learning rates, making it hard to tune in practice. We provide asymptotic analysis for $A^\top_t$TD and finite sample analysis for a variant of $A^\top_t$TD that additionally involves a projection operator. The convergence
In 2023, 639,300 people died in France, 35,900 fewer than in 2022, a year of high mortality. Over the last twenty years, from 2004 to 2023, January 3rd was the deadliest day, while August 15th was the least deadly one. Elderly people die significantly less often in the summer. Deaths are also less frequent on public holidays and Sundays. Finally, the risk of dying is higher on one's birthday, especially for young people.
Bangladesh's worsening dengue crisis, fueled by its tropical climate, poor waste management infrastructure, rapid urbanization, and dense population, has led to increasingly deadly outbreaks, posing a significant public health threat. To address this, we propose a nonlinear, time-nonhomogeneous SEIR model incorporating seasonality through a novel transmission rate function. The model parameters are estimated using Bayesian inference with the Metropolis-Hastings algorithm in a Markov Chain Monte Carlo (MCMC) framework, calibrated with real-life dengue data from Bangladesh. To account for stochasticity and better assess outbreak probabilities, we extend the model to a time-nonhomogeneous continuous-time Markov chain (CTMC) framework. Our model provides new insights that can guide policymakers and offer a robust mathematical framework to better combat this crisis.
Colorectal cancer is a deadly disease that has become increasingly prevalent in recent years. Early detection is crucial for saving lives, but traditional diagnostic methods such as colonoscopy and biopsy have limitations. Colonoscopy cannot provide detailed information within the tissues affected by cancer, while biopsy involves tissue removal, which can be painful and invasive. In order to improve diagnostic efficiency and reduce patient suffering, we studied machine-learningbased approach for colorectal tissue classification that uses acoustic resolution photoacoustic microscopy (ARPAM). With this tool, we were able to classify benign and malignant tissue using multiple machine learning methods. Our results were analyzed both quantitatively and qualitatively to evaluate the effectiveness of our approach.
As the population grows and more land is being used for urbanization, ecosystems are disrupted by our roads and cars. This expansion of infrastructure cuts through wildlife territories, leading to many instances of Wildlife-Vehicle Collision (WVC). These instances of WVC are a global issue that is having a global socio-economic impact, resulting in billions of dollars in property damage and, at times, fatalities for vehicle occupants. In Saudi Arabia, this issue is similar, with instances of Camel-Vehicle Collision (CVC) being particularly deadly due to the large size of camels, which results in a 25% fatality rate [4]. The focus of this work is to test different object detection models on the task of detecting camels on the road. The Deep Learning (DL) object detection models used in the experiments are: CenterNet, EfficientDet, Faster R-CNN, and SSD. Results of the experiments show that CenterNet performed the best in terms of accuracy and was the most efficient in training. In the future, the plan is to expand on this work by developing a system to make countryside roads safer.