Dynamic 3D hand reconstruction from egocentric videos is essential for next-generation computing platforms such as AR/VR and AI glasses. Despite its importance, most prior works focus either on multi-view 3D hand reconstruction or on 4D human body reconstruction. Egocentric 4D hand reconstruction remains challenging due to fast head motion, rapid hand dynamics, severe occlusions, and inherent ambiguity from single-view observations. To address these challenges, we introduce Hand-4DGS, the first feed-forward framework for reconstructing dynamic 4D hands directly from egocentric videos, enabling both fast (~60 FPS) inference and strong generalization. Our approach incorporates a mesh-guided representation for structural priors and temporal convolutions to model dynamic motion. We evaluate our framework on two challenging egocentric datasets, H2O and ARCTIC, and demonstrate significant improvements over baselines. Our method benefits from the generalization capability of feed-forward networks and effective 2D image supervision through Gaussian splatting, without requiring expensive 3D hand pose ground-truth annotations.
Acupoint therapy is a core therapeutic method of Traditional Chinese Medicine (TCM), and it requires a high level of expertise and skills to detect acupoints and perform acupuncture and moxibustion. Existing mixed reality (MR)-based training methods often fall short in accurate real-time detection and visualization of acupoints on the hand, limb, or torso of a real person and do not support various techniques of acupuncture and moxibustion. Moreover, evaluation standards and visual guidance with fine details for each step during MR-based training are typically missing. To this end, we propose the MR-based TCM Acupoint Therapy Teaching System (MRATTS)--an MR-based acupoint therapy teaching and training framework. MRATTS is based on a real-time hand, limb, and torso acupoint detection method to accurately track and visualize acupoints on real patients through MR. On top of that, in collaboration with an experienced acupoint therapist, we design a practice method with interactive visual guidance for various acupoint therapy techniques that simulate acupressure, acupuncture (insertion, lifting-thrusting, and twisting), and moxibustion (mild, sparrow-pecking, and whirling). A set of TCM
Adaptive therapy (AT) is designed to postpone the emergence of drug resistance by exploiting evolutionary competition among tumor subclones. Most mathematical models of AT assume a binary population structure of drug-sensitive and drug-resistant cells, which neglects the continuous nature of phenotypic plasticity. In this study, we propose a mathematical model that integrates a continuous drug susceptibility index with a probabilistic inheritance function to describe clonal dynamics under therapy. The resulting integro-differential system generalizes traditional two-type competition models and captures both heterogeneity and plasticity of tumor cells. Analytical and numerical studies show that (i) continuous therapy drives rapid expansion of resistant clones, (ii) adaptive therapy maintains long-term tumor control by dynamically regulating sensitive populations, and (iii) high phenotypic plasticity accelerates phenotype switching, leading to earlier tumor relapse following continuous therapy. These results identify critical parameter regimes where adaptive therapy outperforms fixed regimens and highlight the essential role of plasticity in shaping treatment outcomes. The proposed f
Improving object manipulation skills through hand-object interaction exercises is crucial for rehabilitation. Despite limited healthcare resources, physical therapists propose remote exercise routines followed up by remote monitoring. However, remote motor skills assessment remains challenging due to the lack of effective motion visualizations. Therefore, exploring innovative ways of visualization is crucial, and virtual reality (VR) has shown the potential to address this limitation. However, it is unclear how VR visualization can represent understandable hand-object interactions. To address this gap, in this paper, we present VRMoVi, a VR visualization system that incorporates multiple levels of 3D visualization layers to depict movements. In a 2-stage study, we showed VRMoVi's potential in representing hand-object interactions, with its visualization outperforming traditional representations, and detailed features improved the hand-object interactions understanding. This study takes the initial step in developing VR visualization of hand-object interaction to support remote physical therapy.
This paper develops Virtual Speech Therapist (VST), an intelligent agent-based platform that streamlines stuttering assessment and delivers customized therapy planning through automated and adaptive AI-driven workflows. VST integrates state-of-the-art deep learning-based stuttering classification, and multi-agent large language model (LLM) reasoning to support evidence-based clinical decision-making. The VST begins with the acquisition and feature extraction of patient speech samples, followed by robust classification of stuttering types. Building on these outputs, VST initiates an agentic reasoning process in which specialized LLM agents autonomously generate, critique, and iteratively refine individualized therapy plans. A dedicated critic agent evaluates all generated therapy plans to ensure clinical safety, methodological soundness, and alignment with peer-reviewed evidence and established professional guidelines. The resulting output is a comprehensive, patient-specific therapy draft intended for clinician review. Incorporating clinician feedback, the system then produces a finalized therapy plan suitable for patient delivery, thereby maintaining a clinician-in-the-loop paradi
3D hand pose estimation is a fundamental task in understanding human hands. However, accurately estimating 3D hand poses remains challenging due to the complex movement of hands, self-similarity, and frequent occlusions. In this work, we address two limitations: the inability of existing 3D hand pose estimation methods to estimate aleatoric (data) uncertainty, and the lack of uncertainty modeling that incorporates joint correlation knowledge, which has not been thoroughly investigated. To this end, we introduce aleatoric uncertainty modeling into the 3D hand pose estimation framework, aiming to achieve a better trade-off between modeling joint correlations and computational efficiency. We propose a novel parameterization that leverages a single linear layer to capture intrinsic correlations among hand joints. This is enabled by formulating the hand joint output space as a probabilistic distribution, allowing the linear layer to capture joint correlations. Our proposed parameterization is used as a task head layer, and can be applied as an add-on module on top of the existing models. Our experiments demonstrate that our parameterization for uncertainty modeling outperforms existing
Hand gesture recognition is an important aspect of human-computer interaction. It forms the basis of sign language for the visually impaired people. This work proposes a novel hand gesture recognizing system for the differently-abled persons. The model uses a convolutional neural network, known as VGG-16 net, for building a trained model on a widely used image dataset by employing Python and Keras libraries. Furthermore, the result is validated by the NUS dataset, consisting of 10 classes of hand gestures, fed to the model as the validation set. Afterwards, a testing dataset of 10 classes is built by employing Google's open source Application Programming Interface (API) that captures different gestures of human hand and the efficacy is then measured by carrying out experiments. The experimental results show that by combining a transfer learning mechanism together with the image data augmentation, the VGG-16 net produced around 98% accuracy.
Fear of flying is a serious problem that affects millions of individuals. Exposure therapy for fear of flying is an effective therapy technique. However, exposure therapy is also expensive, logistically difficult to arrange, and presents significant problems of patient confidentiality and potential embarrassment. We have developed a virtual airplane for use in fear of flying therapy. Using the virtual airplane for exposure therapy is a potential solution to many of the current problems of fear of flying exposure therapy. We describe the design of the virtual airplane and present a case report on its use for fear of flying exposure therapy.
We revisit the role of texture in monocular 3D hand reconstruction, not as an afterthought for photorealism, but as a dense, spatially grounded cue that can actively support pose and shape estimation. Our observation is simple: even in high-performing models, the overlay between predicted hand geometry and image appearance is often imperfect, suggesting that texture alignment may be an underused supervisory signal. We propose a lightweight texture module that embeds per-pixel observations into UV texture space and enables a novel dense alignment loss between predicted and observed hand appearances. Our approach assumes access to a differentiable rendering pipeline and a model that maps images to 3D hand meshes with known topology, allowing us to back-project a textured hand onto the image and perform pixel-based alignment. The module is self-contained and easily pluggable into existing reconstruction pipelines. To isolate and highlight the value of texture-guided supervision, we augment HaMeR, a high-performing yet unadorned transformer architecture for 3D hand pose estimation. The resulting system improves both accuracy and realism, demonstrating the value of appearance-guided ali
Combination therapy, a treatment approach that involves two or more monotherapies, is widely considered to enhance therapeutic efficacy across different medical conditions. It was often believed that combination therapy is beneficial because of pharmacological interactions between its component monotherapies. However, through laboratory experiments, pharmacologists have recently noted that the benefits of some combination therapies might be largely driven by varying patient-level responses to their component monotherapies. Without accounting for such patient-level heterogeneity, classical statistical inference frameworks for combination therapy might be inadequate and overly optimistic. In this paper, we introduce a novel and model-free statistical inference framework to complement the classical one and evaluate combination therapy after adjusting for patient-level heterogeneity in responses to monotherapies. We address the non-identifiability and nonlinearity issue inherent in adjustment of patient-level heterogeneity and establish conditions for the (partial) identifiability of the cross-world target parameter. We develop an outcome-based optimal matching scheme to achieve asympt
Personalized therapy, in which a therapeutic practice is adapted to an individual patient, can lead to improved health outcomes. Typically, this is accomplished by relying on a therapist's training and intuition along with feedback from a patient. However, this requires the therapist to become an expert on any technological components, such as in the case of Virtual Reality Exposure Therapy (VRET). While there exist approaches to automatically adapt therapeutic content to a patient, they generally rely on hand-authored, pre-defined rules, which may not generalize to all individuals. In this paper, we propose an approach to automatically adapt therapeutic content to patients based on physiological measures. We implement our approach in the context of virtual reality arachnophobia exposure therapy, and rely on experience-driven procedural content generation via reinforcement learning (EDPCGRL) to generate virtual spiders to match an individual patient. Through a human subject study, we demonstrate that our system significantly outperforms a more common rules-based method, highlighting its potential for enhancing personalized therapeutic interventions.
Phage therapy is an alternative treatment method for bacterial infections. It has shown particular promise in reducing bacterial load while preventing antibiotic resistance. Here, we develop a mathematical model of a bacterial infection within a host to study phage therapy. It incorporates interactions between phages, bacteria, the immune system, and antibiotics. Additionally, the model includes bacterial social dynamics that provide protection from treatments and the innate immune response. We analytically and numerically identify all of the equilibria of the model and derive insights regarding the overall effectiveness of phage therapy. Without phage therapy, the model exhibits bistability: bacteria populations above a threshold grow and become entrenched, while those below it can be effectively suppressed by the immune system. We find that that phages destabilize the former equilibrium, and thus in combination with the immune system are able to suppress the bacteria. We conducted bifurcation analyses, which show that the equilibrium with a suppressed population of bacteria can become unstable. In this scenario, the system undergoes oscillations. However, these oscillations -- wh
Purpose: Proton therapy provides superior dose conformity compared to photon therapy, but its treatment planning is challenged by sensitivity to anatomical changes, setup/range uncertainties, and computational complexity. This review evaluates the role of artificial intelligence (AI) in improving proton therapy treatment planning. Materials and methods: Recent studies on AI applications in image reconstruction, image registration, dose calculation, plan optimization, and quality assessment were reviewed and summarized by application domain and validation strategy. Results: AI has shown promise in automating contouring, enhancing imaging for dose calculation, predicting dose distributions, and accelerating robust optimization. These methods reduce manual workload, improve efficiency, and support more personalized planning and adaptive planning. Limitations include data scarcity, model generalizability, and clinical integration. Conclusion: AI is emerging as a key enabler of efficient, consistent, and patient-specific proton therapy treatment planning. Addressing challenges in validation and implementation will be essential for its translation into routine clinical practice.
This paper proposes the second version of the widespread Hand Gesture Recognition dataset HaGRID -- HaGRIDv2. We cover 15 new gestures with conversation and control functions, including two-handed ones. Building on the foundational concepts proposed by HaGRID's authors, we implemented the dynamic gesture recognition algorithm and further enhanced it by adding three new groups of manipulation gestures. The ``no gesture" class was diversified by adding samples of natural hand movements, which allowed us to minimize false positives by 6 times. Combining extra samples with HaGRID, the received version outperforms the original in pre-training models for gesture-related tasks. Besides, we achieved the best generalization ability among gesture and hand detection datasets. In addition, the second version enhances the quality of the gestures generated by the diffusion model. HaGRIDv2, pre-trained models, and a dynamic gesture recognition algorithm are publicly available.
Chimeric antigen receptor T-cell (CAR-T) therapy is considered a promising cancer treatment. The dynamic response to this therapy can be broadly divided into a short-term phase, ranging from weeks to months, and a long-term phase, ranging from months to years. While the short-term response, encompassing the multiphasic kinetics of CAR-T cells, is better understood, the mechanisms underlying the outcomes of the long-term response, characterized by sustained remission, relapse, or disease progression, remain less understood due to limited clinical data. Here, we analyze the long-term dynamics of a previously validated mathematical model of CAR-T cell therapy. We perform a comprehensive stability and bifurcation analysis, examining model equilibria and their dynamics over the entire parameter space. Our results show that therapy failure results from a combination of insufficient CAR-T cell proliferation and increased tumor immunosuppression. By combining different techniques of nonlinear dynamics, we identify Hopf and Bogdanov-Takens bifurcations, which allow to elucidate the mechanisms behind oscillatory remissions and transitions to tumor escape. In particular, rapid expansion of CA
With the increase number of companies focusing on commercializing Augmented Reality (AR), Virtual Reality (VR) and wearable devices, the need for a hand based input mechanism is becoming essential in order to make the experience natural, seamless and immersive. Hand pose estimation has progressed drastically in recent years due to the introduction of commodity depth cameras. Hand pose estimation based on vision is still a challenging problem due to its complexity from self-occlusion (between fingers), close similarity between fingers, dexterity of the hands, speed of the pose and the high dimension of the hand kinematic parameters. Articulated hand pose estimation is still an open problem and under intensive research from both academia and industry. The 2 approaches used for hand pose estimation are: discriminative and generative. Generative approach is a model based that tries to fit a hand model to the observed data. Discriminative approach is appearance based, usually implemented with machine learning (ML) and require a large amount of training data. Recent hand pose estimation uses hybrid approach by combining both discriminative and generative methods into a single hand pipeli
This paper presents a Virtual Reality (VR) art therapy known as "Break Times" which aims to enhance students' mental well-being and foster creative expression. The proposed "Break Times" application mimics the art therapy sessions in the VR environment design. Pilot user acceptance test with 10 participants showed a notable reduction in stress levels, with 50% reporting normal stress levels post-intervention, compared to 20% pre-intervention. Participants praised the "Break Times" therapy's functionality and engagement features and suggested improvements such as saving creations, incorporating 3D painting, and expanding the artmaking scene variety. The study highlights that VR art therapy has potential as an effective tool for stress management, emphasizing the need for continued refinement to maximize its therapeutic benefits.
Virtual Reality (VR) has been utilized for several applications and has shown great potential for rehabilitation, especially for home therapy. However, these systems solely rely on information from VR hand controllers, which do not fully capture the individual movement of the joints. In this paper, we propose a creative VR therapy exergame for upper extremity rehabilitation using multi-dimensional reaching tasks while simultaneously capturing hand movement from the VR controllers and elbow joint movement from a flexible carbon nanotube sleeve. We conducted a preliminary study with non-clinical participants (n = 12, 7 F). In a 2x2 within-subjects study (orientation (vertical, horizontal) x configuration (flat, curved)), we evaluated the effectiveness and enjoyment of the exergame in different study conditions. The results show that there was a statistically significant difference in terms of task completion time between the two orientations. However, no significant differences were found in the number of mistakes in both orientation and configuration of the virtual exergame. This can lead to customizing therapy while maintaining the same level of intensity. That is, if a patient has
Despite the high prevalence and burden of mental health conditions, there is a global shortage of mental health providers. Artificial Intelligence (AI) methods have been proposed as a way to address this shortage, by supporting providers with less extensive training as they deliver care. To this end, we developed the AI-Assisted Provider Platform (A2P2), a text-based virtual therapy interface that includes a response suggestion feature, which supports providers in delivering protocolized therapies empathetically. We studied providers with and without expertise in mental health treatment delivering a therapy session using the platform with (intervention) and without (control) AI-assistance features. Upon evaluation, the AI-assisted system significantly decreased response times by 29.34% (p=0.002), tripled empathic response accuracy (p=0.0001), and increased goal recommendation accuracy by 66.67% (p=0.001) across both user groups compared to the control. Both groups rated the system as having excellent usability.
The term bacteriophage means killer or eater of bacteria. They were initially discovered by F.W. Twort and later on, Felix d'Herelle unveiled them to the world in 1910. Phage therapy has arisen as a favorable option to conventional antibiotics by reducing the multinational problem of increasing antibacterial resistance. These virulent viruses particularly prey on and contaminate bacterial strains and aid in fighting bacterial diseases. Researchers are performing various clinical trials on the bacteriophage to tackle pathogenic bacterial infections, varying from typical illnesses to highly invulnerable biofilms that cannot be treated with antibiotics. The primary experiments demonstrated that phage therapy has fewer consequences than traditional antimicrobial drugs. It is safer to use and show results within a few days. Although phage therapy has a wide range of promising results, but it also encounters diverse obstacles. One is that they are host-specific and can merely be used for personalized therapy. As thousands of bacteria can cause disease, clinicians have to construct a library of phage viruses. For successful treatment, an analysis of versatility, stability, and immune inte