Dancers often prototype movements themselves or with each other during improvisation and choreography. How are these interactions altered when physically manipulable technologies are introduced into the creative process? To understand how dancers design and improvise movements while working with instruments capable of non-humanoid movements, we engaged dancers in workshops to co-create movements with a robot arm in one-human-to-one-robot and three-human-to-one-robot settings. We found that dancers produced more fluid movements in one-to-one scenarios, experiencing a stronger sense of connection and presence with the robot as a co-dancer. In three-to-one scenarios, the dancers divided their attention between the human dancers and the robot, resulting in increased perceived use of space and more stop-and-go movements, perceiving the robot as part of the stage background. This work highlights how technologies can drive creativity in movement artists adapting to new ways of working with physical instruments, contributing design insights supporting artistic collaborations with non-humanoid agents.
Speech-driven 3D face animation poses significant challenges due to the intricacy and variability inherent in human facial movements. This paper emphasizes the importance of considering both the composite and regional natures of facial movements in speech-driven 3D face animation. The composite nature pertains to how speech-independent factors globally modulate speech-driven facial movements along the temporal dimension. Meanwhile, the regional nature alludes to the notion that facial movements are not globally correlated but are actuated by local musculature along the spatial dimension. It is thus indispensable to incorporate both natures for engendering vivid animation. To address the composite nature, we introduce an adaptive modulation module that employs arbitrary facial movements to dynamically adjust speech-driven facial movements across frames on a global scale. To accommodate the regional nature, our approach ensures that each constituent of the facial features for every frame focuses on the local spatial movements of 3D faces. Moreover, we present a non-autoregressive backbone for translating audio to 3D facial movements, which maintains high-frequency nuances of facial m
Visual impairments create barriers to learning physical activities, since conventional training methods rely on visual demonstrations or often inadequate verbal descriptions. This research explores 3D-printed human body models to enhance movement comprehension for blind individuals. Through a participatory design approach in collaboration with a blind designer, we developed detailed 3D models representing various body movements and incorporated tactile reference elements to enhance spatial understanding. We conducted two user studies with 10 blind participants across different activities: static yoga poses and sequential calisthenic movements. The results demonstrated that 3D models significantly improved understanding speed, reduced questions for clarification, and enhanced movement accuracy compared to conventional teaching methods. Participants consistently rated 3D models higher for ease of understanding, effectiveness, and motivation.
The present work proposes a Deep Learning architecture for the prediction of various consumer choice behaviors from time series of raw gaze or eye fixations on images of the decision environment, for which currently no foundational models are available. The architecture, called STARE (Spatio-Temporal Attention Representation for Eye Tracking), uses a new tokenization strategy, which involves mapping the x- and y- pixel coordinates of eye-movement time series on predefined, contiguous Regions of Interest. That tokenization makes the spatio-temporal eye-movement data available to the Chronos, a time-series foundation model based on the T5 architecture, to which co-attention and/or cross-attention is added to capture directional and/or interocular influences of eye movements. We compare STARE with several state-of-the art alternatives on multiple datasets with the purpose of predicting consumer choice behaviors from eye movements. We thus make a first step towards developing and testing DL architectures that represent visual attention dynamics rooted in the neurophysiology of eye movements.
In geographic data videos, camera movements are frequently used and combined to present information from multiple perspectives. However, creating and editing camera movements requires significant time and professional skills. This work aims to lower the barrier of crafting diverse camera movements for geographic data videos. First, we analyze a corpus of 66 geographic data videos and derive a design space of camera movements with a dimension for geospatial targets and one for narrative purposes. Based on the design space, we propose a set of adaptive camera shots and further develop an interactive tool called GeoCamera. This interactive tool allows users to flexibly design camera movements for geographic visualizations. We verify the expressiveness of our tool through case studies and evaluate its usability with a user study. The participants find that the tool facilitates the design of camera movements.
As a communication channel, body movements have been widely explored in behavioral studies and kinesics. Performing and visual arts share the same interests but focus on documenting and representing human body movements, such as for dance notation and visual work creation. This paper investigates body movements in oriental calligraphy and how to apply calligraphy principles to stimulate and archive body movements. Through an artwork (Wushu), the authors experiment with an interactive and generative approach to engage the audience's bodily participation and archive the body movements as a compendium of generated calligraphy. The audience assumes the role of both writers and readers; creating ("writing") and appreciating ("reading") the generated calligraphy becomes a cyclical process within this infinite "Book," which can motivate further attention and discussions concerning Chinese characters and calligraphy.
In this paper, we present an acoustic side channel attack which makes use of smartphone microphones recording a robot in operation to exploit acoustic properties of the sound to fingerprint a robot's movements. In this work we consider the possibility of an insider adversary who is within physical proximity of a robotic system (such as a technician or robot operator), equipped with only their smartphone microphone. Through the acoustic side-channel, we demonstrate that it is indeed possible to fingerprint not only individual robot movements within 3D space, but also patterns of movements which could lead to inferring the purpose of the movements (i.e. surgical procedures which a surgical robot is undertaking) and hence, resulting in potential privacy violations. Upon evaluation, we find that individual robot movements can be fingerprinted with around 75% accuracy, decreasing slightly with more fine-grained movement meta-data such as distance and speed. Furthermore, workflows could be reconstructed with around 62% accuracy as a whole, with more complex movements such as pick-and-place or packing reconstructed with near perfect accuracy. As well as this, in some environments such as
The influence of individual differences on the perception and evaluation of interactions with robots has been researched for decades. Some human demographic characteristics have been shown to affect how individuals perceive interactions with robots. Still, it is to-date not clear whether, which and to what extent individual differences influence how we perceive robots, and even less is known about human factors and their effect on the perception of robot movements. In addition, most results on the relevance of individual differences investigate human-robot interactions with humanoid or social robots whereas interactions with industrial robots are underrepresented. We present a literature review on the relationship of robot movements and the influence of demographic variation. Our review reveals a limited comparability of existing findings due to a lack of standardized robot manipulations, various dependent variables used and differing experimental setups including different robot types. In addition, most studies have insufficient sample sizes to derive generalizable results. To overcome these shortcomings, we report the results from a Web-based experiment with 930 participants that
By employing the closest point method, we extend the applicability of minimizing movements to the surface PDE setting. The corresponding approximation methods are created, and their convergence is observed. The numerical methods are then used to develop threshold dynamics algorithms for surface-constrained interfacial motions. In particular, show how the minimizing movements enable one to approximate multiphase, volume-preserving, curvature flows on surfaces via generalized MBO and HMBO algorithms.
From the urbanists' perspective, the everyday experience of young people, as an underrepresented group in the design of public spaces, includes tactics they use to challenge the strategies which rule over urban spaces. In this regard, youth led social movements are a set of collective tactics which groups of young people use to resist power structures. Social informational streams have revolutionized the way youth organize and mobilize for social movements throughout the world, especially in urban areas. However, just like public spaces, these algorithm based platforms have been developed with a great power imbalance between the developers and users which results in the creation of non inclusive social informational streams for young activists. Social activism grows agency and confidence in youth which is critical to their development. This paper employs a youth centric lens, which is used in designing public spaces, for designing algorithmic spaces that can improve bottom up youth led movements. By reviewing the structure of these spaces and how young people interact with these structures in the different cultural contexts of Iran and the US, we propose a humanistic approach to de
From smoothly pursuing moving objects to rapidly shifting gazes during visual search, humans employ a wide variety of eye movement strategies in different contexts. While eye movements provide a rich window into mental processes, building generative models of eye movements is notoriously difficult, and to date the computational objectives guiding eye movements remain largely a mystery. In this work, we tackled these problems in the context of a canonical spatial planning task, maze-solving. We collected eye movement data from human subjects and built deep generative models of eye movements using a novel differentiable architecture for gaze fixations and gaze shifts. We found that human eye movements are best predicted by a model that is optimized not to perform the task as efficiently as possible but instead to run an internal simulation of an object traversing the maze. This not only provides a generative model of eye movements in this task but also suggests a computational theory for how humans solve the task, namely that humans use mental simulation.
Humans constantly move their eyes, even during visual fixations, where miniature (or fixational) eye movements occur involuntarily. Fixational eye movements comprise slow components (physiological drift and tremor) and fast components (microsaccades). The complex dynamics of physiological drift can be modeled qualitatively as a statistically self-avoiding random walk (SAW model, Engbert, Mergenthaler, Sinn, & Pikovsky, 2011). In this study, we implement a data assimilation approach for the SAW model to explain statistics of fixational eye movements and microsaccades in experimental data obtained from high-resolution eye-tracking. We discuss and analyze the likelihood function for the SAW model, which allows us to apply Bayesian parameter estimation at the level of individual human observers. Based on model fitting, we find a relationship between the activation predicted by the SAW model and the occurrence of microsaccades. The model's latent activation relative to microsaccade onsets and offsets using experimental data lends support to the existence of a triggering mechanism for microsaccades. Our findings suggest that the SAW model can capture individual differences and serve
The end-user programming of social robot behavior is usually limited by a predefined set of movements. We are proposing a puppeteering robotic interface that provides a more intuitive method of programming robot expressive movements. As the user manipulates the puppet of a robot, the actual robot replicates the movements, providing real-time visual feedback. Through this proposed interface, even with limited training, a novice user can design and program expressive movements efficiently. We present our preliminary user study results in this extended abstract.
Eye movements during fixation of a stationary target prevent the adaptation of the photoreceptors to continuous illumination and inhibit fading of the image. These random, involuntary, small, movements are restricted at long time scales so as to keep the target at the center of the field of view. Here we use the Detrended Fluctuation Analysis (DFA) in order to study the properties of fixational eye movements at different time scales. Results show different scaling behavior between horizontal and vertical movements. When the small ballistics movements, i.e. micro-saccades, are removed, the scaling exponents in both directions become similar. Our findings suggest that micro-saccades enhance the persistence at short time scales mostly in the horizontal component and much less in the vertical component. This difference may be due to the need of continuously moving the eyes in the horizontal plane, in order to match the stereoscopic image for different viewing distance.
Crowd simulation is used for evacuation and crowd safety inspections, study of performance in crowd systems and animations. Cellular automata has been extensively used in modelling the crowd. In regular cellular automata models, each pedestrian occupies a single cell with the size of a pedestrian body. Since the space is divided into relatively large cells, the movements of pedestrians look like the movements of pieces on a chess board. Furthermore, all pedestrians have the same body size and speed. In this paper, a method called fine grid cellular automata is proposed in which smaller cells are used and pedestrian body may occupy several cells. The model allows the use of different body sizes, shapes and speeds for pedestrian. The proposed model is used for simulating movements of pedestrians toward a target. A typical walkway scenario is used to test and evaluate the model. The movements of pedestrians are smoother because of the finer grain discretization of movements and the simulation results match empirical speed-density graphs with good accuracy.
In the dominant livestock systems of Sahelian countries herds have to move across territories. Their mobility is often a source of conflict with farmers in the areas crossed, and helps spread diseases such as Rift Valley Fever. Knowledge of the routes followed by herds is therefore core to guiding the implementation of preventive and control measures for transboundary animal diseases, land use planning and conflict management. However, the lack of quantitative data on livestock movements, together with the high temporal and spatial variability of herd movements, has so far hampered the production of fine resolution maps of animal movements. This paper proposes a general framework for mapping potential paths for livestock movements and identifying areas of high animal passage potential for those movements. The method consists in combining the information contained in livestock mobility networks with landscape connectivity, based on different mobility conductance layers. We illustrate our approach with a livestock mobility network in Senegal and Mauritania in the 2014 dry and wet seasons.
Rapid, targeted hand movements exhibit a regular movement pattern described by Fitts law. We develop a model of these movements in which this movement pattern results from an optimal control model describing rapid hand movements and a utility model describing the speed/accuracy trade-off between moving the hand rapidly to the target and hitting the target accurately. The optimal control model is constructed using principled approach in which we forbid the muscle forces to exhibit any discontinuities and require the cost to be expressed in terms of a psychophysical representation of the movement. This yields a yank-control or jerk-control model of the movement which exhibits two constants of the motion that are closely related to the energy and momentum in classical mechanics. We force the optimal control model to obey Fitts law by requiring a particular relationship hold between the constants of the motion and the size of the target and show that the resulting model compares well to a standard expression of Fitts law obtained empirically using observations of computer mouse movements. We then proceed to further show how this relationship may be obtained as the result of a simple mo
Inertial measurement units (IMUs) increasingly function as a basic component of wearable sensor network (WSN)systems. IMU-based joint angle estimation (JAE) is a relatively typical usage of IMUs, with extensive applications. However, the issue that IMUs move with respect to their original placement during JAE is still a research gap, and limits the robustness of deploying the technique in real-world application scenarios. In this study, we propose to detect and correct the IMU movement online in a relatively computationally lightweight manner. Particularly, we first experimentally investigate the influence of IMU movements. Second, we design the metrics for detecting IMU movements by mathematically formulating how the IMU movement affects the IMU measurements. Third, we determine the optimal thresholds of metrics by synthetic IMU data from a significantly amended simulation model. Finally, a correction method is proposed to correct the effects of IMU movements. We demonstrate our method on both synthetic data and real-user data. The results demonstrate our method is a promising solution to detecting and correcting IMU movements during JAE.
Slow and accurate finger and limb movements are essential to daily activities, but their neural control and governing mechanics are relatively unexplored. We consider neuromechanical systems where slow movements are produced by neural commands that modulate muscle stiffness. This formulation based on strain-energy equilibria is in agreement with prior work on neural control of muscle and limb impedance. Slow limb movements are driftless in the sense that movement stops when neural commands stop. We demonstrate, in the context of two planar tendon-driven systems representing a finger and a leg, that the control of muscle stiffness suffices to produce stable and accurate limb postures and quasi-static (slow) transitions among them. We prove, however, that stable postures are achievable only when muscles are pre-tensioned, as is the case for natural muscle tone. Our results further indicate, in accordance with experimental findings, that slow movements are non-smooth. The non-smoothness arises because the precision with which individual muscle stiffnesses need to be controlled changes substantially throughout the limb's motion. These results underscore the fundamental roles of muscle
High-frequency gaze data contains more user-specific information than low-frequency data, promising for various applications. However, existing gaze modelling methods focus on low-frequency data, ignoring user-specific subtle eye movements in high-frequency eye movements. We present DiffEyeSyn -- the first computational method to synthesise eye movements specific to individual users. The key idea is to consider the user-specific information as a special type of noise in eye movement data. This perspective reshapes eye movement synthesis into the task of injecting this user-specific noise into any given eye movement sequence. We formulate this injection task as a conditional diffusion process in which the synthesis is conditioned on user-specific embeddings extracted from the gaze data using pre-trained models for user authentication. We propose user identity guidance -- a novel loss function that allows our model to preserve user identity while generating human-like eye movements in the spatial domain. Experiments on two public datasets show that our synthetic eye movements preserve user-specific characteristics and are more realistic than baseline approaches. Furthermore, we demon