搜索结果：movement

Understanding camera movement in natural language is critical for training and evaluating video generation models, among other applications. However, we demonstrate that existing vision-language models (VLMs) fail this task in surprising ways, frequently confusing translation with rotation, left with right, and object movement with camera movement. To address these limitations, we establish natural language camera movement understanding as a standalone research task. We introduce a two-level cinematographic taxonomy and an extensive, atomic benchmark featuring both real and synthetic videos. Furthermore, we curate a large-scale, multi-source training set enhanced by targeted camera movement augmentation. Our fine-tuned VLM-8B outperforms Gemini 3.1 Pro by 10% and 11% on our benchmark's real and synthetic videos, respectively. Despite these gains, a significant gap remains relative to human performance, underscoring the need to promote and facilitate future research on natural language camera movement understanding.

Biological systems exhibit a continuous stream of movements, consisting of sequential segments, that allow them to perform complex tasks in a creative and versatile fashion. This observation has led researchers towards identifying elementary building blocks of motion known as movement primitives, which are well-suited for generating motor commands in autonomous systems, such as robots. In this survey, we provide an encyclopedic overview of movement primitive approaches and applications in chronological order. Concretely, we present movement primitive frameworks as a way of representing robotic control trajectories acquired through human demonstrations. Within the area of robotics, movement primitives can encode basic motions at the trajectory level, such as how a robot would grasp a cup or the sequence of motions necessary to toss a ball. Furthermore, movement primitives have been developed with the desirable analytical properties of a spring-damper system, probabilistic coupling of multiple demonstrations, using neural networks in high-dimensional systems, and more, to address difficult challenges in robotics. Although movement primitives have widespread application to a variety o

Recent advances in machine learning have dramatically improved our ability to model language, vision, and other high-dimensional data, yet they continue to struggle with one of the most fundamental aspects of biological systems: movement. Across neuroscience, medicine, robotics, and ethology, movement is essential for interpreting behavior, predicting intent, and enabling interaction. Despite its core significance in our intelligence, movement is often treated as an afterthought rather than as a rich and structured modality in its own right. This reflects a deeper fragmentation in how movement data is collected and modeled, often constrained by task-specific goals and domain-specific assumptions. But movement is not domain-bound. It reflects shared physical constraints, conserved morphological structures, and purposeful dynamics that cut across species and settings. We argue that movement should be treated as a primary modeling target for AI. It is inherently structured and grounded in embodiment and physics. This structure, often allowing for compact, lower-dimensional representations (e.g., pose), makes it more interpretable and computationally tractable to model than raw, high-d

The #StopAsianHate (SAH) movement is a broad social movement against violence targeting Asians and Asian Americans, beginning in 2021 in response to racial discrimination related to COVID-19 and sparking worldwide conversation about anti-Asian hate. However, research on the online SAH movement has focused on English-speaking participants so the spread of the movement outside of the United States is largely unknown. In addition, there have been no long-term studies of SAH so the extent to which it has been successfully sustained over time is not well understood. We present an analysis of 6.5 million "#StopAsianHate" tweets from 2.2 million users all over the globe and spanning 60 different languages, constituting the first study of the non-English and transnational component of the online SAH movement. Using a combination of topic modeling, user modeling, and hand annotation, we identify and characterize the dominant discussions and users participating in the movement and draw comparisons of English versus non-English topics and users. We discover clear differences in events driving topics, where spikes in English tweets are driven by violent crimes in the US but spikes in non-Engli

The biomechanics of the human body allow humans a range of possible ways of executing movements to attain specific goals. Nevertheless, humans exhibit significant patterns in how they execute movements. We propose that the observed patterns of human movement arise because subjects select those ways to execute movements that are, in a rigorous sense, optimal. In this project, we show how this proposition can guide the development of computational models of movement selection and thereby account for human movement patterns. We proceed by first developing a movement utility formalism that operationalizes the concept of a best or optimal way of executing a movement using a utility function so that the problem of movement selection becomes the problem of finding the movement that maximizes the utility function. Since the movement utility formalism includes a contribution of the metabolic energy of the movement (maximum utility movements try to minimize metabolic energy), we also develop a metabolic energy formalism that we can use to construct estimators of the metabolic energies of particular movements. We then show how we can construct an estimator for the metabolic energies of normal

In non-invasive brain-computer interface systems, pre-movement decoding plays an important role in the detection of movement before limbs actually move. Movement-related cortical potential is a kind of brain activity associated with pre-movement decoding. In current studies, patterns decoded from movement are mainly applied to the binary classification between movement state and resting state, such as elbow flexion and rest. The classifications between two movement states and among multiple movement states are still challenging. This study proposes a new method, the star-arrangement spectral filtering (SASF), to solve the multi-class pre-movement classification problem. We first design a referenced task-related component analysis (RTRCA) framework that consists of two modules. This first module is the classification between movement state and resting state; the second module is the classification of multiple movement states. SASF is developed by optimizing the features in RTRCA. In SASF, feature selection on filter banks is used on the first module of RTRCA, and feature selection on time windows is used on the second module of RTRCA. A linear discriminant analysis classifier is use

Large vision-language models have achieved remarkable capabilities by training on massive internet-scale data, yet a fundamental asymmetry persists: while LLMs can leverage self-supervised pretraining on abundant text and image data, the same is not true for many behavioral modalities. Video-based behavioral data -- gestures, eye movements, social signals -- remains scarce, expensive to annotate, and privacy-sensitive. A promising alternative is simulation: replace real data collection with controlled synthetic generation to produce automatically labeled data at scale. We introduce infrastructure for this paradigm applied to eye movement, a behavioral signal with applications across vision-language modeling, virtual reality, robotics, accessibility systems, and cognitive science. We present a pipeline for generating synthetic labeled eye movement video by extracting real human iris trajectories from reference videos and replaying them on a 3D eye movement simulator via headless browser automation. Applying this to the task of script-reading detection during video interviews, we release final_dataset_v1: 144 sessions (72 reading, 72 conversation) totaling 12 hours of synthetic eye m

Wild animals are commonly fitted with trackers that record their position through time, and statistical models for tracking data broadly fall into two categories: models focused on small-scale movement decisions, and models for large-scale spatial distributions. Due to this dichotomy, it is challenging to describe mathematically how animals' distributions arise from their short-term movement patterns, and to combine data sets collected at different scales. We propose a multiscale model of animal movement and space use based on the underdamped Langevin process, widely used in statistical physics. The model is convenient to describe animal movement for three reasons: it is specified in continuous time (such that its parameters are not dependent on an arbitrary time scale), its speed and direction are autocorrelated (similarly to real animal trajectories), and it has a closed form stationary distribution that we can view as a model of long-term space use. We use the common form of a resource selection function for the stationary distribution, to model the environmental drivers behind the animal's movement decisions. We further increase flexibility by allowing movement parameters to be

We record and analyze the movement patterns of the marsupial {\it Didelphis aurita} at different temporal scales. Animals trajectories are collected at a daily scale by using spool-and-line techniques, and with the help of radio-tracking devices animals traveled distances are estimated at intervals of weeks. Small-scale movements are well described by truncated Lévy flight, while large-scale movements produce a distribution of distances which is compatible with a Brownian motion. A model of the movement behavior of these animals, based on a truncated Lévy flight calibrated on the small scale data, converges towards a Brownian behavior after a short time interval of the order of one week. These results show that whether Lévy flight or Brownian motion behaviors apply, will depend on the scale of aggregation of the animals paths. In this specific case, as the effect of the rude truncation present in the daily data generates a fast convergence towards Brownian behaviors, Lévy flights become of scarce interest for describing the local dispersion properties of these animals, which result well approximated by a normal diffusion process and not a fast, anomalous one. Interestingly, we are

Exploration in deep reinforcement learning (RL) is commonly implemented as temporally uncorrelated white noise. However, recent works show that temporally correlated colored noise can improve exploration efficiency by producing smooth trajectories with better coverage of the state space. We inquire whether action noise inspired by infant spontaneous movements can also improve exploration in deep RL. We find that the power spectral densities of babies' end-effector velocities follow a colored noise process where the spectral exponent increases with age. Inspired by this developmental pattern, we introduce a mechanism that progressively increases the temporal auto-correlation of exploration noise during RL training, matching the infant statistics. Experiments across several RL environments show that infant-inspired noise produces structured exploratory behavior and can improve learning efficiency compared to conventional exploration strategies. These findings suggest that human motor and cognitive development can provide useful guidance for designing learning mechanisms in artificial agents. Our code is available at https://github.com/trieschlab/baby-noise-rl.

Understanding movement described in text documents is important since text descriptions of movement contain a wealth of geographic and contextual information about the movement of people, wildlife, goods, and much more. Our research makes several contributions to improve our understanding of movement descriptions in text. First, we show how interpreting geographic movement described in text is challenging because of general spatial terms, linguistic constructions that make the thing(s) moving unclear, and many types of temporal references and groupings, among others. Next, as a step to overcome these challenges, we report on an experiment with human subjects through which we identify multiple important characteristics of movement descriptions (found in text) that humans use to differentiate one movement description from another. Based on our empirical results, we provide recommendations for computational analysis using movement described in text documents. Our findings contribute towards an improved understanding of the important characteristics of the underused information about geographic movement that is in the form of text descriptions.

Movement primitives are trainable parametric models that reproduce robotic movements starting from a limited set of demonstrations. Previous works proposed simple linear models that exhibited high sample efficiency and generalization power by allowing temporal modulation of movements (reproducing movements faster or slower), blending (merging two movements into one), via-point conditioning (constraining a movement to meet some particular via-points) and context conditioning (generation of movements based on an observed variable, e.g., position of an object). Previous works have proposed neural network-based motor primitive models, having demonstrated their capacity to perform tasks with some forms of input conditioning or time-modulation representations. However, there has not been a single unified deep motor primitive's model proposed that is capable of all previous operations, limiting neural motor primitive's potential applications. This paper proposes a deep movement primitive architecture that encodes all the operations above and uses a Bayesian context aggregator that allows a more sound context conditioning and blending. Our results demonstrate our approach can scale to repr

To allow the development and assessment of physical rehabilitation by an intelligent tutoring system, we propose a medical dataset of clinical patients carrying out low back-pain rehabilitation exercises and benchmark on state of the art human movement analysis algorithms. This dataset is valuable because it includes rehabilitation motions in a clinical setting with patients in their rehabilitation program. This paper introduces the Keraal dataset, a clinically collected dataset to enable intelligent tutoring systems (ITS) for rehabilitation. It addresses four challenges in exercise monitoring: motion assessment, error recognition, spatial localization, temporal localization

This paper explores the design of an expressive visual instrument that embraces the unique movement style of a dancer living with physical disability. Through a collaboration between the dancer and an interaction designer/visual artist, the creative qualities of wearable devices for motion tracking are investigated, with emphasis on integrating the dancer's specific movement capabilities with their creative goals. The affordances of this technology for imagining new forms of creative expression play a critical role in the design process. These themes are drawn together through an experiential performance which augments an improvised dance with an ephemeral real-time visualisation of the performer's movements. Through practice-based research, the design, development and presentation of this performance work is examined as a 'testbed' for new ideas, allowing for the exploration of HCI concepts within a creative context. This paper outlines the creative process behind the development of the work, the insights derived from the practice-based research enquiry, and the role of movement technology in encouraging new ways of moving through creative expression.

We define a complete metric structure on the family $\text{PL}_q^p(\mathbb{R}^n)$ of probability measures with densities in $L^p(\mathbb{R}^n)$ and finite $q$-moments. We establish the existence of generalized minimizing movements for the isoperimetric ratio and characterize absolutely continuous curves in this space through weak solutions of the continuity equation with velocity fields satisfying a first-order integral condition. We also characterize absolutely continuous curves in the $\infty$-Wasserstein space and prove a Benamou--Brenier formula for $W_\infty$.

While the use of technology to provide accurate and objective measurements of human movement performance is presently an area of great interest, efforts to quantify the performance of movement are hampered by the lack of a principled model that describes how a subject goes about making a movement. We put forward a principled mathematical formalism that describes human movements using an optimal control model in which the subject controls the jerk of the movement. We construct the formalism by assuming that the movement a subject chooses to make is better than the alternatives. We quantify the relative quality of movements mathematically by specifying a cost functional that assigns a numerical value to every possible movement; the subject makes the movement that minimizes the cost functional. We develop the mathematical structure of movements that minimize a cost functional, and observe that this development parallels the development of analytical mechanics from the Principle of Least Action. We derive a constant of the motion for human movements that plays a role that is analogous to the role that the energy plays in classical mechanics. We apply the formalism to the description of

Probabilistic Movement Primitives (ProMPs) are a widely used representation of movements for human-robot interaction. They also facilitate the factorization of temporal and spatial structure of movements. In this work we investigate a method to temporally align observations so that when learning ProMPs, information in the spatial structure of the observed motion is maximized while maintaining a smooth phase velocity. We apply the method on recordings of hand trajectories in a two-dimensional reaching task. A system for simultaneous recognition of movement and phase is proposed and performance of movement recognition and movement reproduction is discussed.

Group dynamic movement is a fundamental aspect of many species' movements. The need to adequately model individuals' interactions with other group members has been recognised, particularly in order to differentiate the role of social forces in individual movement from environmental factors. However, to date, practical statistical methods which can include group dynamics in animal movement models have been lacking. We consider a flexible modelling framework that distinguishes a group-level model, describing the movement of the group's centre, and an individual-level model, such that each individual makes its movement decisions relative to the group centroid. The basic idea is framed within the flexible class of hidden Markov models, extending previous work on modelling animal movement by means of multi-state random walks. While in simulation experiments parameter estimators exhibit some bias in non-ideal scenarios, we show that generally the estimation of models of this type is both feasible and ecologically informative. We illustrate the approach using real movement data from 11 reindeer (Rangifer tarandus). Results indicate a directional bias towards a group centroid for reindeer

Wearable movement data is collected by nearly all commercially available smartwatches and is a valuable resource for mental health research, reflecting fine-grained temporal behavioral trends. Despite its promise, the development of foundation models for health wearable modeling remains limited when compared to clinical image and text analysis. We designed transformers with patch embeddings and used self-supervised masked autoencoder pretraining on minute-level week-long actigraphy (physical activity intensity measurement) sequences to develop and evaluate the Pretrained Actigraphy Transformer (PAT). PAT is an open-source foundation model for wearable movement time series that combines week-long temporal modeling, psychiatric outcome evaluation, and reproducibility on public data. Pretrained on data from 21,538 U.S. participants in a nationally representative cohort from the National Health and Nutrition Examination Survey (NHANES), PAT consistently outperformed non-foundation-model baselines across mental health prediction tasks-including benzodiazepine and SSRI use, depression, and sleep abnormalities. During the benzodiazepine medication usage prediction task, PAT demonstrated t