搜索结果：Time

We formulate long-context language modeling as a problem in continual learning rather than architecture design. Under this formulation, we only use a standard architecture -- a Transformer with sliding-window attention. However, our model continues learning at test time via next-token prediction on the given context, compressing the context it reads into its weights. In addition, we improve the model's initialization for learning at test time via meta-learning at training time. Overall, our method, a form of Test-Time Training (TTT), is End-to-End (E2E) both at test time (via next-token prediction) and training time (via meta-learning), in contrast to previous forms. We conduct extensive experiments with a focus on scaling properties. In particular, for 3B models trained with 164B tokens, our method (TTT-E2E) scales with context length in the same way as Transformer with full attention, while others, such as Mamba 2 and Gated DeltaNet, do not. However, similar to RNNs, TTT-E2E has constant inference latency regardless of context length, making it 2.7 times faster than full attention for 128K context. Our code is publicly available.

Recently, there has been a great deal of attention in a class of controllers based on time-varying gains, called prescribed-time controllers, that steer the system's state to the origin in the desired time, a priori set by the user, regardless of the initial condition. Furthermore, such a class of controllers has been shown to maintain a prescribed-time convergence in the presence of disturbances even if the disturbance bound is unknown. However, such properties require a time-varying gain that becomes singular at the terminal time, which limits its application to scenarios under quantization or measurement noise. This chapter presents a methodology to design a broader class of controllers, called predefined-time controllers, with a prescribed convergence-time bound. Our approach allows designing robust predefined-time controllers based on time-varying gains while maintaining uniformly bounded time-varying gains. We analyze the condition for uniform Lyapunov stability under the proposed time-varying controllers.

Large Language Models (LLMs) have recently demonstrated a remarkable ability to model time series data. These capabilities can be partly explained if LLMs understand basic time series concepts. However, our knowledge of what these models understand about time series data remains relatively limited. To address this gap, we introduce TimeSeriesExam, a configurable and scalable multiple-choice question exam designed to assess LLMs across five core time series understanding categories: pattern recognition, noise understanding, similarity analysis, anomaly detection, and causality analysis. TimeSeriesExam comprises of over 700 questions, procedurally generated using 104 carefully curated templates and iteratively refined to balance difficulty and their ability to discriminate good from bad models. We test 7 state-of-the-art LLMs on the TimeSeriesExam and provide the first comprehensive evaluation of their time series understanding abilities. Our results suggest that closed-source models such as GPT-4 and Gemini understand simple time series concepts significantly better than their open-source counterparts, while all models struggle with complex concepts such as causality analysis. We be

How can we use AI to discover a new state of the art for a scientific problem? Prior work in test-time scaling, such as AlphaEvolve, performs search by prompting a frozen LLM. We perform reinforcement learning at test time, so the LLM can continue to train, but now with experience specific to the test problem. This form of continual learning is quite special, because its goal is to produce one great solution rather than many good ones on average, and to solve this very problem rather than generalize to other problems. Therefore, our learning objective and search subroutine are designed to prioritize the most promising solutions. We call this method Test-Time Training to Discover (TTT-Discover). Following prior work, we focus on problems with continuous rewards. We report results for every problem we attempted, across mathematics, GPU kernel engineering, algorithm design, and biology. TTT-Discover sets the new state of the art in almost all of them: (i) Erdős' minimum overlap problem and an autocorrelation inequality; (ii) a GPUMode kernel competition (up to $2\times$ faster than prior art); (iii) past AtCoder algorithm competitions; and (iv) denoising problem in single-cell analysi

Current theoretical physics suggests the flow of time is an illusion: the entire universe just is, with no special meaning attached to the present time. This paper points out that this view, in essence represented by usual space-time diagrams, is based on time-reversible microphysical laws, which fail to capture essential features of the time-irreversible nature of decoherence and the quantum measurement process, as well as macro-physical behaviour and the development of emergent complex systems, including life, which exist in the real universe. When these are taken into account, the unchanging block universe view of spacetime is best replaced by an evolving block universe which extends as time evolves, with the potential of the future continually becoming the certainty of the past; spacetime itself evolves, as do the entities within it. However this time evolution is not related to any preferred surfaces in spacetime; rather it is associated with the evolution of proper time along families of world lines. The default state of fundamental physics should not be taken to be a time irreversible evolution of physical states: it is an ongoing irreversible development of time itself.

Time series forecasting remains a challenging task, particularly in the context of complex multiscale temporal patterns. This study presents LLM-Mixer, a framework that improves forecasting accuracy through the combination of multiscale time-series decomposition with pre-trained LLMs (Large Language Models). LLM-Mixer captures both short-term fluctuations and long-term trends by decomposing the data into multiple temporal resolutions and processing them with a frozen LLM, guided by a textual prompt specifically designed for time-series data. Extensive experiments conducted on multivariate and univariate datasets demonstrate that LLM-Mixer achieves competitive performance, outperforming recent state-of-the-art models across various forecasting horizons. This work highlights the potential of combining multiscale analysis and LLMs for effective and scalable time-series forecasting.

We introduce bounded generation as a generalized task to control video generation to synthesize arbitrary camera and subject motion based only on a given start and end frame. Our objective is to fully leverage the inherent generalization capability of an image-to-video model without additional training or fine-tuning of the original model. This is achieved through the proposed new sampling strategy, which we call Time Reversal Fusion, that fuses the temporally forward and backward denoising paths conditioned on the start and end frame, respectively. The fused path results in a video that smoothly connects the two frames, generating inbetweening of faithful subject motion, novel views of static scenes, and seamless video looping when the two bounding frames are identical. We curate a diverse evaluation dataset of image pairs and compare against the closest existing methods. We find that Time Reversal Fusion outperforms related work on all subtasks, exhibiting the ability to generate complex motions and 3D-consistent views guided by bounded frames. See project page at https://time-reversal.github.io.

Recent studies have suggested frequency-domain Data augmentation (DA) is effec tive for time series prediction. Existing frequency-domain augmentations disturb the original data with various full-spectrum noises, leading to excess domain gap between augmented and original data. Although impressive performance has been achieved in certain cases, frequency-domain DA has yet to be generalized to time series prediction datasets. In this paper, we found that frequency-domain augmentations can be significantly improved by two modifications that limit the perturbations. First, we found that limiting the perturbation to only dominant frequencies significantly outperforms full-spectrum perturbations. Dominant fre quencies represent the main periodicity and trends of the signal and are more important than other frequencies. Second, we found that simply shuffling the dominant frequency components is superior over sophisticated designed random perturbations. Shuffle rearranges the original components (magnitudes and phases) and limits the external noise. With these two modifications, we proposed dominant shuffle, a simple yet effective data augmentation for time series prediction. Our method i

Vehicular Edge Computing (VEC) has emerged as a promising paradigm for enhancing the computational efficiency and service quality in intelligent transportation systems by enabling vehicles to wirelessly offload computation-intensive tasks to nearby Roadside Units. However, efficient task offloading and resource allocation for time-critical applications in VEC remain challenging due to constrained network bandwidth and computational resources, stringent task deadlines, and rapidly changing network conditions. To address these challenges, we formulate a Deadline-Constrained Task Offloading and Resource Allocation Problem (DOAP), denoted as $\mathbf{P}$, in VEC with both bandwidth and computational resource constraints, aiming to maximize the total vehicle utility. To solve $\mathbf{P}$, we propose $\mathtt{SARound}$, an approximation algorithm based on Linear Program rounding and local-ratio techniques, that improves the best-known approximation ratio for DOAP from $\frac{1}{6}$ to $\frac{1}{4}$. Additionally, we design an online service subscription and offloading control framework to address the challenges of short task deadlines and rapidly changing wireless network conditions. To

The problem of the Nature of Time is twofold: whether or not time is a fundamental quantity of Nature, and how does clock time of metrology emerge in the experimental description of dynamics. This work strongly supports the fundamental timelessness of Nature. However, the correct view that physics is described by relations between variables does not address the second problem of how time does emerge at the macroscopic scale on the ground of a timeless framework. In this work ordinary Hamiltonian dynamics is recast in a timeless formalism capable to provide a definition of parameter time on the basis of the only generalized coordinates, together with the Hamiltonian invariance on trajectories, and a variational principle. Next, by relaxing the assumption of periodicity of real clocks to the only cyclicity in the phase space, the second problem is addressed. Physical systems, if complex enough, can be separated in a subsystem whose dynamics is described, and another cyclic subsystem which behaves as a clock. The dynamics of the first is mapped in the states of the second cyclic subsystem which provides a discrete approximation of the parameter time. Useful clocks fulfill a stability

A reliable and efficient representation of multivariate time series is crucial in various downstream machine learning tasks. In multivariate time series forecasting, each variable depends on its historical values and there are inter-dependencies among variables as well. Models have to be designed to capture both intra- and inter-relationships among the time series. To move towards this goal, we propose the Time Series Attention Transformer (TSAT) for multivariate time series representation learning. Using TSAT, we represent both temporal information and inter-dependencies of multivariate time series in terms of edge-enhanced dynamic graphs. The intra-series correlations are represented by nodes in a dynamic graph; a self-attention mechanism is modified to capture the inter-series correlations by using the super-empirical mode decomposition (SMD) module. We applied the embedded dynamic graphs to times series forecasting problems, including two real-world datasets and two benchmark datasets. Extensive experiments show that TSAT clearly outerperforms six state-of-the-art baseline methods in various forecasting horizons. We further visualize the embedded dynamic graphs to illustrate th

It is often said that in general relativity time does not exist. This is because the Einstein equations generate motion in time that is a symmetry of the theory, not true time evolution. In quantum gravity, the timelessness of general relativity clashes with time in quantum theory and leads to the ``problem of time'' which, in its various forms, is the main obstacle to a successful quantum theory of gravity. I argue that the problem of time is a paradox, stemming from an unstated faulty premise. Our faulty assumption is that space is real. I propose that what does not fundamentally exist is not time but space, geometry and gravity. The quantum theory of gravity will be spaceless, not timeless. If we are willing to throw out space, we can keep time and the trade is worth it.

The conceptual definition and understanding of the nature of time, both qualitatively and quantitatively is of the utmost difficulty and importance, and plays a fundamental role in physics. Physical systems seem to evolve in paths of increasing entropy and of complexity, and thus, the arrow of time shall be explored in the context of thermodynamic irreversibility and quantum physics. In Newtonian physics, time flows at a constant rate, the same for all observers; however, it necessarily flows at different rates for different observers in special and general relativity. Special relativity provides important quantitative elucidations of the fundamental processes related to time dilation effects, and general relativity provides a deep analysis to effects of time flow, such as in the presence of gravitational fields. Through the special theory of relativity, time became intimately related with space, giving rise to the notion of spacetime, in which both parameters cannot be considered as separate entities. As time is incorporated into the proper structure of the fabric of spacetime, it is interesting to note that general relativity is contaminated with non-trivial geometries that gener

A novel methodology is proposed for clustering multivariate time series data using energy distance defined in Székely and Rizzo (2013). Specifically, a dissimilarity matrix is formed using the energy distance statistic to measure separation between the finite dimensional distributions for the component time series. Once the pairwise dissimilarity matrix is calculated, a hierarchical clustering method is then applied to obtain the dendrogram. This procedure is completely nonparametric as the dissimilarities between stationary distributions are directly calculated without making any model assumptions. In order to justify this procedure, asymptotic properties of the energy distance estimates are derived for general stationary and ergodic time series. The method is illustrated in a simulation study for various component time series that are either linear or nonlinear. Finally the methodology is applied to two examples; one involves GDP of selected countries and the other is population size of various states in the U.S.A. in the years 1900 -1999.

I dissent from the standard assertion of a "Two Times Problem," in which physical time is taken as being at odds with the human sense of a "flow of time." I provide a brief overview of the case to be made for the contrary view: namely, that physical theory is indeed consistent with a genuine temporal dynamism that takes into account the quantum level in connection with spacetime emergence, the latter being supervenient on specific quantum processes.

We study the large-time asymptotic of renewal-reward processes with a heavy-tailed waiting time distribution. It is known that the heavy tail of the distribution produces an extremely slow dynamics, resulting in a singular large deviation function. When the singularity takes place, the bottom of the large deviation function is flattened, manifesting anomalous fluctuations of the renewal-reward processes. In this article, we aim to study how these singularities emerge as the time increases. Using a classical result on the sum of random variables with regularly varying tail, we develop an expansion approach to prove an upper bound of the finite-time moment generating function for the Pareto waiting time distribution (power law) with an integer exponent. We perform numerical simulations using Pareto (with a real value exponent), inverse Rayleigh and log-normal waiting time distributions, and demonstrate similar results are anticipated in these waiting time distributions.

One of the most distinguished features of our algebraic geometrical, pencil concept of space-time is the fact that spatial dimensions and time stand, as far as their intrinsic structure is concerned, on completely different footings: the former being represented by pencils of lines, the latter by a pencil of conics. As a consequence, we argue that even at the classical (macroscopic) level there exists a much more intricate and profound coupling between space and time than that dictated by (general) relativity theory. It is surmised that this coupling can be furnished by so-called Cremona (or birational) transformations between two projective spaces of three dimensions, being fully embodied in the structure of configurations of their fundamental elements. We review properties of some of the simplest Cremona transformations and show that the corresponding "fundamental" space-times exhibit an intimate connection between the extrinsic geometry of time dimension and the dimensionality of space. Moreover, these Cremonian space-times seem to provide us with a promising conceptual basis for the possible reconciliation between two extreme concepts of (space-)time, viz. physical and psycholo

In most image retrieval systems, images include various high-level semantics, called tags or annotations. Virtually all the state-of-the-art image annotation methods that handle imbalanced labeling are search-based techniques which are time-consuming. In this paper, a novel coupled dictionary learning approach is proposed to learn a limited number of visual prototypes and their corresponding semantics simultaneously. This approach leads to a real-time image annotation procedure. Another contribution of this paper is that utilizes a marginalized loss function instead of the squared loss function that is inappropriate for image annotation with imbalanced labels. We have employed a marginalized loss function in our method to leverage a simple and effective method of prototype updating. Meanwhile, we have introduced ${\ell}_1$ regularization on semantic prototypes to preserve the sparse and imbalanced nature of labels in learned semantic prototypes. Finally, comprehensive experimental results on various datasets demonstrate the efficiency of the proposed method for image annotation tasks in terms of accuracy and time. The reference implementation is publicly available on https://github

We introduce the notion of the relaxation time for noisy quantum maps on the 2d-dimensional torus - a generalization of previously studied dissipation time. We show that relaxation time is sensitive to the chaotic behavior of the corresponding classical system if one simultaneously considers the semiclassical limit ($\hbar$ -> 0) together with the limit of small noise strength ($\ep$ -> 0). Focusing on quantized smooth Anosov maps, we exhibit a semiclassical regime $\hbar<\ep^{E}$ << 1 (where E>1) in which classical and quantum relaxation times share the same asymptotics: in this regime, a quantized Anosov map relaxes to equilibrium fast, as the classical map does. As an intermediate result, we obtain rigorous estimates of the quantum-classical correspondence for noisy maps on the torus, up to times logarithmic in $\hbar^{-1}$. On the other hand, we show that in the ``quantum regime'' $\ep$ << $\hbar$ << 1, quantum and classical relaxation times behave very differently. In the special case of ergodic toral symplectomorphisms (generalized ``Arnold's cat'' maps), we obtain the exact asymptotics of the quantum relaxation time and precise the regime of cor