搜索结果：delayed

Multi-agent large language model (LLM) systems often rely on verifier and critic agents to suppress hallucinations, but verification is delayed. During this delay, false claims can propagate through the agent network. We model this process as delayed consensus on a graph with grounded corrector nodes. Spectral decomposition by the grounded Laplacian yields a closed-form stability threshold for the verification dose: correction that is too strong or too delayed can turn consensus into oscillation. The most unstable regime occurs when the communication and verification delays coincide; for delay two, the threshold is the inverse golden ratio. The same framework gives a supermodular placement objective and a greedy (1-1/e)-approximation rule for assigning a limited corrector budget to influential nodes. Experiments across five open models confirm the predicted dose-delay oscillations. By contrast, grounded factual answering makes truth an absorbing boundary and eliminates the effect, suggesting that the instability is specific to signed-belief tasks while grounded verification remains stabilizing

Reinforcement learning in real-world systems often involves delayed feedback, which breaks the Markov assumption and impedes both learning and control. Canonical augmentation-based approaches cause state-space explosion, which imposes a severe sample-complexity burden. Despite recent progress, state-of-the-art augmentation-based baselines either mainly alleviate the burden on the critic or rely on non-unified treatments for the actor and critic. In this study, we propose delayed homomorphic reinforcement learning (DHRL), a framework grounded in MDP homomorphisms that defines a belief-equivalence relation over the augmented state space to collapse control-redundant augmented states. In principle, this yields exact abstraction under deterministic dynamics and approximate abstraction under stochastic dynamics, enabling both the actor and critic to benefit from a structured abstraction mechanism. In finite domains, exact abstraction preserves optimality and recovers the delay-free sample-complexity order, whereas approximate abstraction admits a value-loss bound on the resulting policy. For continuous domains, we introduce deep delayed homomorphic policy gradient (D$^2$HPG), a deep act

This paper introduces a novel class of delayed functional observers specifically designed to reconstruct delayed control laws under severe output measurement lags, directly complementing recent literature \cite{trinhnn26, trinhnam26}. By systematically mitigating simultaneous, unequal delays across both the actuator and sensor channels, the proposed architecture resolves dual-channel latency without requiring full-state estimation or computationally intensive real-time distributed integration. Ultimately, this work provides a powerful, low-order framework that bridges the gap between idealized control theory and the practical constraints of modern networked engineering systems.

Building on the collective advancements in the literature \cite{trinh1, trinh2, trinhnn26, trinhnam26, trinhnam1}, this paper proposes the design of delayed functional observers to asymptotically estimate a generalized delayed control law under significant input and output delays. This framework enables designers to extend the allowable bounds for input delays while ensuring that the observer-based control scheme stabilizes the system despite simultaneous mismatched input and output time-delays.

Regulatory institutions (from content moderation platforms to financial supervisors) observe, deliberate, and intervene only after a characteristic delay. We ask whether this processing lag alone can destabilize a multi-agent system that would otherwise remain stable, without exogenous shocks, coordination among agents, or malicious actors. We study this in two stages. First, we analyze a delayed replicator equation in which autonomous agents benefit from radical behavior but face punishment based on a lagged institutional alarm signal. We derive a closed-form critical delay beyond which the unique interior equilibrium loses stability through a Hopf bifurcation, and prove via center manifold reduction that the bifurcation is supercritical (bounded oscillations, not explosive growth) for the entire sigmoid response family. Second, we embed N=240 agents on a network with reinforcement learning (tabular Q-learning) and cross institutional delay with three decision architectures: fixed-policy, reactive (a memoryless threshold heuristic), and Q-learning. The hierarchy is opposite to the naive expectation that learning amplifies instability. Reactive agents are perfectly stable without d

This paper provides a comprehensive framework for designing functional observers for linear systems subject to delayed output measurements. Moving beyond traditional methodologies, the proposed observer generates an estimate $\hat{z}(t)$ that predicts the current state functional $z(t)=Fx(t)$ using delayed data. By neutralizing sensing latency, the observer serves as a potent time-delay compensator, effectively expanding the practical utility of functional observer theory. The proposed observer architecture offers greater robustness and versatility than traditional Luenberger-type observers by leveraging multiple delayed components to preserve accuracy despite latency. A key contribution of this work is a novel method for extending the maximum allowable measurement delay while maintaining the asymptotic stability of the estimation-error system. Existence conditions are established together with constructive synthesis procedures. Extensive numerical examples are given to illustrate the proposed theory.

When the agent's observations or interactions are delayed, classic reinforcement learning tools usually fail. In this paper, we propose a simple yet new and efficient solution to this problem. We assume that, in the undelayed environment, an efficient policy is known or can be easily learned, but the task may suffer from delays in practice and we thus want to take them into account. We present a novel algorithm, Delayed Imitation with Dataset Aggregation (DIDA), which builds upon imitation learning methods to learn how to act in a delayed environment from undelayed demonstrations. We provide a theoretical analysis of the approach that will guide the practical design of DIDA. These results are also of general interest in the delayed reinforcement learning literature by providing bounds on the performance between delayed and undelayed tasks, under smoothness conditions. We show empirically that DIDA obtains high performances with a remarkable sample efficiency on a variety of tasks, including robotic locomotion, classic control, and trading.

Inverse Reinforcement Learning (IRL) has demonstrated effectiveness in a variety of imitation tasks. In this paper, we introduce an IRL framework designed to extract rewarding features from expert trajectories affected by delayed disturbances. Instead of relying on direct observations, our approach employs an efficient off-policy adversarial training framework to derive expert features and recover optimal policies from augmented delayed observations. Empirical evaluations in the MuJoCo environment under diverse delay settings validate the effectiveness of our method. Furthermore, we provide a theoretical analysis showing that recovering expert policies from augmented delayed observations outperforms using direct delayed observations.

In a conventional contextual multi-armed bandit problem, the feedback (or reward) is immediately observable after an action. Nevertheless, delayed feedback arises in numerous real-life situations and is particularly crucial in time-sensitive applications. The exploration-exploitation dilemma becomes particularly challenging under such conditions, as it couples with the interplay between delays and limited resources. Besides, a limited budget often aggravates the problem by restricting the exploration potential. A motivating example is the distribution of medical supplies at the early stage of COVID-19. The delayed feedback of testing results, thus insufficient information for learning, degraded the efficiency of resource allocation. Motivated by such applications, we study the effect of delayed feedback on constrained contextual bandits. We develop a decision-making policy, delay-oriented resource allocation with learning (DORAL), to optimize the resource expenditure in a contextual multi-armed bandit problem with arm-dependent delayed feedback.

This paper advances the practical utility of functional observer theory by addressing sensing latency in linear time-delay systems. We address the estimation of the functional $z(t)=Fx(t)$ in cases where the measurement delay $h$ is independent of the internal state delay $τ$, with a specific focus on the condition $0 < h < τ$. To compensate for sensing lags, we propose a functional observer structure characterized by multiple internal delays and an augmented architecture. Algebraic existence conditions are established alongside a constructive synthesis procedure. By incorporating an additional delayed measurement vector, we demonstrate that this approach significantly expands the design space and is applicable to a wider class of systems with larger state and output delays.

In environments with delayed observation, state augmentation by including actions within the delay window is adopted to retrieve Markovian property to enable reinforcement learning (RL). However, state-of-the-art (SOTA) RL techniques with Temporal-Difference (TD) learning frameworks often suffer from learning inefficiency, due to the significant expansion of the augmented state space with the delay. To improve learning efficiency without sacrificing performance, this work introduces a novel framework called Variational Delayed Policy Optimization (VDPO), which reformulates delayed RL as a variational inference problem. This problem is further modelled as a two-step iterative optimization problem, where the first step is TD learning in the delay-free environment with a small state space, and the second step is behaviour cloning which can be addressed much more efficiently than TD learning. We not only provide a theoretical analysis of VDPO in terms of sample complexity and performance, but also empirically demonstrate that VDPO can achieve consistent performance with SOTA methods, with a significant enhancement of sample efficiency (approximately 50\% less amount of samples) in the

In the setting of the two-user broadcast channel, where a two-antenna transmitter communicates information to two single-antenna receivers, recent work by Maddah-Ali and Tse has shown that perfect knowledge of delayed channel state information at the transmitter (perfect delayed CSIT) can be useful, even in the absence of any knowledge of current CSIT. Similar benefits of perfect delayed CSIT were revealed in recent work by Kobayashi et al., Yang et al., and Gou and Jafar, which extended the above to the case of perfect delayed CSIT and imperfect current CSIT. The work here considers the general problem of communicating, over the aforementioned broadcast channel, with imperfect delayed and imperfect current CSIT, and reveals that even substantially degraded and imperfect delayed-CSIT is in fact sufficient to achieve the aforementioned gains previously associated to perfect delayed CSIT. The work proposes novel multi-phase broadcasting schemes that properly utilize knowledge of imperfect delayed and imperfect current CSIT, to match in many cases the optimal degrees-of-freedom (DoF) region achieved with perfect delayed CSIT. In addition to the theoretical limits and explicitly constr

The gaming industry is rapidly expanding. With over 2.7 billion players worldwide, game development has become increasingly challenging. To meet the ever-changing demands and expectations of players and due to unforeseen hindrances in the development process, game developers may require to delay the release of their game. We conducted an empirical study of 23,485 games on the Steam platform to analyze how often, and which games delayed their release date. We find that delaying a release is common: 48% of the studied games had a delayed initial release. Games delayed their release by a median of 14 days. Games for which a release date range (e.g., "Q1 2019") was specified, rather than a specific date were more likely to release within that range. Across different game genres, the percentage of games that delay release is similar (ranging from 48% to 52%). Finally, games with a delayed release are rated lower than games that release on time, but the difference is negligible.

Caches are a fundamental component of latency-sensitive computer systems. Recent work of [ASWB20] has initiated the study of delayed hits: a phenomenon in caches that occurs when the latency between the cache and backing store is much larger than the time between new requests. We present two results for the delayed hits caching model. (1) Competitive ratio lower bound. We prove that the competitive ratio of the algorithm in [ASWB20], and more generally of any deterministic online algorithm for delayed hits, is at least Omega(kZ), where k is the cache size and Z is the delay parameter. (2) Antimonotonicity of the delayed hits latency. Antimonotonicity is a naturally desirable property of cache latency: having a cache hit instead of a cache miss should result in lower overall latency. We prove that the latency of the delayed hits model is not antimonotone by exhibiting a scenario where having a cache hit instead of a miss results in an increase in overall latency. We additionally present a modification of the delayed hits model that makes the latency antimonotone.

We investigate a hypothesis that energy accumulation and the subsequent release in NaI(Tl) may lead to pulse-like events in the few-keV energy regime, a phenomenon suggested by the crystal manufacturing company Saint-Gobain, who provided the crystals for DAMA-LIBRA. While we observed delayed long-lasting light emission in a 3" NaI(Tl) crystal after exposing it to UV light, the delayed light consists primarily of single photons that are uncorrelated with each other. We also observe delayed light emission in NaI(Tl) following gamma radiation and large ionization events like cosmic-ray muons. We found that irradiating the crystal with red light after UV exposure significantly suppressed delayed photon emissions.

We compare games under delayed control and delay games, two types of infinite games modelling asynchronicity in reactive synthesis. Our main result, the interreducibility of the existence of sure winning strategies for the protagonist, allows to transfer known complexity results and bounds on the delay from delay games to games under delayed control, for which no such results had been known. We furthermore analyze existence of randomized strategies that win almost surely, where this correspondence between the two types of games breaks down.

We propose a new best-of-both-worlds algorithm for bandits with variably delayed feedback. In contrast to prior work, which required prior knowledge of the maximal delay $d_{\mathrm{max}}$ and had a linear dependence of the regret on it, our algorithm can tolerate arbitrary excessive delays up to order $T$ (where $T$ is the time horizon). The algorithm is based on three technical innovations, which may all be of independent interest: (1) We introduce the first implicit exploration scheme that works in best-of-both-worlds setting. (2) We introduce the first control of distribution drift that does not rely on boundedness of delays. The control is based on the implicit exploration scheme and adaptive skipping of observations with excessive delays. (3) We introduce a procedure relating standard regret with drifted regret that does not rely on boundedness of delays. At the conceptual level, we demonstrate that complexity of best-of-both-worlds bandits with delayed feedback is characterized by the amount of information missing at the time of decision making (measured by the number of outstanding observations) rather than the time that the information is missing (measured by the delays).

The multi-armed bandit (MAB) models have attracted significant research attention due to their applicability and effectiveness in various real-world scenarios such as resource allocation, online advertising, and dynamic pricing. As an important branch, the adversarial MAB problems with delayed feedback have been proposed and studied by many researchers recently where a conceptual adversary strategically selects the reward distributions associated with each arm to challenge the learning algorithm and the agent experiences a delay between taking an action and receiving the corresponding reward feedback. However, the existing models restrict the feedback to be generated from only one user, which makes models inapplicable to the prevailing scenarios of multiple users (e.g. ad recommendation for a group of users). In this paper, we consider that the delayed feedback results are from multiple users and are unrestricted on internal distribution. In contrast, the feedback delay is arbitrary and unknown to the player in advance. Also, for different users in a round, the delays in feedback have no assumption of latent correlation. Thus, we formulate an adversarial MAB problem with multi-user

We propose new ergodic interference alignment techniques for $K$-user interference channels with delayed feedback. Two delayed feedback scenarios are considered -- delayed channel information at transmitter (CIT) and delayed output feedback. It is proved that the proposed techniques achieve total $2K/(K+2)$ DoF which is higher than that by the retrospective interference alignment for the delayed feedback scenarios.