Peer incentivization (PI) is a popular multi-agent reinforcement learning approach where all agents can reward or penalize each other to achieve cooperation in social dilemmas. Despite their potential for scalable cooperation, current PI methods heavily depend on fixed incentive values that need to be appropriately chosen with respect to the environmental rewards and thus are highly sensitive to their changes. Therefore, they fail to maintain cooperation under changing rewards in the environment, e.g., caused by modified specifications, varying supply and demand, or sensory flaws - even when the conditions for mutual cooperation remain the same. In this paper, we propose Dynamic Reward Incentives for Variable Exchange (DRIVE), an adaptive PI approach to cooperation in social dilemmas with changing rewards. DRIVE agents reciprocally exchange reward differences to incentivize mutual cooperation in a completely decentralized way. We show how DRIVE achieves mutual cooperation in the general Prisoner's Dilemma and empirically evaluate DRIVE in more complex sequential social dilemmas with changing rewards, demonstrating its ability to achieve and maintain cooperation, in contrast to curr
We investigate the long-term variability of the known Changing Look Active Galactic Nuclei (CL AGN) Mrk 1018, whose second change we discovered as part of the Close AGN Reference Survey (CARS). Collating over a hundred years worth of photometry from scanned photographic plates and five modern surveys we find a historic outburst between ~1935-1960, with variation in Johnson B magnitude of ~0.8 that is consistent with Mrk 1018's brightness before and after its latest changing look event in the early 2010s. Using the combined modern and historic data, a Generalised Lomb-Scargle suggests broad feature with P = 29-47 years. Its width and stability across tests, as well as the turn-on speed and bright phase duration of the historic event suggests a timescale associated with long-term modulation, such as via rapid flickering in the accretion rate caused by the Chaotic Cold Accretion model rather than a strictly periodic CL mechanism driving changes in Mrk 1018. We also use the modern photometry to constrain Mrk 1018's latest turn-off duration to less than ~1.9 years, providing further support for a CL mechanism with rapid transition timescales, such as a changing mode of accretion.
A Random Walk in Changing Environment (RWCE) is a weighted random walk on a locally finite, connected graph $G$ with random, time-dependent edge-weights. This includes self-interacting random walks, where the edge-weights depend on the history of the process. In general, even the basic question of recurrence or transience for RWCEs is difficult, especially when the underlying graph contains cycles. In this note, we derive a condition for recurrence or transience that is too restrictive for classical RWCEs but instead works for any graph $G.$ Namely, we show that any bounded RWCE on $G$ with "slowly" changing edge-weights inherits the recurrence or transience of the initial weighted graph.
We consider spin chain models with exotic symmetries that change the length of the spin chain. It is known that the XXZ Heisenberg spin chain at the supersymmetric point $Δ=-1/2$ possesses such a symmetry: it is given by the supersymmetry generators, which change the length of the chain by one unit. We show that volume changing symmetries exist also in other spin chain models, and that they can be constructed using a special tensor network, which is a simple generalization of a Matrix Product Operator. As examples we consider the folded XXZ model and its perturbations, and also a new hopping model that is defined on constrained Hilbert spaces. We show that the volume changing symmetries are not related to integrability: the symmetries can survive even non-integrable perturbations. We also show that the known supersymmetry generator of the XXZ chain with $Δ=-1/2$ can also be expressed as a generalized Matrix Product Operator.
The rapidly changing landscapes of modern optimization problems require algorithms that can be adapted in real-time. This paper introduces an Adaptive Metaheuristic Framework (AMF) designed for dynamic environments. It is capable of intelligently adapting to changes in the problem parameters. The AMF combines a dynamic representation of problems, a real-time sensing system, and adaptive techniques to navigate continuously changing optimization environments. Through a simulated dynamic optimization problem, the AMF's capability is demonstrated to detect environmental changes and proactively adjust its search strategy. This framework utilizes a differential evolution algorithm that is improved with an adaptation module that adjusts solutions in response to detected changes. The capability of the AMF to adjust is tested through a series of iterations, demonstrating its resilience and robustness in sustaining solution quality despite the problem's development. The effectiveness of AMF is demonstrated through a series of simulations on a dynamic optimization problem. Robustness and agility characterize the algorithm's performance, as evidenced by the presented fitness evolution and solu
Existing AI alignment approaches assume that preferences are static, which is unrealistic: our preferences change, and may even be influenced by our interactions with AI systems themselves. To clarify the consequences of incorrectly assuming static preferences, we introduce Dynamic Reward Markov Decision Processes (DR-MDPs), which explicitly model preference changes and the AI's influence on them. We show that despite its convenience, the static-preference assumption may undermine the soundness of existing alignment techniques, leading them to implicitly reward AI systems for influencing user preferences in ways users may not truly want. We then explore potential solutions. First, we offer a unifying perspective on how an agent's optimization horizon may partially help reduce undesirable AI influence. Then, we formalize different notions of AI alignment that account for preference change from the outset. Comparing the strengths and limitations of 8 such notions of alignment, we find that they all either err towards causing undesirable AI influence, or are overly risk-averse, suggesting that a straightforward solution to the problems of changing preferences may not exist. As there i
Whether examinees' answer changing behavior while taking multiple-choice exams is beneficial or harmful is a long-standing puzzle in the educational and psychological measurement literature. Formalizing the problem using the potential outcomes framework, this article shows that the traditional method of comparing the proportions of "wrong to right" and "right to wrong" answer changing patterns--a method that has recently been criticized by van der Linden, Jeon, and Ferrara (2011)--indeed correctly identify the sign of the average answer changing effect, but only for those examinees who actually changed their initial responses. This subgroup effect is referred to as the average treatment effect on the treated (ATT) and generally differs from the average treatment effect on the untreated (ATU), that is, those who did not change their initial responses. Analyzing two real data sets, including van der Linden et al.'s (2011) controversial data, this article finds that the ATT of answer changing is positive while the ATU of answer changing is negative, therefore, the debate on answer changing effects can be easily resolved. The article also shows that answer changing and answer reviewing
A decision-maker periodically acquires information about a changing state, controlling both the timing and content of updates. I characterize optimal policies using a decomposition of the dynamic problem into optimal stopping and static information acquisition. Eventually, information acquisition either stops or follows a simple cycle in which updates occur at regular intervals to restore prescribed levels of relative certainty. This enables precise analysis of long run dynamics across environments. As fixed costs of information vanish, belief changes become lumpy: it is optimal to either wait or acquire information so as to exactly confirm the current belief until rare news prompts a sudden change. The long run solution admits a closed-form characterization in terms of the "virtual flow payoff". I highlight an illustrative application to portfolio diversification.
Experiments in engineering are typically conducted in controlled environments where parameters can be set to any desired value. This assumes that the same applies in a real-world setting -- an assumption that is often incorrect as many experiments are influenced by uncontrollable environmental conditions such as temperature, humidity and wind speed. When optimising such experiments, the focus should lie on finding optimal values conditionally on these uncontrollable variables. This article extends Bayesian optimisation to the optimisation of systems in changing environments that include controllable and uncontrollable parameters. The extension fits a global surrogate model over all controllable and environmental variables but optimises only the controllable parameters conditional on measurements of the uncontrollable variables. The method is validated on two synthetic test functions and the effects of the noise level, the number of the environmental parameters, the parameter fluctuation, the variability of the uncontrollable parameters, and the effective domain size are investigated. ENVBO, the proposed algorithm resulting from this investigation, is applied to a wind farm simulato
The real-world deployment of fully autonomous mobile robots depends on a robust SLAM (Simultaneous Localization and Mapping) system, capable of handling dynamic environments, where objects are moving in front of the robot, and changing environments, where objects are moved or replaced after the robot has already mapped the scene. This paper presents Changing-SLAM, a method for robust Visual SLAM in both dynamic and changing environments. This is achieved by using a Bayesian filter combined with a long-term data association algorithm. Also, it employs an efficient algorithm for dynamic keypoints filtering based on object detection that correctly identify features inside the bounding box that are not dynamic, preventing a depletion of features that could cause lost tracks. Furthermore, a new dataset was developed with RGB-D data especially designed for the evaluation of changing environments on an object level, called PUC-USP dataset. Six sequences were created using a mobile robot, an RGB-D camera and a motion capture system. The sequences were designed to capture different scenarios that could lead to a tracking failure or a map corruption. To the best of our knowledge, Changing-SL
With the next generation of big telescopes such as the ELT and SKA it might become possible to measure changes in the expansion rate of the Universe in real time by measuring the change of the redshifts of a large number of galaxies over a period of the order of 10 years. This phenomenon, known as 'redshift drift,' will provide a crucial direct test of cosmological models. The change in redshift is readily explained using the concept of conformal time which is the comoving distance of a galaxy in lightyears. We emphasize that the redshift drift is directly proportional to the average change in the cosmic expansion rate between the time of a galaxy's light emission and its absorption. This phenomenon is illustrated within the framework of the concordance model, the Lambda-CDM model of the universe.
We provide a new method for online learning, specifically prediction with expert advice, in a changing environment. In a non-changing environment the Squint algorithm has been designed to always function at least as well as other known algorithms and in specific cases it functions much better. However, when using a conventional black-box algorithm to make Squint suitable for a changing environment, it loses its beneficial properties. Hence, we provide a new algorithm, Squint-CE, which is suitable for a changing environment and preserves the properties of Squint.
The initial phase of the deployment of Vehicular Ad-Hoc Networks (VANETs) has begun and many research challenges still need to be addressed. Location privacy continues to be in the top of these challenges. Indeed, both of academia and industry agreed to apply the pseudonym changing approach as a solution to protect the location privacy of VANETs'users. However, due to the pseudonyms linking attack, a simple changing of pseudonym shown to be inefficient to provide the required protection. For this reason, many pseudonym changing strategies have been suggested to provide an effective pseudonym changing. Unfortunately, the development of an effective pseudonym changing strategy for VANETs is still an open issue. In this paper, we present a comprehensive survey and classification of pseudonym changing strategies. We then discuss and compare them with respect to some relevant criteria. Finally, we highlight some current researches, and open issues and give some future directions.
Many tasks in computer vision can be cast as a "label changing" problem, where the goal is to make a semantic change to the appearance of an image or some subject in an image in order to alter the class membership. Although successful task-specific methods have been developed for some label changing applications, to date no general purpose method exists. Motivated by this we propose deep manifold traversal, a method that addresses the problem in its most general form: it first approximates the manifold of natural images then morphs a test image along a traversal path away from a source class and towards a target class while staying near the manifold throughout. The resulting algorithm is surprisingly effective and versatile. It is completely data driven, requiring only an example set of images from the desired source and target domains. We demonstrate deep manifold traversal on highly diverse label changing tasks: changing an individual's appearance (age and hair color), changing the season of an outdoor image, and transforming a city skyline towards nighttime.
We propose a modified primal-dual method for general convex optimization problems with changing constraints. We obtain properties of Lagrangian saddle points for these problems which enable us to establish convergence of the proposed method. We describe specializations of the proposed approach to multi-agent optimization problems under changing communication topology and to feasibility problems.
We perform one-zone simulations of the infall epoch of a pre-supernova stellar core in the presence of neutrino flavor changing scattering interactions. Our calculations give a self-consistent assessment of the relationship between flavor changing rates and the reduction in electron fraction and re-distribution of initial electron lepton number among the neutrino flavors. We discuss and include in our calculations sub-nuclear density medium corrections for flavor changing scattering coherence factors. We find that flavor changing couplings epsilon > 3x10^(-4) in either the electron neutrino - muon neutrino or electron neutrino - tau neutrino channels result in a dynamically significant reduction in core electron fraction relatively soon after neutrino trapping and well before the core reaches nuclear matter density.
Adiabatic approximations break down classically when a constant-energy contour splits into separate contours, forcing the system to choose which daughter contour to follow; the choices often represent qualitatively different behavior, so that slowly changing conditions induce a sudden and drastic change in dynamics. The Kruskal-Henrard-Neishtadt theorem relates the probability of each choice to the rates at which the phase space areas enclosed by the different contours are changing. This represents a connection within closed-system mechanics, and without dynamical chaos, between spontaneous change and increase in phase space measure, as required by the Second Law of Thermodynamics. Quantum mechanically, in contrast, dynamical tunneling allows adiabaticity to persist, for very slow parameter change, through a classical splitting of energy contours; the classical and adiabatic limits fail to commute. Here we show that a quantum form of the Kruskal-Neishtadt-Henrard theorem holds nonetheless, due to unitarity.
Over the last few years, the interlibrary loan (ILL) service of the University of Liège Library has evolved considerably, both in terms of habits and workflows. In this article, we will explain the main stages of this evolution: (1) first reduction in the number of ILL units (from eight to five) and involved operators (from 15 to 10) within the homemade ILL solution (2015); (2) use of the resource sharing (RS) functionality in the new Alma library management system (2015); (3) second reduction in the number of ILL units (from five to only one) and in involved operators (from 10 to six) (2018); (4) subscription to an international broker ILL system (RapidILL) for electronic and digital materials and its integration with Alma (2020); (5) project of peer-to-peer resource sharing for print materials between Alma instances of university and research libraries in Belgium (2022), and (temporary) free ILL service to all University users (2020-2022). The aim of these changes is to harmonise the practices of ILL operators, reduce the quantity of manual and administrative operations and tasks devoted to ILL and supply materials that do not belong to the library collections in a fairer, faster
We present a detailed and complete calculation of the gluino and scalar quarks contribution to the flavour-changing top quark decay into a charm quark and a photon, gluon, or a Z boson within the minimal supersymmetric standard model including flavour changing gluino-quarks-scalar quarks couplings in the right-handed sector. We compare the results with the ones presented in an earlier paper where we considered flavour changing couplings only in the left-handed sector. We show that these new couplings have important consequences leading to a large enhancement when the mixing of the scalar partners of the left- and right-handed top quark is included. Furthermore CP violation in the flavour changing top quark decay will occur when a SUSY phase is taken into account.
The broad range of accumulated experimental data on the binding energies for single-particle states in nuclei is examined as a function of the constituent number of neutrons and protons and an unexpectedly simple pattern emerges. The dependence of the energies of neutron states on the number of constituent protons, or of proton states on the number of neutrons, are very similar to each other and the sign reflects the well-known strong attraction. For the same kind of nucleons changing as in the state -- energies for neutron states with neutron number changing or proton states with protons -- the dependence is at least a factor of four weaker in magnitude and slightly repulsive, except when the changing nucleons are only within the same orbit as the state. The systematics of the accumulated data are presented with a minimum of use made of model assumptions.