To make reinforcement learning more sample efficient, we need better credit assignment methods that measure an action's influence on future rewards. Building upon Hindsight Credit Assignment (HCA), we introduce Counterfactual Contribution Analysis (COCOA), a new family of model-based credit assignment algorithms. Our algorithms achieve precise credit assignment by measuring the contribution of actions upon obtaining subsequent rewards, by quantifying a counterfactual query: 'Would the agent still have reached this reward if it had taken another action?'. We show that measuring contributions w.r.t. rewarding states, as is done in HCA, results in spurious estimates of contributions, causing HCA to degrade towards the high-variance REINFORCE estimator in many relevant environments. Instead, we measure contributions w.r.t. rewards or learned representations of the rewarding objects, resulting in gradient estimates with lower variance. We run experiments on a suite of problems specifically designed to evaluate long-term credit assignment capabilities. By using dynamic programming, we measure ground-truth policy gradients and show that the improved performance of our new model-based cred
Coherence Neuro has started testing a brain-computer interface that could one day use electrical stimulation to prevent tumors from growing
A long-overlooked organ may hold surprising clues to healthy aging and cancer survival。 Researchers at Mass General Brigham used AI to analyze CT scans from tens of thousands of adults and found that people with healthier thymuses—a small immune-system organ once thought to become largely irrelevant after childhood—lived longer and had substantiall
This work has investigated the Magneto-Optical Trap (MOT) system used to produce Bose-Einstein Condensate (BEC). A primary challenge addressed in this study concerns the geometric limitations of traditional single-pair anti-Helmholtz coil configurations, where the magnetic field peaks occur outside the accessible inter-coil region. To overcome this limitation, we have explored the use of double-pair anti-Helmholtz coil configurations that create well-shaped magnetic field potentials centered at the experimentally accessible $z=0$ location. This investigation encompasses the three sequential processes of atom cooling: cooling in a linear external magnetic field through Doppler cooling, cooling in a well-shaped magnetic field through trapping, and evaporative cooling of atoms to achieve sub-microkelvin temperatures. Through theoretical analysis and numerical simulation, we have determined optimal geometric parameters for the coil configuration and operational parameters including laser detuning, saturation intensity, and initial atom populations for ${}^{87}\text{Rb}$ BEC production. The results indicate that with the optimized configuration, the system can achieve final temperatures
In this paper, we present a logic for conditional strong historical necessity in branching time and apply it to analyze a nontheological version of Lavenham's argument for future determinism. Strong historical necessity is motivated from a linguistical perspective, and an example of it is ``If I had not gotten away, I must have been dead''. The approach of the logic is as follows. The agent accepts ontic rules concerning how the world evolves over time. She takes some rules as indefeasible, which determine acceptable timelines. When evaluating a sentence with conditional strong historical necessity, we introduce its antecedent as an indefeasible ontic rule and then check whether its consequent holds for all acceptable timelines. The argument is not sound by the logic.
Positions of a charged particle's equilibrium orbits and spatial regions where the chaos bound is violated are found through circular motions of the particle around charged Taub-NUT black holes. Lyapunov exponent is gotten by calculating eigenvalues of a Jacobian matrix in a phase space $(r,π_r)$. When the particle's charge is fixed, the positions of the equilibrium orbits gradually move away from the event horizons with the increase of the angular momentum.The result shows that the bound is violated in the near-horizon regions and at a certain distance from the horizons when the charge and NUT parameter are fixed. The spatial regions increase with the increase of the NUT parameter's value.
Assessing the maturity of security practices during the development of Machine Learning (ML) based software components has not gotten as much attention as traditional software development. In this Blue Sky idea paper, we propose an initial Machine Learning Security Maturity Model (MLSMM) which organizes security practices along the ML-development lifecycle and, for each, establishes three levels of maturity. We envision MLSMM as a step towards closer collaboration between industry and academia.
In this paper, we investigate the photon sphere, shadow radius and quasinormal modes of a $4$-dimensional black hole with a deficit solid angle and quintessence-like matter. We find that the radii of the photon sphere and shadow decrease with the decreases of the deficit solid angle and density of quintessence-like matter. The quasinormal modes are gotten by the sixth order WKB approximation method and shadow radius, respectively. The values of the real part and imaginary parts of the quasinormal modes increase with the decrease of the values of the deficit solid angle and density of quintessence-like matter when the multipole number is fixed. The quasinormal modes gotten by these two methods are in good agreement, especially when the multipole number is large. It shows the correspondence between the quasinormal modes in the eikonal limit and shadow.
Lattice simulations of non-zero density QCD introduce the so-called sign problem (complex or negative probabilities), which invalidates importance sampling methods. To circumvent this, we use the Complex Langevin Equation (CLE), to measure the boundary terms and then compare these results with the ones gotten from reweighting, confirming the expectations from previous studies. We also investigate boundary terms in simulations using CLE with dynamic stabilization and compare this, to results calculated with reweighting.
In the article we obtain almost global existence for Dirac Equations with high regularity and small initial datum on Tori. Besides, the global existence with low regularity and small initial datum is gotten. The approaches are mainly Gagliardo-Nirenberg-Moser estimates and Bernstein-Type Lemma.
The perturbation method in supersymmetric quantum mechanics (SUSYQM) is used to study the spheroidal wave functions' recurrence relations, which are revealed by the shape-invariance property of the super-potential. The super-potential is expanded by the parameter alpha and could be gotten by approximation method. Up to the first order, it has the shape-invariance property and the excited spheroidal wave functions are gotten. Also, all the first term eigenfunctions obtained are in closed form. They are advantageous to investigating for involved physical problems of spheroidal wave function.
A universal relation between the leading correction to the entropy and extremality was gotten in the work of Goon and Penco. In this paper, we extend this work to the massive gravity and investigate thermodynamic extremality relations in a topologically higher-dimensional black hole. A rescaled cosmological constant is added to the action of the massive gravity as a perturbative correction. This correction modifies the extremality bound of the black hole and leads to the shifts of the mass, entropy, etc. The Goon-Penco relation is gotten. Regarding the cosmological constant as a variable related to pressure, we get the thermodynamic extremality relations between the mass and pressure, charge, parameters $c_i$ by accurate calculations, respectively. Finally, these relations are verified by a triple product identity, which shows that the universal relation exists in black holes.
Principal Component Analysis (PCA) is known to be the most widely applied dimensionality reduction approach. A lot of improvements have been done on the traditional PCA, in order to obtain optimal results in the dimensionality reduction of various datasets. In this paper, we present an improvement to the traditional PCA approach called Multiplicative factoring Principal Component Analysis (MPCA). The advantage of MPCA over the traditional PCA is that a penalty is imposed on the occurrence space through a multiplier to make negligible the effect of outliers in seeking out projections. Here we apply two multiplier approaches, total distance and cosine similarity metrics. These two approaches can learn the relationship that exists between each of the data points and the principal projections in the feature space. As a result of this, improved low-rank projections are gotten through multiplying the data iteratively to make negligible the effect of corrupt data in the training set. Experiments were carried out on YaleB, MNIST, AR, and Isolet datasets and the results were compared to results gotten from some popular dimensionality reduction methods such as traditional PCA, RPCA-OM, and a
The strongest type of coloring of pairs of countable ordinals, gotten by Todorcevic from a strongly Luzin set, is shown to be equivalent to the existence of a nonmeager set of reals of size $\aleph_1$. In the other direction, it is shown that the existence of both a strongly Luzin set and a coherent Souslin tree is compatible with the existence of a countable partition of pairs of countable ordinals such that no coloring is strong over it. This clarifies the interaction between a gallery of coloring assertions going back to Luzin and Sierpinski a hundred years ago.
The rising algebra is a subalgebra of the group algebra of the symmetric group S_n, gotten by lumping together permutations having the same number of rising sequences. This well-known algebra arises naturally when studying riffle shuffles. Here we introduce a number of other subalgebras that arise naturally when studying `ruffles', which are like riffles except that after cutting the deck you turn over the bunch of cards that were on the bottom. This orphaned draft offers no context or motivation, and uses idiosyncratic notation and terminology that `seemed like a good idea at the time'. We're making it available because it has been cited in this form.
We discuss the vacuum energy density and the cosmological constant of dS$_5$ brane world with a dilaton field. It is shown that a stable AdS$_4$ brane can be constructed and gravity localization can be realized. An explicit relation between the dS bulk cosmological constant and the brane cosmological constant is obtained. The discrete mass spectrum of the massive scalar field in the AdS$_4$ brane is used to acquire the relationship between the brane cosmological constant and the vacuum energy density. The vacuum energy density in the brane gotten by this method is in agreement with astronomical observations.
Parallel lines are very important objects in Euclid plane geometry and its behaviors can be gotten by one's intuition. But in a planar map geometry, a kind of the Smarandache geometries, the sutation is complex since it may contains elliptic or hyperbolic points. This paper concentrates on the behavior of parallel bundles, a generazation of parallel lines in plane geometry and obtains characteristics for for parallel bundles.
A necessary condition for the validity of the holographic principle is the holographic bound: the entropy of a system is bounded from above by a quarter of the area of a circumscribing surface measured in Planck areas. This bound cannot be derived at present from consensus fundamental theory. We show with suitable {\it gedanken} experiments that the holographic bound follows from the generalized second law of thermodynamics for both generic weakly gravitating isolated systems and for isolated, quiescent and nonrotating strongly gravitating configurations well above Planck mass. These results justify Susskind's early claim that the holographic bound can be gotten from the second law.
In order to investigate the longitudinal beam dynamics during the adiabatic capture and acceleration processes of the CRing in HIAF project, a simulation of both processes above is carried out with $U^{34+}$ ions. The ions will be captured into a bucket adiabatically and accelerated from 800 MeV/u to 1130 MeV/u . Simulation of these processes by tracking appropriate distributions with the longitudinal beam dynamics code ESME has been used to find optimum parameters such as RF phase, RF voltage and RF frequency etc. An enhanced capture and acceleration efficiency can be gotten from the simulation results, with the optimized RF voltage and RF phase program.
The perturbation method in supersymmetric quantum mechanics (SUSYQM) is used to study the spheroidal wave functions' eigenvalue problem. Expanding the super-potential in series of the parameter alpha, the first order term of ground eigen-value and the eigen-function are gotten. In the paper, the very excellent results are that all the first two terms approximation on eigenfunctions obtained are in closed form. They give useful information for the involved physical problems in application of spheroidal wave functions.