共找到 20 条结果
Computing optimal conditional reachability probabilities in Markov decision processes (MDPs) is tractable by a reduction to reachability probabilities. Yet, this reduction yields cyclic, challenging MDPs that are often notoriously hard to solve. We present an alternative, practically efficient method to compute optimal conditional reachabilities. This new method is numerically stable, can decide the threshold problem in linear time on acyclic MDPs, and yields performance comparable to standard reachability queries. We also integrate the method in an abstraction-refinement framework to analyse millions of Markov chains at once. We demonstrate the efficacy of the new methods on benchmarks from Bayesian network analysis, probabilistic programs, and runtime monitoring and show speed-ups up to multiple orders of magnitude.
Shielding is a prominent model-based technique to ensure safety of autonomous agents. Classical shielding aims to ensure that nothing bad ever happens and comes with strong guarantees about safety and maximal permissiveness. However, shielding systems for probabilistic safety, where something bad is allowed to happen with an acceptable probability, has proven to be more intricate. This paper presents a formal framework that conservatively extends classical shields to probabilistic safety. In this framework, we (i) demonstrate the impossibility of preserving the strong guarantees on safety and permissiveness, (ii) provide natural shields with weaker guarantees, and (iii) introduce offline and online shield constructions ensuring strong safety guarantees. The empirical evaluation highlights the practical advantages of the new shields, as well as their computational feasibility.
Solving partially observable Markov decision processes (POMDPs) requires computing policies under imperfect state information. Despite recent advances, the scalability of existing POMDP solvers remains limited. Moreover, many settings require a policy that is robust across multiple POMDPs, further aggravating the scalability issue. We propose the Lexpop framework for POMDP solving. Lexpop (1) employs deep reinforcement learning to train a neural policy, represented by a recurrent neural network, and (2) constructs a finite-state controller mimicking the neural policy through efficient extraction methods. Crucially, unlike neural policies, such controllers can be formally evaluated, providing performance guarantees. We extend Lexpop to compute robust policies for hidden-model POMDPs (HM-POMDPs), which describe finite sets of POMDPs. We associate every extracted controller with its worst-case POMDP. Using a set of such POMDPs, we iteratively train a robust neural policy and consequently extract a robust controller. Our experiments show that on problems with large state spaces, Lexpop outperforms state-of-the-art solvers for POMDPs as well as HM-POMDPs.
Jets are collimated clusters of particles formed by the hadronization of partons following a hard interaction. In proton-proton ($pp$) collisions at the Relativistic Heavy Ion Collider (RHIC), jet production is dominated by $gg$ and $qg$ partonic processes, allowing us to directly probe the gluon parton distribution function (PDF) in the proton in a way complementary to deep inelastic scattering. In this paper, we report the double-differential inclusive-jet cross sections as a function of jet transverse momentum, $p_{\rm T}$, and pseudorapidity, $η$, at center-of-mass energies $\sqrt{s} = 200$ and $510$~GeV, from $pp$ collisions studied with the STAR detector. The jet $p_{\rm T}$ is corrected for underlying event contributions by applying an off-axis cone method. At mid-pseudorapidity, $|η| < 0.9$, the kinematic coverage of our data extends to $0.07 < x_{\rm T} \text{ (}= 2p_{\rm T}{} / \sqrt{s} \text{)} < 0.5$ and $0.03 < x_{\rm T} < 0.31$ at $\sqrt{s} = 200$~and 510 GeV, respectively, where the gluon PDF is poorly constrained by the TeV-scale $pp$~($p\bar{p}$) colliders. The inclusive jet cross sections are compared to the next-to-next-to-leading order perturbativ
A precision measurement of the $K^{*0}$ meson yield is reported in Au+Au collisions at $\sqrt{s_{NN}} = 7.7,\; 11.5,\; 14.6,\; 19.6,$ and $27~\mathrm{GeV}$ using the high-statistics data sample collected by the STAR experiment during the Beam Energy Scan II (BES-II) program at RHIC. The transeverse momentum ($p_{T}$)-integrated yield ratios $(K^{*0} + \overline{K^{*0}})/(K^{+} + K^{-})$ in central collisions show a suppression relative to peripheral collisions at the $(1.7\text{-}3.6)\,σ$ level, while a thermal model without final-stage rescattering overpredicts this ratio with a deviation of $(6.9\text{-}8.2)\,σ$. These results indicate a loss of the measured $K^{*0}$ signal in central collisions due to re-scattering of its hadronic decay products in the hadronic phase. The $p_{T}$-integrated yield of charged kaons exhibits an approximate scaling with charged-particle multiplicity, independent of collision energy and system size. A similar trend is observed for the short-lived $K^{*0}$ resonance, although significant deviations emerge at lower energies. At BES energies, the $K^{*0}/K$ ratio shows stronger suppression than at the highest RHIC and LHC energies within a given multipl
We introduce missingness-MDPs (miss-MDPs), a novel subclass of partially observable Markov decision processes (POMDPs) that incorporates the theory of missing data. A miss-MDP is a POMDP whose observation function is a missingness function, specifying the probability that individual state features are missing (i.e., unobserved) at a time step. The literature distinguishes three canonical missingness types: missing (1) completely at random (MCAR), (2) at random (MAR), and (3) not at random (MNAR). Our planning problem is to compute near-optimal policies for a miss-MDP with an unknown missingness function, given a dataset of action-observation trajectories. Achieving such optimality guarantees for policies requires learning the missingness function from data, which is infeasible for general POMDPs. To overcome this challenge, we exploit the structural properties of different missingness types to derive probably approximately correct (PAC) algorithms for learning the missingness function. These algorithms yield an approximate but fully specified miss-MDP that we solve using off-the-shelf planning methods. We prove that, with high probability, the resulting policies are epsilon-optimal
Markov decision processes (MDPs) describe sequential decision-making processes; MDP policies return for every state in that process an advised action. Classical algorithms can efficiently compute policies that are optimal with respect to, e.g., reachability probabilities. However, these policies are then given in a tabular format. A longstanding challenge is to represent optimal or almost-optimal policies concisely, e.g., as decision trees. This paper makes two contributions towards this challenge: first, an SMT-based approach to encode a given (optimal) policy as a small decision tree, and second, an abstraction-refinement loop that searches for policies that are optimal within the set of policies that can be represented with a small tree. Technically, the latter combines the SMT encoding with verification approaches for families of Markov chains. The empirical evaluation demonstrates the feasibility of these approaches and shows how they can outperform the state-of-the-art on various benchmarks, yielding up to 20 times smaller trees representing (almost) optimal policies for models with up to 10k states and 19 variables.
Partially observable Markov decision processes (POMDPs) model specific environments in sequential decision-making under uncertainty. Critically, optimal policies for POMDPs may not be robust against perturbations in the environment. Hidden-model POMDPs (HM-POMDPs) capture sets of different environment models, that is, POMDPs with a shared action and observation space. The intuition is that the true model is hidden among a set of potential models, and it is unknown which model will be the environment at execution time. A policy is robust for a given HM-POMDP if it achieves sufficient performance for each of its POMDPs. We compute such robust policies by combining two orthogonal techniques: (1) a deductive formal verification technique that supports tractable robust policy evaluation by computing a worst-case POMDP within the HM-POMDP, and (2) subgradient ascent to optimize the candidate policy for a worst-case POMDP. The empirical evaluation shows that, compared to various baselines, our approach (1) produces policies that are more robust and generalize better to unseen POMDPs, and (2) scales to HM-POMDPs that consist of over a hundred thousand environments.
We report measurements of charmonium sequential suppression in Ru+Ru and Zr+Zr collisions at $\sqrt{s_{\mathrm {NN}}}$ = 200 GeV with the STAR experiment at the Relativistic Heavy Ion Collider (RHIC). The inclusive yield ratio of $ψ$(2S) to J/$ψ$ as a function of transverse momentum is reported, along with the centrality dependence of the double ratio, defined as the $ψ$(2S) to J/$ψ$ ratio in heavy-ion collisions relative to that in $p$+$p$ collisions. In the 0-80% centrality class, the double ratio is found to be 0.41 $\pm$ 0.10 (stat) $\pm$ 0.03 (syst) $\pm$ 0.02 (ref), lower than unity with a significance of 5.6 standard deviations. This provides experimental evidence that $ψ$(2S) is significantly more suppressed than J/$ψ$ in heavy-ion collisions at RHIC. This sequential suppression pattern seems to increase from peripheral to central collisions, but with no significant dependence on the transverse momentum.
The STAR experiment reports new, high-precision measurements of the transverse single-spin asymmetries for $π^{\pm}$ within jets, namely the Collins asymmetries, from transversely polarized ${p^{\uparrow}p}$ collisions at $\sqrt{s}$ = 510 GeV. The energy-scaled distribution of jet transverse momentum, $x_{\mathrm{T}} = 2p_{\mathrm{T,jet}}/\sqrt s$, shows a remarkable consistency for Collins asymmetries of $π^{\pm}$ in jets between $\sqrt{s}$ = 200 GeV and 510 GeV. This indicates that the Collins asymmetries are nearly energy independent with, at most, a very weak scale dependence in $p^{\uparrow}p$ collisions. These results extend to high-momentum scales ($Q^2 \leq 3400$ GeV$^2$) and enable unique tests of evolution and universality in the transverse-momentum-dependent formalism, thus providing important constraints for the Collins fragmentation functions.
We report on the first measurement of the Central Exclusive Production process in proton-proton collisions: $pp \ \rightarrow \ p h^+ h^- p$ (where $h = π, K, p$) at the center-of-mass energy $\sqrt{s} = 510$ GeV with the STAR experiment at RHIC. At this energy, the process is dominated by a double Pomeron exchange mechanism. Hence, it provides a clean environment for investigating Pomeron interactions by measuring fully reconstructed final states involving only two hadrons and two forward scattered protons. The oppositely charged hadron pairs are measured within the central detector of STAR. The forward scattered protons are measured in the Roman Pot system allowing the verification of the event's exclusivity. Differential fiducial cross sections within the STAR acceptance are presented as a function of the difference in the azimuthal angle between the outgoing protons. The invariant masses of the charged hadron pairs are measured up to approximately 3 GeV and the square of the four-momentum transfer ($t_1$ and $t_2$) of the two forward-scattered protons in the range $0.3 \text{ GeV}^2 < -t_1 , -t_2 < 1.6 \text{ GeV}^2$. The differential fiducial cross sections of the forwar
Rapidity-odd directed flow $v_1$ measurements are presented for $K^{\pm}$ and $K^0_S$ in Au$+$Au collisions at $\sqrt{s_{\text{NN}}}$ = 3.0, 3.2, 3.5, and 3.9 GeV with the STAR experiment. For comparison, $v_1$ of $π^{\pm}$, protons, and $Λ$ from the same collisions are also discussed. The mid-rapidity $v_1$ slope $\text{d}v_1/\text{d}y|_{y=0}$ for protons and $Λ$ is positive in these collisions. On the other hand, $v_1$ slope of kaons exhibits a strong $p_\text{T}$ dependence: negative at $p_\text{T} <$ 0.6 GeV/$c$ and positive at higher $p_\text{T}$. A similar $p_\text{T}$ dependence is also evident for the $v_1$ slope of charged pions. Compared to the spectator-removed calculations in Au$+$Au collisions at $\sqrt{s_{\text{NN}}} =$ 3.0-3.9 GeV, the JAM model demonstrates a pronounced shift of the $v_1$ slopes of mesons towards the negative direction. It suggests that the shadowing effect of the spectators plays an important role in the observed kaon anti-flow at low $p_\text{T}$ in the high baryon density region of non-central collisions.
Partonic collectivity is one of the necessary signatures for the formation of quark-gluon plasma in high-energy nuclear collisions. Number of constituent quarks (NCQ) scaling has been observed for hadron elliptic flow $v_2$ in top energy nuclear collisions at the Relativistic Heavy Ion Collider and the LHC, and this has been theoretically suggested as strong evidence for partonic collectivity. In this Letter, a systematic analysis of $v_2$ of $π^{\pm}$, $K^{\pm}$, $K^{0}_{S}$, $p$, and $Λ$ in Au+Au collisions at ${\sqrt{s_{_{\rm{NN}}}}}$ = 3.2, 3.5, 3.9, and 4.5 GeV, with the STAR experiment at the Relativistic Heavy Ion Collider, is presented. NCQ scaling is markedly violated at 3.2 GeV, consistent with a hadronic-interaction dominated equation of state. However, as the collision energy increases, a gradual evolution to NCQ scaling is observed. This beam-energy dependence of $v_2$ for all hadrons studied provides evidence for the onset of dominant partonic interactions by ${\sqrt{s_{_{\rm{NN}}}}}$ = 4.5 GeV.
A surprisingly large transverse polarization of $Λ$ hyperons in unpolarized hadron-nucleon/nucleus collisions has been observed for 50 years, and the origin of this polarization remains an important open question. Recently, theoretical frameworks have advanced in describing this puzzle with the polarizing fragmentation function (PFF). We report the first measurement of $Λ$ and $\overlineΛ$ transverse polarization inside jets in unpolarized proton-proton collisions, which is directly attributed to the PFF. The polarization is measured as a function of the jet transverse momentum, the fraction of the jet momentum carried by $Λ$($\overlineΛ$) hyperons, and the transverse momentum of $Λ(\overlineΛ)$ hyperons relative to the jet axis. Covering a wide jet-energy range, these data provide the first constraints on the gluon PFF and allow tests of TMD evolution and its universality.
Multi-agent planning under stochastic dynamics is usually formalised using decentralized (partially observable) Markov decision processes ( MDPs) and reachability or expected reward specifications. In this paper, we propose a different approach: we use an MDP describing how a single agent operates in an environment and probabilistic hyperproperties to capture desired temporal objectives for a set of decentralized agents operating in the environment. We extend existing approaches for model checking probabilistic hyperproperties to handle temporal formulae relating paths of different agents, thus requiring the self-composition between multiple MDPs. Using several case studies, we demonstrate that our approach provides a flexible and expressive framework to broaden the specification capabilities with respect to existing planning techniques. Additionally, we establish a close connection between a subclass of probabilistic hyperproperties and planning for a particular type of Dec-MDPs, for both of which we show undecidability. This lays the ground for the use of existing decentralized planning tools in the field of probabilistic hyperproperty verification.
The ability to compute reward-optimal policies for given and known finite Markov decision processes (MDPs) underpins a variety of applications across planning, controller synthesis, and verification. However, we often want policies (1) to be robust, i.e., they perform well on perturbations of the MDP and (2) to satisfy additional structural constraints regarding, e.g., their representation or implementation cost. Computing such robust and constrained policies is indeed computationally more challenging. This paper contributes the first approach to effectively compute robust policies subject to arbitrary structural constraints using a flexible and efficient framework. We achieve flexibility by allowing to express our constraints in a first-order theory over a set of MDPs, while the root for our efficiency lies in the tight integration of satisfiability solvers to handle the combinatorial nature of the problem and probabilistic model checking algorithms to handle the analysis of MDPs. Experiments on a few hundred benchmarks demonstrate the feasibility for constrained and robust policy synthesis and the competitiveness with state-of-the-art methods for various fragments of the problem.
We present results on the production of $π^{\pm}$, $K^{\pm}$, $p$, and $\bar{p}$ in Au+Au collisions at $\sqrt{s_\mathrm{NN}}$ = 54.4~GeV using the STAR detector at RHIC, at midrapidity ($|y| <$ 0.1). Invariant yields of these particles as a function of transverse momentum are shown. We determine bulk properties such as integrated particle yields ($dN/dy$), mean transverse momentum ($\langle p_{T} \rangle$), particle ratios, which provide insight into the particle production mechanisms. Additionally, the kinetic freezeout parameters ($T_\text{kin}$ and $\langle β_{T} \rangle$), which provide information about the dynamics of the system at the time of freezeout, are obtained. The Bjorken energy density ($ε_{\rm{BJ}}$), which gives an estimate of the energy density in the central rapidity region of the collision zone at the formation time $τ$, is calculated and presented as a function of multiplicity for various energies. The results are compared with those from the models such as A Multi-Phase Transport (AMPT) and Heavy Ion Jet INteraction Generator (HIJING) for further insights.
The analysis of formal models that include quantitative aspects such as timing or probabilistic choices is performed by quantitative verification tools. Broad and mature tool support is available for computing basic properties such as expected rewards on basic models such as Markov chains. Previous editions of QComp, the comparison of tools for the analysis of quantitative formal models, focused on this setting. Many application scenarios, however, require more advanced property types such as LTL and parameter synthesis queries as well as advanced models like stochastic games and partially observable MDPs. For these, tool support is in its infancy today. This paper presents the outcomes of QComp 2023: a survey of the state of the art in quantitative verification tool support for advanced property types and models. With tools ranging from first research prototypes to well-supported integrations into established toolsets, this report highlights today's active areas and tomorrow's challenges in tool-focused research for quantitative verification.
With the STAR experiment at the BNL Relativisic Heavy Ion Collider, we characterize $\sqrt{s_\mathrm{NN}}$ = 200 GeV p+Au collisions by event activity (EA) measured within the pseudorapidity range $eta$ $in$ [-5, -3.4] in the Au-going direction and report correlations between this EA and hard- and soft- scale particle production at midrapidity ($η$ $\in$ [-1, 1]). At the soft scale, charged particle production in low-EA p+Au collisions is comparable to that in p+p collisions and increases monotonically with increasing EA. At the hard scale, we report measurements of high transverse momentum (pT) jets in events of different EAs. In contrast with the soft particle production, high-pT particle production and EA are found to be inversely related. To investigate whether this is a signal of jet quenching in high-EA events, we also report ratios of pT imbalance and azimuthal separation of dijets in high- and low-EA events. Within our measurement precision, no significant differences are observed, disfavoring the presence of jet quenching in the highest 30% EA p+Au collisions at $\sqrt{s_\mathrm{NN}}$ = 200 GeV.
Markov decision processes (MDPs) provide a fundamental model for sequential decision making under process uncertainty. A classical synthesis task is to compute for a given MDP a winning policy that achieves a desired specification. However, at design time, one typically needs to consider a family of MDPs modelling various system variations. For a given family, we study synthesising (1) the subset of MDPs where a winning policy exists and (2) a preferably small number of winning policies that together cover this subset. We introduce policy trees that concisely capture the synthesis result. The key ingredient for synthesising policy trees is a recursive application of a game-based abstraction. We combine this abstraction with an efficient refinement procedure and a post-processing step. An extensive empirical evaluation demonstrates superior scalability of our approach compared to naive baselines. For one of the benchmarks, we find 246 winning policies covering 94 million MDPs. Our algorithm requires less than 30 minutes, whereas the naive baseline only covers 3.7% of MDPs in 24 hours.