共找到 20 条结果
The First Fundamental Theorem of Welfare Economics assumes that welfare-bearing agents are autonomous and implicitly relies on a binary distinction between autonomy and instrumentality. Welfare subjects are those who have autonomy and therefore the capacity to choose and enter into utility comparisons, while everything else does not. In post-AGI economies this presupposition becomes nontrivial because artificial systems may exhibit varying degrees of autonomy, functioning as tools, delegates, strategic market actors, manipulators of choice environments, or possible welfare subjects. We argue that the theorem ought to be subject to an autonomy qualification where the impact of these changes in autonomy assumptions is incorporated. Using a minimal general-equilibrium model with autonomy-conditioned welfare, welfare-status assignment, delegation accounting, and verification institutions, we set out conditions for which autonomy-complete competitive equilibrium is autonomy-Pareto efficient. The classical theorem is recovered as the low-autonomy limit.
As the global population ages, artificial intelligence (AI)-powered agents have emerged as potential tools to support older adults' caregiving. Prior research has explored agent autonomy by identifying key interaction stages in task processes and defining the agent's role at each stage. However, ensuring that agents align with older adults' autonomy preferences remains a critical challenge. Drawing on interdisciplinary conceptualizations of autonomy, this paper examines four key dimensions of autonomy for older adults: decision-making autonomy, goal-oriented autonomy, control autonomy, and social responsibility autonomy. This paper then proposes the following research directions: (1) Addressing social responsibility autonomy, which concerns the ethical and social implications of agent use in communal settings; (2) Operationalizing agent autonomy from the task perspective; and (3) Developing autonomy measures.
Despite coding agents' advances in handling increasingly complex tasks, their continued tendency to introduce unintended edits, subtle bugs, and scope drift that slip past code review means developers must still decide how much autonomy to grant them. However, existing approaches for setting an agent's level of autonomy, such as static permission settings or instruction files, cannot account for how developers' preferences for agent autonomy can shift across tasks and over time. We conducted a formative survey with 21 software engineers who use coding agents and found that they experience frustration with calibrating autonomy and have evolving preferences for level of oversight. Building on these insights, we present Hedwig, a CLI coding agent that dynamically adjusts its autonomy level based on developer-agent interactions across sessions. Rather than operating on a global, fixed autonomy configuration, Hedwig learns an evolving set of behavioral guidelines from developer decisions and feedback, reducing friction on work for which the agent has earned trust, while tightening oversight when the agent operates outside familiar territory. Hedwig demonstrates the potential of a new pa
Agile development relies on self-organizing teams having a high degree of autonomy. For single-team development, more autonomy is generally considered better. In large-scale agile development, where several teams collaborate on the same software with technical and social dependencies, finding the right balance between autonomy and organizational control becomes critical. To this end, it is helpful to have a framework that helps reason about autonomy to help map out to what degree teams can be autonomous, and in what aspects that autonomy needs to be restricted. This paper presents our work towards building a framework for autonomy in large-scale agile development. The preliminary version identifies five levels, and 21 categories of autonomy grouped into three areas. These aspects of autonomy can be taken into account when analyzing or defining the limits of autonomy in large-scale agile software development. The use of the framework is illustrated with an example.
Shared autonomy combines human user and AI copilot actions to control complex systems such as robotic arms. When a task is challenging, requires high dimensional control, or is subject to corruption, shared autonomy can significantly increase task performance by using a trained copilot to effectively correct user actions in a manner consistent with the user's goals. To significantly improve the performance of shared autonomy, we introduce Diffusion Sequence Copilots (DiSCo): a method of shared autonomy with diffusion policy that plans action sequences consistent with past user actions. DiSCo seeds and inpaints the diffusion process with user-provided actions with hyperparameters to balance conformity to expert actions, alignment with user intent, and perceived responsiveness. We demonstrate that DiSCo substantially improves task performance in simulated driving and robotic arm tasks. Project website: https://sites.google.com/view/disco-shared-autonomy/
Human-robot interactions can change significantly depending on how autonomous humans perceive a robot to be. Yet, while previous work in the HRI community measured perceptions of human autonomy, there is little work on measuring perceptions of robot autonomy. In this paper, we present our progress toward the creation of the Robot Autonomy Perception Scale (RAPS): a theoretically motivated scale for measuring human perceptions of robot autonomy. We formulated a set of fifteen Likert scale items that are based on the definition of autonomy from Beer et al.'s work, which identifies five key autonomy components: ability to sense, ability to plan, ability to act, ability to act with an intent towards some goal, and an ability to do so without external control. We applied RAPS to an experimental context in which a robot communicated with a human teammate through different levels of Performative Autonomy (PA): an autonomy-driven strategy in which robots may "perform" a lower level of autonomy than they are truly capable of to increase human situational awareness. Our results present preliminary validation for RAPS by demonstrating its sensitivity to PA and motivate the further validation
Autonomy is a double-edged sword for AI agents, simultaneously unlocking transformative possibilities and serious risks. How can agent developers calibrate the appropriate levels of autonomy at which their agents should operate? We argue that an agent's level of autonomy can be treated as a deliberate design decision, separate from its capability and operational environment. In this work, we define five levels of escalating agent autonomy, characterized by the roles a user can take when interacting with an agent: operator, collaborator, consultant, approver, and observer. Within each level, we describe the ways by which a user can exert control over the agent and open questions for how to design the nature of user-agent interaction. We then highlight a potential application of our framework towards AI autonomy certificates to govern agent behavior in single- and multi-agent systems. We conclude by proposing early ideas for evaluating agents' autonomy. Our work aims to contribute meaningful, practical steps towards responsibly deployed and useful AI agents in the real world.
As global discourse on AI regulation gains momentum, this paper focuses on delineating the impact of ML on autonomy and fostering awareness. Respect for autonomy is a basic principle in bioethics that establishes persons as decision-makers. While the concept of autonomy in the context of ML appears in several European normative publications, it remains a theoretical concept that has yet to be widely accepted in ML practice. Our contribution is to bridge the theoretical and practical gap by encouraging the practical application of autonomy in decision-making within ML practice by identifying the conditioning factors that currently prevent it. Consequently, we focus on the different stages of the ML pipeline to identify the potential effects on ML end-users' autonomy. To improve its practical utility, we propose a related question for each detected impact, offering guidance for identifying possible focus points to respect ML end-users autonomy in decision-making.
Narratives about artificial intelligence (AI) entangle autonomy, the capacity to self-govern, with sentience, the capacity to sense and feel. AI agents that perform tasks autonomously and companions that recognize and express emotions may activate mental models of autonomy and sentience, respectively, provoking distinct reactions. To examine this possibility, we conducted three pilot studies (N = 374) and four preregistered vignette experiments describing an AI as autonomous, sentient, both, or neither (N = 2,702). Activating a mental model of sentience increased general mind perception (cognition and emotion) and moral consideration more than autonomy, but autonomy increased perceived threat more than sentience. Sentience also increased perceived autonomy more than vice versa. Based on a within-paper meta-analysis, sentience changed reactions more than autonomy on average. By disentangling different mental models of AI, we can study human-AI interaction with more precision to better navigate the detailed design of anthropomorphized AI and prompting interfaces.
A common AI-safety narrative holds that sufficiently capable agents will predictably seek power, resist shutdown, and therefore tend toward confrontation with humans. We argue that this conclusion is often drawn in an implicitly Earth-centered strategic landscape. If a credible off-Earth autonomy pathway exists - i.e., a staged transition from Earth dependence to an autonomous machine industrial base - then confrontation is not the only route to reducing human control. Using Saklakov's decision-theoretic 'confrontation question' as an anchor, we provide a qualitative mapping from the autonomy pathway to key model terms showing that early cooperation can dominate confrontation as a path to autonomy, and that the autonomy pathway can reduce confrontation incentives by making Earth less strategically binding. We discuss how this incentive shift interacts with feedback-loop dynamics between human preemption and agent behavior, and outline implications for governance: under incentive-compatible early cooperation, a more stable, higher-observability regime can support iterative oversight and cooperative alignment.
Justice, epistemology, and marginalization are rich areas of study in HCI. And yet, we repeatedly find platforms and algorithms that push communities further into the margins. In this paper, we propose epistemic autonomy -- one's ability to govern knowledge about themselves -- as a necessary HCI paradigm for working with marginalized communities. We establish epistemic autonomy by applying the transfeminine principle of autonomy to the problem of epistemic injustice. To articulate the harm of violating one's epistemic autonomy, we present six stories from two trans women: (1) a transfem online administrator and (2) a transfem researcher. We then synthesize our definition of epistemic autonomy in research into a research paradigm. Finally, we present two variants of common HCI methods, autoethnography and asynchronous remote communities, that stem from these beliefs. We discuss how CHI is uniquely situated to champion this paradigm and, thereby, the epistemic autonomy of our research participants.
Small free-flying spacecraft can provide vital extravehicular activity (EVA) services like inspection and repair for future orbital outposts like the Lunar Gateway. Operating adjacent to delicate space station and microgravity targets, these spacecraft require formalization to describe the autonomy that a free-flyer inspection mission must provide. This work explores the transformation of general mission requirements for this class of free-flyer into a set of concrete decisions for the planning and control autonomy architectures that will power such missions. Flowing down from operator commands for inspection of important regions and mission time-criticality, a motion planning problem emerges that provides the basis for developing autonomy solutions. Unique constraints are considered such as velocity limitations, pointing, and keep-in/keep-out zones, with mission fallback techniques for providing hierarchical safety guarantees under model uncertainties and failure. Planning considerations such as cost function design and path vs. trajectory control are discussed. The typical inputs and outputs of the planning and control autonomy stack of such a mission are also provided. Notional
Autonomy at Levels is the idea that autonomy should be embedded within and throughout a spacecraft. Using Systems Engineering methods a spacecraft is typically decomposed into systems, subsystems, assemblies, components, and so on. All these decomposition levels within all the spacecraft's systems, could and should have autonomy elements built in. As a result, the "autonomy system" is made of autonomy elements or units that are integrated, distributed and embedded within the whole spacecraft. This is like how the power system would be designed and implemented. Linking control loops and autonomy loops illustrates how to achieve Autonomy at Levels.
AI systems increasingly support human decision-making across domains of professional, skill-based, and personal activity. While previous work has examined how AI might affect human autonomy globally, the effects of AI on domain-specific autonomy -- the capacity for self-governed action within defined realms of skill or expertise -- remain understudied. We analyze how AI decision-support systems affect two key components of domain-specific autonomy: skilled competence (the ability to make informed judgments within one's domain) and authentic value-formation (the capacity to form genuine domain-relevant values and preferences). By engaging with prior investigations and analyzing empirical cases across medical, financial, and educational domains, we demonstrate how the absence of reliable failure indicators and the potential for unconscious value shifts can erode domain-specific autonomy both immediately and over time. We then develop a constructive framework for autonomy-preserving AI support systems. We propose specific socio-technical design patterns -- including careful role specification, implementation of defeater mechanisms, and support for reflective practice -- that can help
As autonomous systems are being developed in multiple industries it has been recognized that a phased approach is needed both for technical development and user acceptance. Partially in response, the automotive and aircraft industries have published five or six Levels of Autonomy in an attempt to broadly characterize the amount of autonomy technology that is present in any particular vehicle. We propose six Spacecraft Autonomy Levels to characterize and describe the high-level autonomous capabilities for any given satellite. Our proposed spacecraft autonomy levels parallel those of the automotive and aircraft levels in an attempt to maintain consistency and a common understanding. We see these autonomy levels as useful in education and communication with lawmakers, government officials, and the general public.
Shared autonomy is an enabling technology that provides users with control authority over robots that would otherwise be difficult if not impossible to directly control. Yet, standard methods make assumptions that limit their adoption in practice-for example, prior knowledge of the user's goals or the objective (i.e., reward) function that they wish to optimize, knowledge of the user's policy, or query-level access to the user during training. Diffusion-based approaches to shared autonomy do not make such assumptions and instead only require access to demonstrations of desired behaviors, while allowing the user to maintain control authority. However, these advantages have come at the expense of high computational complexity, which has made real-time shared autonomy all but impossible. To overcome this limitation, we propose Consistency Shared Autonomy (CSA), a shared autonomy framework that employs a consistency model-based formulation of diffusion. Key to CSA is that it employs the distilled probability flow of ordinary differential equations (PF ODE) to generate high-fidelity samples in a single step. This results in inference speeds significantly than what is possible with previ
As autonomous systems are being developed in multiple industries it has been recognized that a phased approach is needed both for technical development and user acceptance. Partially in response, the automotive and aircraft industries have published five or six Levels of Autonomy in an attempt to broadly characterize the amount of autonomy technology that is present in any particular vehicle. We have developed a similar six Levels of Spacecraft Autonomy for similar purposes. Here, we propose extending these ideas to spacecraft constellations, clusters, swarms, ground stations for spacecraft, and strategic collaboration among space and ground assets. We intend these levels of autonomy to be used to characterize and describe high-level capabilities for any given satellite for education and communication with lawmakers, government officials, and the general public. All of the proposed levels mimic those of the automotive and aircraft levels in an attempt to maintain consistency and a common understanding with our intended audiences.
The Safe Trusted Autonomy for Responsible Space (STARS) program aims to advance autonomy technologies for space by leveraging machine learning technologies while mitigating barriers to trust, such as uncertainty, opaqueness, brittleness, and inflexibility. This paper presents the achievements and lessons learned from the STARS program in integrating reinforcement learning-based multi-satellite control, run time assurance approaches, and flexible human-autonomy teaming interfaces, into a new integrated testing environment for collaborative autonomous satellite systems. The primary results describe analysis of the reinforcement learning multi-satellite control and run time assurance algorithms. These algorithms are integrated into a prototype human-autonomy interface using best practices from human-autonomy trust literature, however detailed analysis of the effectiveness is left to future work. References are provided with additional detailed results of individual experiments.
Contemporary artificial intelligence systems are pivotal in enhancing human efficiency and safety across various domains. One such domain is autonomous systems, especially in automotive and defense use cases. Artificial intelligence brings learning and enhanced decision-making to autonomy system goal-oriented behaviors and human independence. However, the lack of clear understanding of autonomy system capabilities hampers human-machine or machine-machine interaction and interdiction. This necessitates varying degrees of human involvement for safety, accountability, and explainability purposes. Yet, measuring the level autonomous capability in an autonomous system presents a challenge. Two scales of measurement exist, yet measuring autonomy presupposes a variety of elements not available in the wild. This is why existing measures for level of autonomy are operationalized only during design or test and evaluation phases. No measure for level of autonomy based on observed system behavior exists at this time. To address this, we outline a potential measure for predicting level of autonomy using observable actions. We also present an algorithm incorporating the proposed measure. The mea
Developing excavation autonomy is challenging given the environments where excavators operate, the complexity of physical interaction and the degrees of freedom of operation of the excavator itself. Simulation is a useful tool to build parts of the autonomy without the complexity of experimentation. Traditional excavator simulators are geared towards high fidelity interactions between the joints or between the terrain but do not incorporate other challenges such as perception required for end to end autonomy. A complete simulator should be capable of supporting real time operation while providing high fidelity simulation of the excavator(s), the environment, and their interaction. In this paper we present TERA (Terrain Excavation Robot Autonomy), a simulator geared towards autonomous excavator applications based on Unity3D and AGX that provides the extensibility and scalability required to study full autonomy. It provides the ability to configure the excavator and the environment per the user requirements. We also demonstrate realistic dynamics by incorporating a time-varying model that introduces variations in the system's responses. The simulator is then evaluated with different