搜索结果：perils

In 2025, coding agents have seen a very rapid adoption. Coding agents leverage Large Language Models (LLMs) in ways that are markedly different from LLM-based code completion, making their study critical. Moreover, unlike LLM-based completion, coding agents leave visible traces in software repositories, enabling the use of MSR techniques to study their impact on SE practices. This paper documents the promises, perils, and heuristics that we have gathered from studying coding agent activity on GitHub.

Governments are the primary providers of essential public services and are responsible for delivering them effectively. In high-stakes decision-making domains such as child welfare (CW), agencies must protect children without unnecessarily prolonging a family's engagement with the system. With growing optimism around AI, governments are pushing for its integration but concerns regarding feasibility and harms remain. Through collaborations with a large Canadian CW agency, we examined how LocalLLM and BERTopic models can track CW case progress. We demonstrate how the tools can potentially assist workers in opportunistically addressing gaps in their work by signaling case progress/deviations. And yet, we also show how they fail to detect case trajectories that require discretionary judgments grounded in social work training, areas where practitioners would actually want support to pre-emptively address substantive case concerns. We also provide a roadmap of future participatory directions to co-design language tools for/with the public sector.

In 1964, Aharonov, Bergmann, and Lebowitz introduced their well-known ABL rule with the intention of providing a time-symmetric formalism for computing novel kinds of conditional probabilities in quantum theory. Later papers attached additional significance to the ABL rule, including assertions that it supported violations of the uncertainty principle. The present work challenges these claims, as well as subsequent attempts to salvage the original interpretation of the ABL rule. Taking a broader view, this paper identifies a subtle category error at the heart of the ABL rule that consists of confusing observables that belong to a single system with emergent observables that arise only for physical ensembles. Along the way, this paper points out other problems and fallacious reasoning in the research literature surrounding the ABL rule, including the misuse of post-selection, a reliance on pattern matching to classical formulas, and a posture of measurementism that takes experimental data as providing answers to interpretational questions.

What does Artificial Intelligence (AI) have to contribute to health care? And what should we be looking out for if we are worried about its risks? In this paper we offer a survey, and initial evaluation, of hopes and fears about the applications of artificial intelligence in medicine. AI clearly has enormous potential as a research tool, in genomics and public health especially, as well as a diagnostic aid. It's also highly likely to impact on the organisational and business practices of healthcare systems in ways that are perhaps under-appreciated. Enthusiasts for AI have held out the prospect that it will free physicians up to spend more time attending to what really matters to them and their patients. We will argue that this claim depends upon implausible assumptions about the institutional and economic imperatives operating in contemporary healthcare settings. We will also highlight important concerns about privacy, surveillance, and bias in big data, as well as the risks of over trust in machines, the challenges of transparency, the deskilling of healthcare practitioners, the way AI reframes healthcare, and the implications of AI for the distribution of power in healthcare ins

Social media platforms are online fora where users engage in discussions, share content, and build connections. This review explores the dynamics of social interactions, user-generated contents, and biases within the context of social media analysis (analyzing works that use the tools offered by complex network analysis and natural language processing) through the lens of three key points of view: online debates, online support, and human-AI interactions. On the one hand, we delineate the phenomenon of online debates, where polarization, misinformation, and echo chamber formation often proliferate, driven by algorithmic biases and extreme mechanisms of homophily. On the other hand, we explore the emergence of online support groups through users' self-disclosure and social support mechanisms. Online debates and support mechanisms present a duality of both perils and possibilities within social media; perils of segregated communities and polarized debates, and possibilities of empathy narratives and self-help groups. This dichotomy also extends to a third perspective: users' reliance on AI-generated content, such as the ones produced by Large Language Models, which can manifest both

As AI systems appear to exhibit ever-increasing capability and generality, assessing their true potential and safety becomes paramount. This paper contends that the prevalent evaluation methods for these systems are fundamentally inadequate, heightening the risks and potential hazards associated with AI. I argue that a reformation is required in the way we evaluate AI systems and that we should look towards cognitive sciences for inspiration in our approaches, which have a longstanding tradition of assessing general intelligence across diverse species. We will identify some of the difficulties that need to be overcome when applying cognitively-inspired approaches to general-purpose AI systems and also analyse the emerging area of "Evals". The paper concludes by identifying promising research pathways that could refine AI evaluation, advancing it towards a rigorous scientific domain that contributes to the development of safe AI systems.

DAO Governance is currently broken. We survey the state of the art and find worrying conclusions. Vote buying, vote selling and coercion are easy. The wealthy rule, decentralisation is a myth. Hostile take-overs are incentivised. Ballot secrecy is non-existent or short lived, despite being a human right. Verifiablity is achieved at the expense of privacy. These privacy concerns are highlighted with case study analyses of Vocdoni's governance protocol. This work presents two contributions: firstly a review of current DAO governance protocols, and secondly, an illustration of their vulnerabilities, showcasing the privacy and security threats these entail.

Schaefer (2024) has recently published observations of binary period derivatives $\dot P$ for 52 cataclysmic variables, and concluded that these strongly conflict with all proposed evolutionary pictures for these systems. We point out once again that using measurements of $\dot P$ is likely in practice to produce misleading evolutionary constraints in almost every case. The one identified exception to this is probably the recently-born X-ray binary SN 2022jli, because of its extremely high mass transfer rate.

Overparameterized deep neural networks (DNNs), if not sufficiently regularized, are susceptible to overfitting their training examples and not generalizing well to test data. To discourage overfitting, researchers have developed multicomponent loss functions that reduce intra-class feature correlation and maximize inter-class feature distance in one or more layers of the network. By analyzing the penultimate feature layer activations output by a DNN's feature extraction section prior to the linear classifier, we find that modified forms of the intra-class feature covariance and inter-class prototype separation are key components of a fundamental Chebyshev upper bound on the probability of misclassification, which we designate the Chebyshev Prototype Risk (CPR). While previous approaches' covariance loss terms scale quadratically with the number of network features, our CPR bound indicates that an approximate covariance loss in log-linear time is sufficient to reduce the bound and is scalable to large architectures. We implement the terms of the CPR bound into our Explicit CPR (exCPR) loss function and observe from empirical results on multiple datasets and network architectures tha

We argue that the trend toward providing users with feasible and actionable explanations of AI decisions, known as recourse explanations, comes with ethical downsides. Specifically, we argue that recourse explanations face several conceptual pitfalls and can lead to problematic explanation hacking, which undermines their ethical status. As an alternative, we advocate that explanations of AI decisions should aim at understanding.

Recommendation systems are widespread, and through customized recommendations, promise to match users with options they will like. To that end, data on engagement is collected and used. Most recommendation systems are ranking-based, where they rank and recommend items based on their predicted engagement. However, the engagement signals are often only a crude proxy for utility, as data on the latter is rarely collected or available. This paper explores the following question: By optimizing for measurable proxies, are recommendation systems at risk of significantly under-delivering on utility? If so, how can one improve utility which is seldom measured? To study these questions, we introduce a model of repeated user consumption in which, at each interaction, users select between an outside option and the best option from a recommendation set. Our model accounts for user heterogeneity, with the majority preferring ``popular'' content, and a minority favoring ``niche'' content. The system initially lacks knowledge of individual user preferences but can learn them through observations of users' choices over time. Our theoretical and numerical analysis demonstrate that optimizing for eng

In order to study updating rules, we consider the problem of a malevolent principal screening an imperfectly Bayesian agent. We uncover a fundamental dichotomy between underreaction and overreaction to information. If an agent's posterior is farther away from the prior than it should be under Bayes' law, she can always be exploited by the principal to an unfettered degree: the agent's ex ante expected loss can be made arbitrarily large. In stark contrast, an agent who underreacts (whose posterior is closer to the prior than the Bayesian posterior) cannot be exploited at all.

The use of third-party packages is becoming increasingly popular and has led to the emergence of large software package ecosystems with a maze of inter-dependencies. Since the reliance on these ecosystems enables developers to reduce development effort and increase productivity, it has attracted the interest of researchers: understanding the infrastructure and dynamics of package ecosystems has given rise to approaches for better code reuse, automated updates, and the avoidance of vulnerabilities, to name a few examples. But the reality of these ecosystems also poses challenges to software engineering researchers, such as: How do we obtain the complete network of dependencies along with the corresponding versioning information? What are the boundaries of these package ecosystems? How do we consistently detect dependencies that are declared but not used? How do we consistently identify developers within a package ecosystem? How much of the ecosystem do we need to understand to analyse a single component? How well do our approaches generalise across different programming languages and package ecosystems? In this chapter, we review promises and perils of mining the rich data related t

When numerically simulating the unitary time evolution of an infinite-dimensional quantum system, one is usually led to treat the Hamiltonian $H$ as an "infinite-dimensional matrix" by expressing it in some orthonormal basis of the Hilbert space, and then truncate it to some finite dimensions. However, the solutions of the Schrödinger equations generated by the truncated Hamiltonians need not converge, in general, to the solution of the Schrödinger equation corresponding to the actual Hamiltonian. In this paper we demonstrate that, under mild assumptions, they converge to the solution of the Schrödinger equation generated by a specific Hamiltonian which crucially depends on the particular choice of basis: the Friedrichs extension of the restriction of $H$ to the space of finite linear combinations of elements of the basis. Importantly, this is generally different from $H$ itself; in all such cases, numerical simulations will unavoidably reproduce the wrong dynamics in the limit, and yet there is no numerical test that can reveal this failure, unless one has the analytical solution to compare with. As a practical demonstration of such results, we consider the quantum particle in the

Generative artificial intelligence (GenAI) has the potential to improve healthcare through automation that enhances the quality and safety of patient care. Powered by foundation models that have been pretrained and can generate complex content, GenAI represents a paradigm shift away from the more traditional focus on task-specific classifiers that have dominated the AI landscape thus far. We posit that the imminent application of GenAI in healthcare will be through well-defined, low risk, high value, and narrow applications that automate healthcare workflows at the point of care using smaller foundation models. These models will be finetuned for different capabilities and application specific scenarios and will have the ability to provide medical explanations, reference evidence within a retrieval augmented framework and utilizing external tools. We contrast this with a general, all-purpose AI model for end-to-end clinical decision making that improves clinician performance, including safety-critical diagnostic tasks, which will require greater research prior to implementation. We consider areas where 'human in the loop' Generative AI can improve healthcare quality and safety by au

The log transformation of the dependent variable is not innocuous when using a difference-in-differences (DD) model. With a dependent variable in logs, the DD term captures an approximation of the proportional difference in growth rates across groups. As I show with both simulations and two empirical examples, if the baseline outcome distributions are sufficiently different across groups, the DD parameter for a log-specification can be different in sign to that of a levels-specification. I provide a condition, based on (i) the aggregate time effect, and (ii) the difference in relative baseline outcome means, for when the sign-switch will occur.

Over the past decade, predictive language modeling for code has proven to be a valuable tool for enabling new forms of automation for developers. More recently, we have seen the advent of general purpose "large language models", based on neural transformer architectures, that have been trained on massive datasets of human written text spanning code and natural language. However, despite the demonstrated representational power of such models, interacting with them has historically been constrained to specific task settings, limiting their general applicability. Many of these limitations were recently overcome with the introduction of ChatGPT, a language model created by OpenAI and trained to operate as a conversational agent, enabling it to answer questions and respond to a wide variety of commands from end users. The introduction of models, such as ChatGPT, has already spurred fervent discussion from educators, ranging from fear that students could use these AI tools to circumvent learning, to excitement about the new types of learning opportunities that they might unlock. However, given the nascent nature of these tools, we currently lack fundamental knowledge related to how well

Statisticians and data scientists find insights that help lead to better understanding and better outcomes. When clients and managers come to us for help (and even when they don't), we want to share our advice. While we should be free to share our recommendations, we need to be clear about what the data is telling us and what is based "only on our judgment". Gelman, et. al. wrote "As we have learned from the replication crisis sweeping the biomedical and social sciences, it is frighteningly easy for motivated researchers working in isolation to arrive at favored conclusions-whether inadvertently or intentionally." One senior business leader I know said, "if you have data, great; if we're just going on intuition we can use mine". However, having data isn't enough. We need to be rigorous in our analysis to avoid finding insights that aren't supported. This paper will go through a number of examples to illustrate common mistakes.

With the advent of new and advanced programming languages, it becomes imperative to migrate legacy software to new programming languages. Unsupervised Machine Learning-based Program Translation could play an essential role in such migration, even without a sufficiently sizeable reliable corpus of parallel source code. However, these translators are far from perfect due to their statistical nature. This work investigates unsupervised program translators and where and why they fail. With in-depth error analysis of such failures, we have identified that the cases where such translators fail follow a few particular patterns. With this insight, we develop a rule-based program mutation engine, which pre-processes the input code if the input follows specific patterns and post-process the output if the output follows certain patterns. We show that our code processing tool, in conjunction with the program translator, can form a hybrid program translator and significantly improve the state-of-the-art. In the future, we envision an end-to-end program translation tool where programming domain knowledge can be embedded into an ML-based translation pipeline using pre- and post-processing steps.