In purely non-dissipative systems, Lagrangian and Hamiltonian reduction have proven to be powerful tools for deriving physical models with exact conservation laws. We have discovered a hint that an analogous reduction method exists also for dissipative systems that respect the First and Second Laws of Thermodynamics. In this paper, we show that modern electrostatic gyrokinetics, a reduced plasma turbulence model, exhibits a serendipitous metriplectic structure. Metriplectic dynamics in general is a well developed formalism for extending the concept of Poisson brackets to dissipative systems. Better yet, our discovery enables an intuitive particle-in-cell discretization of the collision operator that also satisfies the First and Second Laws of thermodynamics. These results suggest that collisional gyrokinetics, and other dissipative physical models that obey the Laws of Thermodynamics, could be obtained using an as-yet undiscovered metriplectic reduction theory and that numerical methods could benefit from such theory significantly. Once uncovered, the theory would generalize Lagrangian and Hamiltonian reduction in a substantial manner.
Artificial neural networks can acquire many aspects of human knowledge from data, making them promising as models of human learning. But what those networks can learn depends upon their inductive biases -- the factors other than the data that influence the solutions they discover -- and the inductive biases of neural networks remain poorly understood, limiting our ability to draw conclusions about human learning from the performance of these systems. Cognitive scientists and machine learning researchers often focus on the architecture of a neural network as a source of inductive bias. In this paper we explore the impact of another source of inductive bias -- the initial weights of the network -- using meta-learning as a tool for finding initial weights that are adapted for specific problems. We evaluate four widely-used architectures -- MLPs, CNNs, LSTMs, and Transformers -- by meta-training 430 different models across three tasks requiring different biases and forms of generalization. We find that meta-learning can substantially reduce or entirely eliminate performance differences across architectures and data representations, suggesting that these factors may be less important as
Most expressivity results for transformers treat them as language recognizers -- devices that accept or reject strings -- rather than as they are used in practice: as language models that generate strings autoregressively and probabilistically. We characterize the probability distributions that transformer language models can express. We show that making transformer language recognizers autoregressive can sometimes increase their expressivity, and that making them probabilistic can break equivalences that hold in the non-probabilistic case. Our overall contribution is to tease apart what functions transformers are capable of expressing in their most common use case as language models.
In a recent paper, Enayat and Le lyk [2024] show that second order arithmetic and countable set theory are not definitionally equivalent. It is well known that these theories are biinterpretable. Thus, we have a pair of natural theories that llustrate a meaningful difference between definitional equivalence and bi-interpretability. This is particularly interesting given that Visser and Friedman [2014] have shown that a wide class of natural foundational theories in mathematics are such that if they are bi-interpretable, then they are also definitionally equivalent. The proof offered by Enayat and Le lyk makes use of an inaccessible cardinal. In this short note, we show that the failure of bi-interpretability can be established in Peano Arithmetic merely supposing that one of our target theories are consistent.
Can language models reliably predict that possible events are more likely than merely improbable ones? By teasing apart possibility, typicality, and contextual relatedness, we show that despite the results of previous work, language models' ability to do this is far from robust. In fact, under certain conditions, all models tested - including Llama 3, Gemma 2, and Mistral NeMo - perform at worse-than-chance level, assigning higher probabilities to impossible sentences such as 'the car was given a parking ticket by the brake' than to merely unlikely sentences such as 'the car was given a parking ticket by the explorer'.
Beyond diagonal reconfigurable intelligent surface (BD-RIS) is a new architecture for RIS where elements are interconnected to provide more wave manipulation flexibility than traditional single connected RIS, enhancing data rate and coverage. However, channel estimation for BD-RIS is challenging due to the more complex multiple-connection structure involving their scattering elements. To address this issue, this paper proposes a decoupled channel estimation method for BD-RIS that yields separate estimates of the involved channels to enhance the accuracy of the overall combined channel by capitalizing on its Kronecker structure. Starting from a least squares estimate of the combined channel and by properly reshaping the resulting filtered signal, the proposed algorithm resorts to a Khatri-Rao Factorization (KRF) method that teases out the individual channels based on simple rank-one matrix approximation steps. Numerical results show that the proposed decoupled channel estimation yields more accurate channel estimates than the classical least squares scheme.
Socio-demographic prompting is a commonly employed approach to study cultural biases in LLMs as well as for aligning models to certain cultures. In this paper, we systematically probe four LLMs (Llama 3, Mistral v0.2, GPT-3.5 Turbo and GPT-4) with prompts that are conditioned on culturally sensitive and non-sensitive cues, on datasets that are supposed to be culturally sensitive (EtiCor and CALI) or neutral (MMLU and ETHICS). We observe that all models except GPT-4 show significant variations in their responses on both kinds of datasets for both kinds of prompts, casting doubt on the robustness of the culturally-conditioned prompting as a method for eliciting cultural bias in models or as an alignment strategy. The work also calls rethinking the control experiment design to tease apart the cultural conditioning of responses from "placebo effect", i.e., random perturbations of model responses due to arbitrary tokens in the prompt.
Given the growing importance of AI literacy, we decided to write this tutorial to help narrow the gap between the discourse among those who study language models -- the core technology underlying ChatGPT and similar products -- and those who are intrigued and want to learn more about them. In short, we believe the perspective of researchers and educators can add some clarity to the public's understanding of the technologies beyond what's currently available, which tends to be either extremely technical or promotional material generated about products by their purveyors. Our approach teases apart the concept of a language model from products built on them, from the behaviors attributed to or desired from those products, and from claims about similarity to human cognition. As a starting point, we (1) offer a scientific viewpoint that focuses on questions amenable to study through experimentation; (2) situate language models as they are today in the context of the research that led to their development; and (3) describe the boundaries of what is known about the models at this writing.
The impressive performance of recent language models across a wide range of tasks suggests that they possess a degree of abstract reasoning skills. Are these skills general and transferable, or specialized to specific tasks seen during pretraining? To disentangle these effects, we propose an evaluation framework based on "counterfactual" task variants that deviate from the default assumptions underlying standard tasks. Across a suite of 11 tasks, we observe nontrivial performance on the counterfactual variants, but nevertheless find that performance substantially and consistently degrades compared to the default conditions. This suggests that while current LMs may possess abstract task-solving skills to an extent, they often also rely on narrow, non-transferable procedures for task-solving. These results motivate a more careful interpretation of language model performance that teases apart these aspects of behavior.
The multiple access channel (MAC) capacity with feedback is considered under feedback models designed to tease out which factors contribute to the MAC feedback capacity benefit. Comparing the capacity of a MAC with ``perfect'' feedback, which causally delivers to the transmitters the true channel output, to that of a MAC with ``independent'' feedback, which causally delivers to the transmitters an independent instance of that same channel output, allows separation of effects like cooperation from alternative feedback benefits such as knowledge of the channel instance. Proving that the Cover-Leung (CL) achievability bound, which is known to be loose for some channels, is achievable also under (shared or distinct) independent feedback at the transmitters shows that the CL bound does not require transmitter knowledge of the channel instance. Proving that each transmitter's maximal rate under independent feedback exceeds that under perfect feedback highlights the potential power of an independent look at the channel output.
The response of metals and their microstructures under extreme dynamic conditions can be markedly different from that under quasistatic conditions. Traditionally, high strain rates and shock stresses are measured using cumbersome and expensive methods such as the Kolsky bar or large spall experiments. These methods are low throughput and do not facilitate high-fidelity microstructure-property linkages. In this work, we combine two powerful small-scale testing methods, custom nanoindentation, and laser-driven micro-flyer shock, to measure the dynamic and spall strength of metals. The nanoindentation system is configured to test samples from quasistatic to dynamic strain rate regimes (10$^{-3}$ s$^{-1}$ to 10$^{+4}$ s$^{-1}$). The laser-driven micro-flyer shock system can test samples through impact loading between 10$^{+5}$ s$^{-1}$ to 10$^{+7}$ s$^{-1}$ strain rates, triggering spall failure. The model material used for testing is Magnesium alloys, which are lightweight, possess high-specific strengths and have historically been challenging to design and strengthen due to their mechanical anisotropy. Here, we modulate their microstructure by adding or removing precipitates to demon
Denoising Diffusion models have demonstrated their proficiency for generative sampling. However, generating good samples often requires many iterations. Consequently, techniques such as binary time-distillation (BTD) have been proposed to reduce the number of network calls for a fixed architecture. In this paper, we introduce TRAnsitive Closure Time-distillation (TRACT), a new method that extends BTD. For single step diffusion,TRACT improves FID by up to 2.4x on the same architecture, and achieves new single-step Denoising Diffusion Implicit Models (DDIM) state-of-the-art FID (7.4 for ImageNet64, 3.8 for CIFAR10). Finally we tease apart the method through extended ablations. The PyTorch implementation will be released soon.
The weak value approximation has been in use for thirty-five years, but it has not as of yet received a truly complete derivation, leaving its mathematical validity in a state of limbo. Herein, I fill this gap, deriving the weak value approximation under the von Neumann and qubit probe models. Not only does this provide a level of mathematical support to the weak value approximation not attained in previous works, but the techniques demonstrated in the process might be usable by others to forge similar derivations for alternative models, thus teasing the possibility of even broader validation in the future.
We investigate the consequences of models where dark sector quarks could be produced at the LHC, which subsequently undergo a dark parton shower, generating jets of dark hadrons that ultimately decay back to Standard Model hadrons. This yields collider objects that can be nearly indistinguishable from Standard Model jets, motivating the reliance on substructure observables to tease out the signal. However, substructure predictions are sensitive to the details of the incalculable dark hadronization. We show that the Lund jet plane provides a very effective tool for designing observables that are resilient against the unknown impact of dark hadronization on the substructure properties of dark sector jets.
We propose a framework by which websites can coordinate to detect credential stuffing on individual user accounts. Our detection algorithm teases apart normal login behavior (involving password reuse, entering correct passwords into the wrong sites, etc.) from credential stuffing, by leveraging modern anomaly detection and carefully tracking suspicious logins. Websites coordinate using a novel private membership-test protocol, thereby ensuring that information about passwords is not leaked; this protocol is highly scalable, partly due to its use of cuckoo filters, and is more secure than similarly scalable alternatives in an important measure that we define. We use probabilistic model checking to estimate our credential-stuffing detection accuracy across a range of operating points. These methods might be of independent interest for their novel application of formal methods to estimate the usability impacts of our design. We show that even a minimal-infrastructure deployment of our framework should already support the combined login load experienced by the airline, hotel, retail, and consumer banking industries in the U.S.
A multi-state version of an animal movement analysis method based on conditional logistic regression, called Step Selection Function (SSF), is proposed. In ecology SSF is developed from a comparison between the observed location of an animal and randomly sampled locations at each time step. Interpretation of the parameters in the multi-state model and the impact of different sampling schemes for the random locations are discussed. We prove the equivalence between the new model and a random walk model on the plane. This equivalence allows one to use both pure movement and local discrete choice behaviors in identifying the model's hidden states. The new method is used to model the movement behavior of GPS-collared bison in Prince Albert National Park, Canada. The multi-state SSF successfully teases apart areas used to forage and to travel. The analysis thus provides valuable insights into how bison adjust their movement to habitat features, thereby revealing spatial determinants of functional connectivity in heterogeneous landscapes.
There are different definitions of what a troll is. Certainly, a troll can be somebody who teases people to make them angry, or somebody who offends people, or somebody who wants to dominate any single discussion, or somebody who tries to manipulate people's opinion (sometimes for money), etc. The last definition is the one that dominates the public discourse in Bulgaria and Eastern Europe, and this is our focus in this paper. In our work, we examine two types of opinion manipulation trolls: paid trolls that have been revealed from leaked reputation management contracts and mentioned trolls that have been called such by several different people. We show that these definitions are sensible: we build two classifiers that can distinguish a post by such a paid troll from one by a non-troll with 81-82% accuracy; the same classifier achieves 81-82% accuracy on so called mentioned troll vs. non-troll posts.
In his 1981 article, Roberts highlights the term 'stellify' defined as "to transform (a person or thing) into a star or constellation, to place among the stars." Using the case of the Tabwa people of central Africa, not the Democratic Republic of Congo, Roberts presents among other things the sky as a mnemonic for remembering migrations and remembering culture heroes. We do not know the details of the processes of stellification, however we do know what has been stellified in many cultures by examining their names for stars and asterisms and their skylore. Of the many ideas presented in his latest book, Aveni teases out the ideas of the sky stories having connections to celestial motions, as well as being a mnemonic for remembering seasonal activities and a mnemonic for remembering locally embedded moral, ethical, and sociocultural codes, thus overlapping with Roberts' supposition of the sky serving as a mnemonic. I draw on case studies to flesh out three themes 1. celestial motions, 2. moral, ethical, and sociocultural codes, and 3. seasonal activities within African sky stories. As previously stated, though the human process of assigning names and stories to the night sky as well
Researchers have proposed that black holes stop evaporating at the last moment, leaving behind tiny remnants that preserve all the information they contain。 The same seven-dimensional geometry behind this idea could also help explain why elementary particles have mass