In purely non-dissipative systems, Lagrangian and Hamiltonian reduction have proven to be powerful tools for deriving physical models with exact conservation laws. We have discovered a hint that an analogous reduction method exists also for dissipative systems that respect the First and Second Laws of Thermodynamics. In this paper, we show that modern electrostatic gyrokinetics, a reduced plasma turbulence model, exhibits a serendipitous metriplectic structure. Metriplectic dynamics in general is a well developed formalism for extending the concept of Poisson brackets to dissipative systems. Better yet, our discovery enables an intuitive particle-in-cell discretization of the collision operator that also satisfies the First and Second Laws of thermodynamics. These results suggest that collisional gyrokinetics, and other dissipative physical models that obey the Laws of Thermodynamics, could be obtained using an as-yet undiscovered metriplectic reduction theory and that numerical methods could benefit from such theory significantly. Once uncovered, the theory would generalize Lagrangian and Hamiltonian reduction in a substantial manner.
In a recent paper, Enayat and Le lyk [2024] show that second order arithmetic and countable set theory are not definitionally equivalent. It is well known that these theories are biinterpretable. Thus, we have a pair of natural theories that llustrate a meaningful difference between definitional equivalence and bi-interpretability. This is particularly interesting given that Visser and Friedman [2014] have shown that a wide class of natural foundational theories in mathematics are such that if they are bi-interpretable, then they are also definitionally equivalent. The proof offered by Enayat and Le lyk makes use of an inaccessible cardinal. In this short note, we show that the failure of bi-interpretability can be established in Peano Arithmetic merely supposing that one of our target theories are consistent.
Artificial neural networks can acquire many aspects of human knowledge from data, making them promising as models of human learning. But what those networks can learn depends upon their inductive biases -- the factors other than the data that influence the solutions they discover -- and the inductive biases of neural networks remain poorly understood, limiting our ability to draw conclusions about human learning from the performance of these systems. Cognitive scientists and machine learning researchers often focus on the architecture of a neural network as a source of inductive bias. In this paper we explore the impact of another source of inductive bias -- the initial weights of the network -- using meta-learning as a tool for finding initial weights that are adapted for specific problems. We evaluate four widely-used architectures -- MLPs, CNNs, LSTMs, and Transformers -- by meta-training 430 different models across three tasks requiring different biases and forms of generalization. We find that meta-learning can substantially reduce or entirely eliminate performance differences across architectures and data representations, suggesting that these factors may be less important as
Can language models reliably predict that possible events are more likely than merely improbable ones? By teasing apart possibility, typicality, and contextual relatedness, we show that despite the results of previous work, language models' ability to do this is far from robust. In fact, under certain conditions, all models tested - including Llama 3, Gemma 2, and Mistral NeMo - perform at worse-than-chance level, assigning higher probabilities to impossible sentences such as 'the car was given a parking ticket by the brake' than to merely unlikely sentences such as 'the car was given a parking ticket by the explorer'.
Most expressivity results for transformers treat them as language recognizers -- devices that accept or reject strings -- rather than as they are used in practice: as language models that generate strings autoregressively and probabilistically. We characterize the probability distributions that transformer language models can express. We show that making transformer language recognizers autoregressive can sometimes increase their expressivity, and that making them probabilistic can break equivalences that hold in the non-probabilistic case. Our overall contribution is to tease apart what functions transformers are capable of expressing in their most common use case as language models.
Beyond diagonal reconfigurable intelligent surface (BD-RIS) is a new architecture for RIS where elements are interconnected to provide more wave manipulation flexibility than traditional single connected RIS, enhancing data rate and coverage. However, channel estimation for BD-RIS is challenging due to the more complex multiple-connection structure involving their scattering elements. To address this issue, this paper proposes a decoupled channel estimation method for BD-RIS that yields separate estimates of the involved channels to enhance the accuracy of the overall combined channel by capitalizing on its Kronecker structure. Starting from a least squares estimate of the combined channel and by properly reshaping the resulting filtered signal, the proposed algorithm resorts to a Khatri-Rao Factorization (KRF) method that teases out the individual channels based on simple rank-one matrix approximation steps. Numerical results show that the proposed decoupled channel estimation yields more accurate channel estimates than the classical least squares scheme.
Socio-demographic prompting is a commonly employed approach to study cultural biases in LLMs as well as for aligning models to certain cultures. In this paper, we systematically probe four LLMs (Llama 3, Mistral v0.2, GPT-3.5 Turbo and GPT-4) with prompts that are conditioned on culturally sensitive and non-sensitive cues, on datasets that are supposed to be culturally sensitive (EtiCor and CALI) or neutral (MMLU and ETHICS). We observe that all models except GPT-4 show significant variations in their responses on both kinds of datasets for both kinds of prompts, casting doubt on the robustness of the culturally-conditioned prompting as a method for eliciting cultural bias in models or as an alignment strategy. The work also calls rethinking the control experiment design to tease apart the cultural conditioning of responses from "placebo effect", i.e., random perturbations of model responses due to arbitrary tokens in the prompt.
Given the growing importance of AI literacy, we decided to write this tutorial to help narrow the gap between the discourse among those who study language models -- the core technology underlying ChatGPT and similar products -- and those who are intrigued and want to learn more about them. In short, we believe the perspective of researchers and educators can add some clarity to the public's understanding of the technologies beyond what's currently available, which tends to be either extremely technical or promotional material generated about products by their purveyors. Our approach teases apart the concept of a language model from products built on them, from the behaviors attributed to or desired from those products, and from claims about similarity to human cognition. As a starting point, we (1) offer a scientific viewpoint that focuses on questions amenable to study through experimentation; (2) situate language models as they are today in the context of the research that led to their development; and (3) describe the boundaries of what is known about the models at this writing.
The multiple access channel (MAC) capacity with feedback is considered under feedback models designed to tease out which factors contribute to the MAC feedback capacity benefit. Comparing the capacity of a MAC with ``perfect'' feedback, which causally delivers to the transmitters the true channel output, to that of a MAC with ``independent'' feedback, which causally delivers to the transmitters an independent instance of that same channel output, allows separation of effects like cooperation from alternative feedback benefits such as knowledge of the channel instance. Proving that the Cover-Leung (CL) achievability bound, which is known to be loose for some channels, is achievable also under (shared or distinct) independent feedback at the transmitters shows that the CL bound does not require transmitter knowledge of the channel instance. Proving that each transmitter's maximal rate under independent feedback exceeds that under perfect feedback highlights the potential power of an independent look at the channel output.
The weak value approximation has been in use for thirty-five years, but it has not as of yet received a truly complete derivation, leaving its mathematical validity in a state of limbo. Herein, I fill this gap, deriving the weak value approximation under the von Neumann and qubit probe models. Not only does this provide a level of mathematical support to the weak value approximation not attained in previous works, but the techniques demonstrated in the process might be usable by others to forge similar derivations for alternative models, thus teasing the possibility of even broader validation in the future.
The impressive performance of recent language models across a wide range of tasks suggests that they possess a degree of abstract reasoning skills. Are these skills general and transferable, or specialized to specific tasks seen during pretraining? To disentangle these effects, we propose an evaluation framework based on "counterfactual" task variants that deviate from the default assumptions underlying standard tasks. Across a suite of 11 tasks, we observe nontrivial performance on the counterfactual variants, but nevertheless find that performance substantially and consistently degrades compared to the default conditions. This suggests that while current LMs may possess abstract task-solving skills to an extent, they often also rely on narrow, non-transferable procedures for task-solving. These results motivate a more careful interpretation of language model performance that teases apart these aspects of behavior.
We investigate the consequences of models where dark sector quarks could be produced at the LHC, which subsequently undergo a dark parton shower, generating jets of dark hadrons that ultimately decay back to Standard Model hadrons. This yields collider objects that can be nearly indistinguishable from Standard Model jets, motivating the reliance on substructure observables to tease out the signal. However, substructure predictions are sensitive to the details of the incalculable dark hadronization. We show that the Lund jet plane provides a very effective tool for designing observables that are resilient against the unknown impact of dark hadronization on the substructure properties of dark sector jets.
The response of metals and their microstructures under extreme dynamic conditions can be markedly different from that under quasistatic conditions. Traditionally, high strain rates and shock stresses are measured using cumbersome and expensive methods such as the Kolsky bar or large spall experiments. These methods are low throughput and do not facilitate high-fidelity microstructure-property linkages. In this work, we combine two powerful small-scale testing methods, custom nanoindentation, and laser-driven micro-flyer shock, to measure the dynamic and spall strength of metals. The nanoindentation system is configured to test samples from quasistatic to dynamic strain rate regimes (10$^{-3}$ s$^{-1}$ to 10$^{+4}$ s$^{-1}$). The laser-driven micro-flyer shock system can test samples through impact loading between 10$^{+5}$ s$^{-1}$ to 10$^{+7}$ s$^{-1}$ strain rates, triggering spall failure. The model material used for testing is Magnesium alloys, which are lightweight, possess high-specific strengths and have historically been challenging to design and strengthen due to their mechanical anisotropy. Here, we modulate their microstructure by adding or removing precipitates to demon
Denoising Diffusion models have demonstrated their proficiency for generative sampling. However, generating good samples often requires many iterations. Consequently, techniques such as binary time-distillation (BTD) have been proposed to reduce the number of network calls for a fixed architecture. In this paper, we introduce TRAnsitive Closure Time-distillation (TRACT), a new method that extends BTD. For single step diffusion,TRACT improves FID by up to 2.4x on the same architecture, and achieves new single-step Denoising Diffusion Implicit Models (DDIM) state-of-the-art FID (7.4 for ImageNet64, 3.8 for CIFAR10). Finally we tease apart the method through extended ablations. The PyTorch implementation will be released soon.
There are different definitions of what a troll is. Certainly, a troll can be somebody who teases people to make them angry, or somebody who offends people, or somebody who wants to dominate any single discussion, or somebody who tries to manipulate people's opinion (sometimes for money), etc. The last definition is the one that dominates the public discourse in Bulgaria and Eastern Europe, and this is our focus in this paper. In our work, we examine two types of opinion manipulation trolls: paid trolls that have been revealed from leaked reputation management contracts and mentioned trolls that have been called such by several different people. We show that these definitions are sensible: we build two classifiers that can distinguish a post by such a paid troll from one by a non-troll with 81-82% accuracy; the same classifier achieves 81-82% accuracy on so called mentioned troll vs. non-troll posts.
In his 1981 article, Roberts highlights the term 'stellify' defined as "to transform (a person or thing) into a star or constellation, to place among the stars." Using the case of the Tabwa people of central Africa, not the Democratic Republic of Congo, Roberts presents among other things the sky as a mnemonic for remembering migrations and remembering culture heroes. We do not know the details of the processes of stellification, however we do know what has been stellified in many cultures by examining their names for stars and asterisms and their skylore. Of the many ideas presented in his latest book, Aveni teases out the ideas of the sky stories having connections to celestial motions, as well as being a mnemonic for remembering seasonal activities and a mnemonic for remembering locally embedded moral, ethical, and sociocultural codes, thus overlapping with Roberts' supposition of the sky serving as a mnemonic. I draw on case studies to flesh out three themes 1. celestial motions, 2. moral, ethical, and sociocultural codes, and 3. seasonal activities within African sky stories. As previously stated, though the human process of assigning names and stories to the night sky as well
We consider time-harmonic acoustic scattering by planar sound-soft (Dirichlet) and sound-hard (Neumann) screens. In contrast to previous studies, in which the domain occupied by the screen is assumed to be Lipschitz or smoother, we consider screens occupying an arbitrary bounded open set in the plane. Thus our study includes cases where the closure of the domain occupied by the screen has larger planar Lebesgue measure than the screen, as can happen, for example, when the screen has a fractal boundary. We show how to formulate well-posed boundary value problems for such scattering problems, our arguments depending on results on the coercivity of the acoustic single-layer and hypersingular boundary integral operators, and on properties of Sobolev spaces on general open sets which appear to be new. Our analysis teases out the explicit wavenumber dependence of the continuity and coercivity constants of the boundary integral operators, viewed as mappings between fractional Sobolev spaces, this in part extending previous results of Ha-Duong. We also consider the complementary problem of propagation through a bounded aperture in an infinite planar screen.
A multi-state version of an animal movement analysis method based on conditional logistic regression, called Step Selection Function (SSF), is proposed. In ecology SSF is developed from a comparison between the observed location of an animal and randomly sampled locations at each time step. Interpretation of the parameters in the multi-state model and the impact of different sampling schemes for the random locations are discussed. We prove the equivalence between the new model and a random walk model on the plane. This equivalence allows one to use both pure movement and local discrete choice behaviors in identifying the model's hidden states. The new method is used to model the movement behavior of GPS-collared bison in Prince Albert National Park, Canada. The multi-state SSF successfully teases apart areas used to forage and to travel. The analysis thus provides valuable insights into how bison adjust their movement to habitat features, thereby revealing spatial determinants of functional connectivity in heterogeneous landscapes.
Well-tempering is a promising classical method of dynamically screening an arbitrarily large vacuum energy and generating a late-time, low energy de Sitter vacuum state. In this paper, we study for the first time self-tuning in teleparallel gravity and obtain well-tempered cosmological models in the teleparallel gravity analogue of Horndeski theory. This broadens the scope of well-tempered cosmology and teases the potentially far richer cosmological dynamics that could be anchored on teleperallel gravity. We expand the well-tempered recipe to its most general form so far and use it to search for the first well-tempered cosmologies in teleparallel gravity. We also study the cosmological dynamics in a well-tempered model and demonstrate the dynamical stability of the vacuum, the compatibility with a matter era, and the stability of the vacuum through a phase transition.
Predicting gender from iris images has been reported by several researchers as an application of machine learning in biometrics. Recent works on this topic have suggested that the preponderance of the gender cues is located in the periocular region rather than in the iris texture itself. This paper focuses on teasing out whether the information for gender prediction is in the texture of the iris stroma, the periocular region, or both. We present a larger dataset for gender from iris, and evaluate gender prediction accuracy using linear SVM and CNN, comparing hand-crafted and deep features. We use probabilistic occlusion masking to gain insight on the problem. Results suggest the discriminative power of the iris texture for gender is weaker than previously thought, and that the gender-related information is primarily in the periocular region.