In purely non-dissipative systems, Lagrangian and Hamiltonian reduction have proven to be powerful tools for deriving physical models with exact conservation laws. We have discovered a hint that an analogous reduction method exists also for dissipative systems that respect the First and Second Laws of Thermodynamics. In this paper, we show that modern electrostatic gyrokinetics, a reduced plasma turbulence model, exhibits a serendipitous metriplectic structure. Metriplectic dynamics in general is a well developed formalism for extending the concept of Poisson brackets to dissipative systems. Better yet, our discovery enables an intuitive particle-in-cell discretization of the collision operator that also satisfies the First and Second Laws of thermodynamics. These results suggest that collisional gyrokinetics, and other dissipative physical models that obey the Laws of Thermodynamics, could be obtained using an as-yet undiscovered metriplectic reduction theory and that numerical methods could benefit from such theory significantly. Once uncovered, the theory would generalize Lagrangian and Hamiltonian reduction in a substantial manner.
Most expressivity results for transformers treat them as language recognizers -- devices that accept or reject strings -- rather than as they are used in practice: as language models that generate strings autoregressively and probabilistically. We characterize the probability distributions that transformer language models can express. We show that making transformer language recognizers autoregressive can sometimes increase their expressivity, and that making them probabilistic can break equivalences that hold in the non-probabilistic case. Our overall contribution is to tease apart what functions transformers are capable of expressing in their most common use case as language models.
In a recent paper, Enayat and Le lyk [2024] show that second order arithmetic and countable set theory are not definitionally equivalent. It is well known that these theories are biinterpretable. Thus, we have a pair of natural theories that llustrate a meaningful difference between definitional equivalence and bi-interpretability. This is particularly interesting given that Visser and Friedman [2014] have shown that a wide class of natural foundational theories in mathematics are such that if they are bi-interpretable, then they are also definitionally equivalent. The proof offered by Enayat and Le lyk makes use of an inaccessible cardinal. In this short note, we show that the failure of bi-interpretability can be established in Peano Arithmetic merely supposing that one of our target theories are consistent.
Artificial neural networks can acquire many aspects of human knowledge from data, making them promising as models of human learning. But what those networks can learn depends upon their inductive biases -- the factors other than the data that influence the solutions they discover -- and the inductive biases of neural networks remain poorly understood, limiting our ability to draw conclusions about human learning from the performance of these systems. Cognitive scientists and machine learning researchers often focus on the architecture of a neural network as a source of inductive bias. In this paper we explore the impact of another source of inductive bias -- the initial weights of the network -- using meta-learning as a tool for finding initial weights that are adapted for specific problems. We evaluate four widely-used architectures -- MLPs, CNNs, LSTMs, and Transformers -- by meta-training 430 different models across three tasks requiring different biases and forms of generalization. We find that meta-learning can substantially reduce or entirely eliminate performance differences across architectures and data representations, suggesting that these factors may be less important as
Can language models reliably predict that possible events are more likely than merely improbable ones? By teasing apart possibility, typicality, and contextual relatedness, we show that despite the results of previous work, language models' ability to do this is far from robust. In fact, under certain conditions, all models tested - including Llama 3, Gemma 2, and Mistral NeMo - perform at worse-than-chance level, assigning higher probabilities to impossible sentences such as 'the car was given a parking ticket by the brake' than to merely unlikely sentences such as 'the car was given a parking ticket by the explorer'.
Beyond diagonal reconfigurable intelligent surface (BD-RIS) is a new architecture for RIS where elements are interconnected to provide more wave manipulation flexibility than traditional single connected RIS, enhancing data rate and coverage. However, channel estimation for BD-RIS is challenging due to the more complex multiple-connection structure involving their scattering elements. To address this issue, this paper proposes a decoupled channel estimation method for BD-RIS that yields separate estimates of the involved channels to enhance the accuracy of the overall combined channel by capitalizing on its Kronecker structure. Starting from a least squares estimate of the combined channel and by properly reshaping the resulting filtered signal, the proposed algorithm resorts to a Khatri-Rao Factorization (KRF) method that teases out the individual channels based on simple rank-one matrix approximation steps. Numerical results show that the proposed decoupled channel estimation yields more accurate channel estimates than the classical least squares scheme.
Socio-demographic prompting is a commonly employed approach to study cultural biases in LLMs as well as for aligning models to certain cultures. In this paper, we systematically probe four LLMs (Llama 3, Mistral v0.2, GPT-3.5 Turbo and GPT-4) with prompts that are conditioned on culturally sensitive and non-sensitive cues, on datasets that are supposed to be culturally sensitive (EtiCor and CALI) or neutral (MMLU and ETHICS). We observe that all models except GPT-4 show significant variations in their responses on both kinds of datasets for both kinds of prompts, casting doubt on the robustness of the culturally-conditioned prompting as a method for eliciting cultural bias in models or as an alignment strategy. The work also calls rethinking the control experiment design to tease apart the cultural conditioning of responses from "placebo effect", i.e., random perturbations of model responses due to arbitrary tokens in the prompt.
Given the growing importance of AI literacy, we decided to write this tutorial to help narrow the gap between the discourse among those who study language models -- the core technology underlying ChatGPT and similar products -- and those who are intrigued and want to learn more about them. In short, we believe the perspective of researchers and educators can add some clarity to the public's understanding of the technologies beyond what's currently available, which tends to be either extremely technical or promotional material generated about products by their purveyors. Our approach teases apart the concept of a language model from products built on them, from the behaviors attributed to or desired from those products, and from claims about similarity to human cognition. As a starting point, we (1) offer a scientific viewpoint that focuses on questions amenable to study through experimentation; (2) situate language models as they are today in the context of the research that led to their development; and (3) describe the boundaries of what is known about the models at this writing.
The impressive performance of recent language models across a wide range of tasks suggests that they possess a degree of abstract reasoning skills. Are these skills general and transferable, or specialized to specific tasks seen during pretraining? To disentangle these effects, we propose an evaluation framework based on "counterfactual" task variants that deviate from the default assumptions underlying standard tasks. Across a suite of 11 tasks, we observe nontrivial performance on the counterfactual variants, but nevertheless find that performance substantially and consistently degrades compared to the default conditions. This suggests that while current LMs may possess abstract task-solving skills to an extent, they often also rely on narrow, non-transferable procedures for task-solving. These results motivate a more careful interpretation of language model performance that teases apart these aspects of behavior.
The response of metals and their microstructures under extreme dynamic conditions can be markedly different from that under quasistatic conditions. Traditionally, high strain rates and shock stresses are measured using cumbersome and expensive methods such as the Kolsky bar or large spall experiments. These methods are low throughput and do not facilitate high-fidelity microstructure-property linkages. In this work, we combine two powerful small-scale testing methods, custom nanoindentation, and laser-driven micro-flyer shock, to measure the dynamic and spall strength of metals. The nanoindentation system is configured to test samples from quasistatic to dynamic strain rate regimes (10$^{-3}$ s$^{-1}$ to 10$^{+4}$ s$^{-1}$). The laser-driven micro-flyer shock system can test samples through impact loading between 10$^{+5}$ s$^{-1}$ to 10$^{+7}$ s$^{-1}$ strain rates, triggering spall failure. The model material used for testing is Magnesium alloys, which are lightweight, possess high-specific strengths and have historically been challenging to design and strengthen due to their mechanical anisotropy. Here, we modulate their microstructure by adding or removing precipitates to demon
The multiple access channel (MAC) capacity with feedback is considered under feedback models designed to tease out which factors contribute to the MAC feedback capacity benefit. Comparing the capacity of a MAC with ``perfect'' feedback, which causally delivers to the transmitters the true channel output, to that of a MAC with ``independent'' feedback, which causally delivers to the transmitters an independent instance of that same channel output, allows separation of effects like cooperation from alternative feedback benefits such as knowledge of the channel instance. Proving that the Cover-Leung (CL) achievability bound, which is known to be loose for some channels, is achievable also under (shared or distinct) independent feedback at the transmitters shows that the CL bound does not require transmitter knowledge of the channel instance. Proving that each transmitter's maximal rate under independent feedback exceeds that under perfect feedback highlights the potential power of an independent look at the channel output.
We investigate the consequences of models where dark sector quarks could be produced at the LHC, which subsequently undergo a dark parton shower, generating jets of dark hadrons that ultimately decay back to Standard Model hadrons. This yields collider objects that can be nearly indistinguishable from Standard Model jets, motivating the reliance on substructure observables to tease out the signal. However, substructure predictions are sensitive to the details of the incalculable dark hadronization. We show that the Lund jet plane provides a very effective tool for designing observables that are resilient against the unknown impact of dark hadronization on the substructure properties of dark sector jets.
The weak value approximation has been in use for thirty-five years, but it has not as of yet received a truly complete derivation, leaving its mathematical validity in a state of limbo. Herein, I fill this gap, deriving the weak value approximation under the von Neumann and qubit probe models. Not only does this provide a level of mathematical support to the weak value approximation not attained in previous works, but the techniques demonstrated in the process might be usable by others to forge similar derivations for alternative models, thus teasing the possibility of even broader validation in the future.
Denoising Diffusion models have demonstrated their proficiency for generative sampling. However, generating good samples often requires many iterations. Consequently, techniques such as binary time-distillation (BTD) have been proposed to reduce the number of network calls for a fixed architecture. In this paper, we introduce TRAnsitive Closure Time-distillation (TRACT), a new method that extends BTD. For single step diffusion,TRACT improves FID by up to 2.4x on the same architecture, and achieves new single-step Denoising Diffusion Implicit Models (DDIM) state-of-the-art FID (7.4 for ImageNet64, 3.8 for CIFAR10). Finally we tease apart the method through extended ablations. The PyTorch implementation will be released soon.
A multi-state version of an animal movement analysis method based on conditional logistic regression, called Step Selection Function (SSF), is proposed. In ecology SSF is developed from a comparison between the observed location of an animal and randomly sampled locations at each time step. Interpretation of the parameters in the multi-state model and the impact of different sampling schemes for the random locations are discussed. We prove the equivalence between the new model and a random walk model on the plane. This equivalence allows one to use both pure movement and local discrete choice behaviors in identifying the model's hidden states. The new method is used to model the movement behavior of GPS-collared bison in Prince Albert National Park, Canada. The multi-state SSF successfully teases apart areas used to forage and to travel. The analysis thus provides valuable insights into how bison adjust their movement to habitat features, thereby revealing spatial determinants of functional connectivity in heterogeneous landscapes.
There is an ongoing debate on personalization, adapting results to the unique user exploiting a user's personal history, versus customization, adapting results to a group profile sharing one or more characteristics with the user at hand. Personal profiles are often sparse, due to cold start problems and the fact that users typically search for new items or information, necessitating to back-off to customization, but group profiles often suffer from accidental features brought in by the unique individual contributing to the group. In this paper we propose a generalized group profiling approach that teases apart the exact contribution of the individual user level and the "abstract" group level by extracting a latent model that captures all, and only, the essential features of the whole group. Our main findings are the followings. First, we propose an efficient way of group profiling which implicitly eliminates the general and specific features from users' models in a group and takes out the abstract model representing the whole group. Second, we employ the resulting models in the task of contextual suggestion. We analyse different grouping criteria and we find that group-based sugges
The ability to minimize the thermal conductivity of dielectrics with minimal structural intervention that could affect electrical properties is an important capability for engineering thermoelectric efficiency in low-cost materials such as Si. We recently reported the discovery of special arrangements for nanoscale pores in Si that produce a particularly large reduction in thermal conductivity accompanied by strongly anticorrelated heat current fluctuations, a phenomenon that is missed by the diffuse adiabatic boundary conditions conventionally used in numerical Boltzmann transport models. This manuscript presents the results of molecular dynamics simulations and a Monte Carlo ray tracing model that teases apart this phenomenon to reveal that special pore layouts elastically backscatter long-wavelength heat-carrying phonons. This means that heat carriage by a phonon before scattering is undone by the scattered phonon, resulting in an effective mean-free-path that is significantly shorter than the geometric line-of-sight to the pores. This effect is particularly noticeable for the long-wavelength, long mean-free-path phonons whose transport is impeded drastically more than is expect
We consider time-harmonic acoustic scattering by planar sound-soft (Dirichlet) and sound-hard (Neumann) screens. In contrast to previous studies, in which the domain occupied by the screen is assumed to be Lipschitz or smoother, we consider screens occupying an arbitrary bounded open set in the plane. Thus our study includes cases where the closure of the domain occupied by the screen has larger planar Lebesgue measure than the screen, as can happen, for example, when the screen has a fractal boundary. We show how to formulate well-posed boundary value problems for such scattering problems, our arguments depending on results on the coercivity of the acoustic single-layer and hypersingular boundary integral operators, and on properties of Sobolev spaces on general open sets which appear to be new. Our analysis teases out the explicit wavenumber dependence of the continuity and coercivity constants of the boundary integral operators, viewed as mappings between fractional Sobolev spaces, this in part extending previous results of Ha-Duong. We also consider the complementary problem of propagation through a bounded aperture in an infinite planar screen.
In his 1981 article, Roberts highlights the term 'stellify' defined as "to transform (a person or thing) into a star or constellation, to place among the stars." Using the case of the Tabwa people of central Africa, not the Democratic Republic of Congo, Roberts presents among other things the sky as a mnemonic for remembering migrations and remembering culture heroes. We do not know the details of the processes of stellification, however we do know what has been stellified in many cultures by examining their names for stars and asterisms and their skylore. Of the many ideas presented in his latest book, Aveni teases out the ideas of the sky stories having connections to celestial motions, as well as being a mnemonic for remembering seasonal activities and a mnemonic for remembering locally embedded moral, ethical, and sociocultural codes, thus overlapping with Roberts' supposition of the sky serving as a mnemonic. I draw on case studies to flesh out three themes 1. celestial motions, 2. moral, ethical, and sociocultural codes, and 3. seasonal activities within African sky stories. As previously stated, though the human process of assigning names and stories to the night sky as well