Large language models (LLMs) have been shown to encode truth of statements in their activation space along a linear truth direction. Previous studies have argued that these directions are universal in certain aspects, while more recent work has questioned this conclusion drawing on limited generalization across some settings. In this work, we identify a number of limits of truth-direction universality that have not been previously understood. We first show that truth directions are highly layer-dependent, and that a full understanding of universality requires probing at many layers in the model. We then show that truth directions depend heavily on task type, emerging in earlier layers for factual and later layers for reasoning tasks; they also vary in performance across levels of task complexity. Finally, we show that model instructions dramatically affect truth directions; simple correctness evaluation instructions significantly affect the generalization ability of truth probes. Our findings indicate that universality claims for truth directions are more limited than previously known, with significant differences observable for various model layers, task difficulties, task types,
Despite increasing progress in development of methods for generating visual counterfactual explanations, especially with the recent rise of Denoising Diffusion Probabilistic Models, previous works consider them as an entirely local technique. In this work, we take the first step at globalizing them. Specifically, we discover that the latent space of Diffusion Autoencoders encodes the inference process of a given classifier in the form of global directions. We propose a novel proxy-based approach that discovers two types of these directions with the use of only single image in an entirely black-box manner. Precisely, g-directions allow for flipping the decision of a given classifier on an entire dataset of images, while h-directions further increase the diversity of explanations. We refer to them in general as Global Counterfactual Directions (GCDs). Moreover, we show that GCDs can be naturally combined with Latent Integrated Gradients resulting in a new black-box attribution method, while simultaneously enhancing the understanding of counterfactual explanations. We validate our approach on existing benchmarks and show that it generalizes to real-world use-cases.
Generative AI is directional: it performs well in some task directions and poorly in others. Knowledge work is directional and endogenous as well: workers can satisfy the same job requirements with different mixes of tasks. We develop a high-dimensional model of AI adoption in which a worker uses a tool when it raises their output. Both the worker and the AI tool can perform a variety of tasks, which we model as convex production possibility sets. Because the tool requires supervision from the worker's own time and attention budget, adoption is a team-production decision, similar to hiring a coworker. The key sufficient statistics are the worker's pre-AI shadow prices: these equal the output gain from a small relaxation in each task direction, and they generally differ from the worker's observed activity mix. As AI capability improves, the set of adopted directions expands in a cone centered on these autarky prices. Near the entry threshold, small capability improvements generate large extensive-margin expansions in adoption. The model also delivers a structured intensive margin: between the entry and all-in thresholds, optimal use is partial. We parametrize the model in a simple b
We give a descent-free, alignment-free measurement of singular structure on trained networks. At a single frozen checkpoint the read recovers the order $k$ of each dead direction from the directional-Fisher rate, the master invariant from which the per-direction learning coefficient $1/(2k)$ follows exactly, in whatever basis the optimizer left. The same read classifies each direction, separating a genuine singularity, whose order the architecture fixes, from a flat gauge symmetry; the directional-Fisher magnitude settles the cases the order cannot. A pluggable detector supplies the directions for transformer, convolutional, and normalisation layers. The read recovers the architecture-predicted order across constructed cells and trained networks, including a fine-tuned vision transformer whose dead structure is the LayerNorm-kernel gauge and a from-scratch one whose compressed MLP forms a node-death at its activation order. Where the singular structure enumerates, the per-direction orders assemble, through the typed intersection of the loci, into the global coefficient $(λ, m)$ matching the closed form. The method removes the canonical-alignment and descent preconditions of the und
This paper presents the hybrid solver for a $CO_2$ sequestration problem. The solver uses the IGA-ADS (IsoGeometric Analysis Alternating Directions solver) to compute the saturation scalar field update using the explicit method, and CRVPINN (Collocation-based Robust Variational Physics Informed Neural Networks solver) to compute the pressure scalar field. The study focuses on simulating the physical behavior of $CO_2$ in porous structures, excluding chemical reactions. The mathematical model is based on Darcy's Law. The CRVPINN is pretrained on the initial pressure configuration, and the time step pressure updates require only 100 iterations of the Adam method per time step. We compare our hybrid IGA-ADS solver, coupled with the CRVPINN method, with a baseline of the IGA-ADS solver coupled with the MUMPS direct solver. Our hybrid solver is over 3 times faster on a single computational node from the ARES cluster of ACK CYFRONET. Future work includes extensive testing, inverse problem solving, and potential application to $H_2$ storage problems.
In this paper we study asymptotic directions in the tangent bundle of the moduli space ${\mathcal M}_g$ of curves of genus $g$, namely those tangent directions that are annihilated by the second fundamental form of the Torelli map. We give examples of asymptotic directions for any $g \geq 4$. We prove that if the rank $d$ of a tangent direction $ζ\in H^1(T_C)$ (with respect to the infinitesimal deformation map) is less than the Clifford index of the curve $C$, then $ζ$ is not asymptotic. If the rank of $ζ$ is equal to the Clifford index of the curve, we give sufficient conditions ensuring that the infinitesimal deformation $ζ$ is not asymptotic. Then we determine all asymptotic directions of rank 1 and we give an almost complete description of asymptotic directions of rank 2.
Model merging has emerged as a practical paradigm for integrating multiple independently trained models into a single model without joint retraining. Previous studies have demonstrated the effectiveness of combining parameters through strategies such as parameter decomposition, coefficient optimization, and subspace learning, significantly reducing the need for expensive joint training and achieving strong empirical performance across diverse tasks. However, these approaches predominantly treat merging as a problem of parameter space decomposition or fusion coefficient optimization, while overlooking the critical role of directional information in both parameter and feature spaces. In practice, naïve merging introduces inconsistencies in dominant parameter directions and disrupts structural coherence across models, which can degrade performance. Moreover, coefficient-based optimization methods implicitly assume compatible feature-space directions across models. However, Neural Collapse indicates that class features follow structured directional patterns, which may differ across independently trained models, making coefficient optimization alone insufficient. In this work, we emphas
Vertical Take-Off and Landing (VTOL) vehicles are gaining traction in both the delivery drone market and passenger transportation, driving the development of Urban Air Mobility (UAM) systems. UAM seeks to alleviate road congestion in dense urban areas by leveraging urban airspace. To handle UAM traffic, vertiport terminals (vertiminals) play a critical role in supporting VTOL vehicle operations such as take-offs, landings, taxiing, passenger boarding, refueling or charging, and maintenance. Efficient scheduling algorithms are essential to manage these operations and optimize vertiminal throughput while ensuring safety protocols. Unlike fixed-wing aircraft, which rely on runways for take-off and climbing in fixed directions, VTOL vehicles can utilize multiple surface directions for climbing and approach. This flexibility necessitates specialized scheduling methods. We propose a Mixed Integer Linear Program (MILP) formulation to holistically optimize vertiminal operations, including taxiing, climbing (or approach) using multiple directions, and turnaround at gates. The proposed MILP reduces delays by up to 50%. Additionally, we derive equations to compute upper bounds of the throughp
We consider the role of supersymmetric flat directions in reheating the Universe after inflation. One or more flat directions can develop large vevs during inflation, which can potentially affect reheating by slowing down scattering processes among inflaton decay products or by coming to dominate the energy density of the Universe. Both effects occur only if flat directions are sufficiently long-lived. The computation of their perturbative decay rate, and a simple estimate of their nonperturbative decay have led to the conclusion that this is indeed the case. In contrast, we show that flat directions can decay quickly through nonperturbative channels in realistic models. The mass matrix for MSSM excitations around flat directions has nondiagonal entries, which vary with the phase of the (complex) flat directions. The quasi-periodic motion of the flat directions results in a strong parametric resonance, leading to the rapid depletion of the flat direction within its first few rotations. This may preclude any significant role for the flat directions in reheating the Universe after inflation in models in which the inflaton decays perturbatively.
Recent advancements in low-Earth orbit (LEO) satellites represented by large constellations and advanced payloads provide great promises for enabling beyond 5G and 6G telecommunications and high-quality and ubiquitous Internet connectivity to everyone anywhere on Earth. LEO satellite networks are envisioned to bridge the urban-rural connectivity gap for the digital divide. However, the digital divide can hardly be closed by only providing connectivity to rural and remote areas. Various unprecedented challenges brought by the emerging satellite Internet still need to be resolved, such as inconsistent end-to-end performance guarantees and a lack of efficient management and operations in these areas, which are referred to as "performance gap" and "management gap", respectively. This position paper will briefly discuss these gaps, approaches to addressing the gaps, and some research directions based on our recent works.
Dimension reduction of multivariate data supervised by auxiliary information is considered. A series of basis for dimension reduction is obtained as minimizers of a novel criterion. The proposed method is akin to continuum regression, and the resulting basis is called continuum directions. With a presence of binary supervision data, these directions continuously bridge the principal component, mean difference and linear discriminant directions, thus ranging from unsupervised to fully supervised dimension reduction. High-dimensional asymptotic studies of continuum directions for binary supervision reveal several interesting facts. The conditions under which the sample continuum directions are inconsistent, but their classification performance is good, are specified. While the proposed method can be directly used for binary and multi-category classification, its generalizations to incorporate any form of auxiliary data are also presented. The proposed method enjoys fast computation, and the performance is better or on par with more computer-intensive alternatives.
Dimensional reduction of the M5 brane on a Lorentzian manifold along a lightlike direction results in a five-dimensional gauge theory, which can be reformulated covariantly in six dimensions, where one puts the Lie derivatives along the lightlike direction of all fields to zero as constraints. Without imposing these constraints, we have a nonsupersymmetric six-dimensional gauge theory that we may expect shall have a six dimensional gauge symmetry. However this gauge symmetry has an anomaly for certain Lorentzian six-manifolds. We show that this gauge anomaly can be canceled by adding a WZW theory in the 2d space that is spanned by two lightlike directions.
Flat directions are a generic feature of supersymmetric theories. They are of cosmological interest because they can lead to coherent production of scalars. In the early universe such flat directions could be dangerous due to the potentially large energy density and the late decay of the associated scalars when they have only $1/M_p$ couplings (Polonyi problem). On the other hand, flat directions among the standard model fields can carry baryon number and lead to a possible mechanism for baryogenesis (Affleck Dine baryogenesis). When considering the cosmological consequences of the flat directions, it is important to take into account the soft potential with curvature of order the Hubble constant due to supersymmetry breaking in the early universe. In this talk, we discuss flat directions, their potential cosmological implications focusing on Affleck-Dine baryogenesis, and how the standard picture of their evolution must be modified in the presence of the large supersymmetry breaking in the early universe.
The latent spaces of GAN models often have semantically meaningful directions. Moving in these directions corresponds to human-interpretable image transformations, such as zooming or recoloring, enabling a more controllable generation process. However, the discovery of such directions is currently performed in a supervised manner, requiring human labels, pretrained models, or some form of self-supervision. These requirements severely restrict a range of directions existing approaches can discover. In this paper, we introduce an unsupervised method to identify interpretable directions in the latent space of a pretrained GAN model. By a simple model-agnostic procedure, we find directions corresponding to sensible semantic manipulations without any form of (self-)supervision. Furthermore, we reveal several non-trivial findings, which would be difficult to obtain by existing methods, e.g., a direction corresponding to background removal. As an immediate practical benefit of our work, we show how to exploit this finding to achieve competitive performance for weakly-supervised saliency detection.
This paper investigates the use of multiple directions of stratification as a variance reduction technique for Monte Carlo simulations of path-dependent options driven by Gaussian vectors. The precision of the method depends on the choice of the directions of stratification and the allocation rule within each strata. Several choices have been proposed but, even if they provide variance reduction, their implementation is computationally intensive and not applicable to realistic payoffs, in particular not to Asian options with barrier. Moreover, all these previously published methods employ orthogonal directions for multiple stratification. In this work we investigate the use of algorithms producing convenient directions, generally non-orthogonal, combining a lower computational cost with a comparable variance reduction. In addition, we study the accuracy of optimal allocation in terms of variance reduction compared to the Latin Hypercube Sampling. We consider the directions obtained by the Linear Transformation and the Principal Component Analysis. We introduce a new procedure based on the Linear Approximation of the explained variance of the payoff using the law of total variance.
This paper is on face/head reenactment where the goal is to transfer the facial pose (3D head orientation and expression) of a target face to a source face. Previous methods focus on learning embedding networks for identity and pose disentanglement which proves to be a rather hard task, degrading the quality of the generated images. We take a different approach, bypassing the training of such networks, by using (fine-tuned) pre-trained GANs which have been shown capable of producing high-quality facial images. Because GANs are characterized by weak controllability, the core of our approach is a method to discover which directions in latent GAN space are responsible for controlling facial pose and expression variations. We present a simple pipeline to learn such directions with the aid of a 3D shape model which, by construction, already captures disentangled directions for facial pose, identity and expression. Moreover, we show that by embedding real images in the GAN latent space, our method can be successfully used for the reenactment of real-world faces. Our method features several favorable properties including using a single source image (one-shot) and enabling cross-person ree
The scalar potential of the Minimal Supersymmetric Standard Model (MSSM) is nearly flat along many directions in field space. We provide a catalog of the flat directions of the renormalizable and supersymmetry-preserving part of the scalar potential of the MSSM, using the correspondence between flat directions and gauge-invariant polynomials of chiral superfields. We then study how these flat directions are lifted by non-renormalizable terms in the superpotential, with special attention given to the subtleties associated with the family index structure. Several flat directions are lifted only by supersymmetry-breaking effects and by supersymmetric terms in the scalar potential of surprisingly high dimensionality.
Every Coxeter group admits a geometric representation as a group generated by reflections in a real vector space. In the projective representation space, limit directions are limits of injective sequences in the orbit of some base point. Limit roots are limit directions that can be obtained starting from simple roots. In this article, we study the limit directions arising from any point when the representation space is a Lorentz space. In particular, we characterize the light-like limit directions using eigenvectors of infinite-order elements. This provides a spectral perspective on limit roots, allowing for efficient computations. Moreover, we describe the space-like limit directions in terms of the projective Coxeter arrangement.
R-symmetries, which are needed for supersymmetry (SUSY) breaking in O'Raifeartaigh models, often lead to SUSY runaway directions trough a complexified R-transformation. Non-R symmetries also lead to runaway directions in a similar way. This work investigates the occurrence of runaway directions of both SUSY and SUSY breaking types. We clarify previous issues on fractional charges and genericness, and make a refined statement on conditions for runaway directions related to either R-symmetries or non-R symmetries. We present a generic and anomaly-free model to show the existence of runaway directions related to non-R symmetries. We also comment on the possibility to combine the non-R symmetry case to the R-symmetry case by an R-charge redefinition.
Collaborative perception (CP) leverages visual data from connected and autonomous vehicles (CAV) to enhance an ego vehicle's field of view (FoV). Despite recent progress, current CP methods expand the ego vehicle's 360-degree perceptual range almost equally, which faces two key challenges. Firstly, in areas with uneven traffic distribution, focusing on directions with little traffic offers limited benefits. Secondly, under limited communication budgets, allocating excessive bandwidth to less critical directions lowers the perception accuracy in more vital areas. To address these issues, we propose Direct-CP, a proactive and direction-aware CP system aiming at improving CP in specific directions. Our key idea is to enable an ego vehicle to proactively signal its interested directions and readjust its attention to enhance local directional CP performance. To achieve this, we first propose an RSU-aided direction masking mechanism that assists an ego vehicle in identifying vital directions. Additionally, we design a direction-aware selective attention module to wisely aggregate pertinent features based on ego vehicle's directional priorities, communication budget, and the positional da