搜索结果：Already

Each LoRA checkpoint compactly stores task-specific updates in low-rank weight matrices, offering an efficient way to adapt large language models to new tasks and domains. In principle, these weights already encode what the adapter does and how well it performs. In this paper, we ask whether this information can be read directly from the weights, without running the base model or accessing training data. A key obstacle is that a single LoRA update can be factorized in infinitely many ways. Without resolving this ambiguity, models trained on the factors may fit the particular factorization rather than the underlying update. To this end, we propose \methodfull, which maps each LoRA update to a provably canonical form via QR decomposition followed by SVD, so that all equivalent factorizations share the same representation. The resulting components are then tokenized and processed by a Transformer to produce a weight-space embedding. Across language and vision LoRA collections, W2T achieves strong results on attribute classification, performance prediction, and adapter retrieval, demonstrating that LoRA weights reliably indicate model behavior once factorization ambiguity is removed. C

We revisit millimeter-wave (mmWave) human pose estimation (HPE) from a signal preprocessing perspective. A single mmWave frame provides structured dimensions that map directly to human geometry and motion: range, angle, and Doppler, offering pose-aligned cues that are not explicitly present in RGB images. However, recent mmWave-based HPE systems require more parameters and compute resources yet yield lower estimation accuracy than vision baselines. We attribute this to preprocessing modules: most systems rely on data-driven modules to estimate phenomena that are already well-defined by mmWave sensing physics, whereas human pose could be captured more efficiently with explicit physical priors. To this end, we introduce processing modules that explicitly model mmWave's inter-dimensional correlations and human kinematics. Our design (1) couples range and angle to preserve spatial human structure, (2) leverages Doppler to retain human motion continuity, and (3) applies multi-scale fusion aligned with the human body. A lightweight MLP is involved as the regressor. In experiments, this framework reduces the number of parameters by 55.7-88.9% on the HPE task relative to existing mmWave ba

Block diffusion LLMs are an emerging paradigm for parallel language generation, but their KV caching makes memory access the dominant bottleneck in long-context inference. Sparse attention, which attends only to a small KV subset per query, can reduce this latency with minimal accuracy loss. In block diffusion, however, the B tokens of each block must share a single KV subset, and we show this per-block constraint degrades existing sparse KV estimators by up to 25% in recall. We address this challenge by exploiting a property that emerges from the block-diffusion training objective: it aligns the block-average query across denoising steps, so the All-[MASK] block at the first step already reveals the per-block KV subset for the entire trajectory. We exploit this in MAGE ([MASK]-Guided Sparse Attention), a training-free method that runs one exact attention pass at the first step and reuses its top-k index sets for all remaining steps within the block. Across three block-diffusion families on LongBench, MAGE matches Exact Attention at k=512 with near-lossless accuracy, achieves up to 6.82x end-to-end speedup at 128K context, and runs up to 3.35x and 2.28x faster than Quest and Sparse

The Categorical Jacobian of Zhang et al. (2024) reads protein contacts from a language model by perturbing every residue with every alternative amino acid, about $19L$ forward passes. We show the signal it reconstructs is already concentrated in a small subset of attention heads: averaging the top-$K$ contact-relevant heads -- selected on as few as 10 labeled proteins, with no fitted per-pair or per-head weights -- recovers contacts in a single forward pass and matches or beats the Categorical Jacobian for every bidirectional model where it is defined (bar the smallest, 8M). Our primary test is leakage-clean: on a CAMEO split where neither selection nor evaluation touches data the models have plausibly memorized, the head readout beats the Categorical Jacobian on ESM-2-650M by +9pp ($N = 29$, $p < 0.001$), with the within-model margin reproducing across architectures. Ablations localize the gain to labeled head selection, not to averaging: at a matched label budget the unweighted mean ties a supervised $L_1$ logistic regression on the same heads. Both methods fall 30-36pp from their in-distribution Zhang numbers to the leakage-clean split, which we read as an upper bound on how

Foundation models for language and vision are powered by internet-scale data, while structured domains such as tabular prediction are powered by synthetic data. This substitute shifts the challenge from collection to prior design. Such priors already exist for many structured tasks: TabPFN and its successors solve tabular classification with a transformer pretrained on a synthetic Bayesian prior. We make two points. \textbf{First}, reinforcement learning is the conspicuous gap: sampling a synthetic MDP is as feasible as sampling a synthetic tabular dataset, yet no in-context RL work treats prior design as a primary objective. \textbf{Second}, MDPs admit a fixed-size sufficient statistic, independent of the episodes observed and tabular in shape, which makes them directly amenable to the attention-based architectures used for tabular foundation models, with a policy head replacing the supervised target. Together these define the agenda for an RL foundation model. As a proof of concept, we train a Graph Attention Network entirely on synthetic MDPs and show that, with no task-specific tuning, it solves held-out tabular benchmarks in context, both online and offline: online, in far few

Large language models are increasingly evaluated by other models, raising a natural question: can a model predict how a judge will score its own output? We find that the ability is largely present before any targeted training: prompted few-shot, a base model already predicts an external judge's multi-attribute quality scores on open-ended responses well above chance across three benchmarks. We introduce Self-Evaluation Elicitation (SEE), a method that surfaces this latent ability through a short cycle comprising a calibration-coupled reinforcement learning phase that improves the answer and predicts the judge, followed by a masked distillation phase that sharpens the prediction while leaving the answer untouched. From 160 unique examples, roughly 31x fewer than a reinforcement learning baseline, SEE improves held-out calibration across three benchmarks while preserving answer quality. The elicited self-evaluation is sharply localized within the model's own token distribution and stable across judges it was never trained against, indicating a transferable notion of quality rather than a single judge's preference. These results reframe judge-aligned self-evaluation as a problem of el

Tool-augmented LLM agents tend to call tools indiscriminately, even when the model can answer directly. Each unnecessary call wastes API fees and latency, yet no existing benchmark systematically studies when a tool call is actually needed. We propose When2Tool, a benchmark of 18 environments (15 single-hop, 3 multi-hop) spanning three categories of tool necessity -- computational scale, knowledge boundaries, and execution reliability -- each with controlled difficulty levels that create a clear decision boundary between tool-necessary and tool-unnecessary tasks. We evaluate two families of training-free baselines: Prompt-only (varying the prompt to discourage unnecessary calls) and Reason-then-Act (requiring the model to reason about tool necessity before acting). Both provide limited control: Prompt-only suppresses necessary calls alongside unnecessary ones, and Reason-then-Act still incurs a disproportionate accuracy cost on hard tasks. To understand why these baselines fail, we probe the models' hidden states and find that tool necessity is linearly decodable from the pre-generation representation with AUROC 0.89--0.96 across six models, substantially exceeding the model's own

Vision-Language-Action (VLA) models have demonstrated impressive end-to-end performance across a variety of robotic manipulation tasks. However, these policies offer no guarantees against collisions with task-irrelevant objects in the scene. Existing safety filters sidestep this problem by querying a vision-language model (VLM) to identify obstacles and their locations. This, however, is too slow to run in the control loop and can only be invoked at episode initialization, leaving the filter unable to track moving obstacles. We discover that a small number of attention heads within a VLA model reliably localize the object the policy intends to approach. These heads can be exploited within a training-free safety framework that obtains the active target from the attention heads at every step, treats the remainder of the scene as obstacles, and feeds these into a Control Barrier Function (CBF) filter. Together with a lightweight real-time object tracker, this allows for collision avoidance for non-static obstacles. We evaluate our framework on SafeLIBERO, which we extend with moving obstacles. On the original static benchmark, our method performs comparably to an oracle that uses priv

Vision--language--action (VLA) policies are trained to imitate actions; their loss never asks them to estimate reward, progress, or future success. Their frozen representations nevertheless carry such information, and it can be read out and used to guide action choice without retraining the policy. From mixed successful and failed manipulation trajectories on LIBERO-Goal, we recover Monte-Carlo outcome targets using lightweight linear probes on frozen features. The targets are consistently predictable from OpenVLA, Pi0.5, DINOv2, and CLIP features, and substantially less so from baselines built on progress, time-to-go, task identity, or proprioception. To rule out task and temporal shortcuts, we evaluate the probes under same-task, same-timestep matched comparisons: Pi0.5 probes still reach roughly 92% pairwise ordering accuracy, while label-shuffled controls stay at chance. Used as a test-time selector over sampled Pi0.5 action prefixes, the same probe turns this offline finding into behavior: on push-plate, success rises from 26.7% under greedy decoding to 44.3%, with a second positive case on wine-rack. The gains are not universal and require additional inference compute, but th

One of the most debated consequences of the Milky Way's last major merger is the so-called $Splash$: stars with disc-like chemistry but halo-like kinematics, often interpreted as evidence for the violent heating of an early protodisc. Using the same high-resolution NIHAO-UHD cosmological simulation analysed in Paper I, we test whether, and if so how, a $Splash$-like population arises in the Milky Way analogue. By tracing stellar birth positions, ages, and present-day orbits, we find that protodisc stars were already born on dynamically hot orbits, with only limited additional dynamical $splashing$ of these particular in-situ stars despite a 1:5 stellar mass merger. A subset of stars, particularly those that end up in the Solar neighbourhood, shows evidence for merger-driven angular-momentum redistribution, but the overall kinematic distribution of stars with $Splash$-like chemistry remains largely unchanged. The observed $Splash$ may therefore primarily reflect the already turbulent early disc, subsequently intermixed with accreted stars and those formed from merger-driven gas inflows, rather than a distinct merger-heated population. When selecting stars with similar chemistry and

The diameter of a polytope is a fundamental geometric parameter that plays a crucial role in understanding the efficiency of the simplex method. Despite its central nature, the computational complexity of computing the diameter of a given polytope is poorly understood. Already in 1994, Frieze and Teng [Comp. Compl.] recognized the possibility that this task could potentially be harder than NP-hard, and asked whether the corresponding decision problem is complete for the second stage of the polynomial hierarchy, i.e. $Π^p_2$-complete. In the following years, partial results could be obtained. In a cornerstone result, Frieze and Teng themselves proved weak NP-hardness for a family of custom defined polytopes. Sanità [FOCS18] in a break-through result proved that already for the much simpler fractional matching polytope the problem is strongly NP-hard. Very recently, Steiner and Nöbel [SODA25] generalized this result to the even simpler bipartite perfect matching polytope and the circuit diameter. In this paper, we finally show that computing the diameter of the bipartite perfect matching polytope is $Π^p_2$-hard. Since the corresponding decision problem is also trivially contained in

We present deep, medium-resolution $λ=1-5\,μ$m JWST/NIRSpec spectroscopy for 14 quiescent galaxies at $3<z<5$ with $\log_{10}(M_*/\mathrm{M_\odot}){\,>\,}10$, obtained as part of the EXCELS survey. We perform a complete re-reduction of these data, including a custom optimal-extraction approach to combat the spectral "wiggles" that result from undersampling of the NIRSpec spatial PSF. We constrain the star-formation histories and stellar metallicities of these objects via full-spectral fitting, finding a clear stellar age vs stellar mass correlation, in which more massive galaxies assembled their stellar mass at earlier times. This confirms spectroscopically that the archaeological "downsizing" trend was already in place by $z\simeq4$. The slope of our measured relation ($\simeq2$ Gyr per dex in stellar mass) is consistent with literature results at $0 < z < 3$. We do not observe objects with $\log_{10}(M_*/\mathrm{M_\odot})\lesssim10.5$ and ages of more than a few hundred Myr at this epoch, suggesting that recently reported examples of higher-redshift quiescent galaxies at these masses are likely to soon rejuvenate. We measure relatively high stellar metallicities fo

Image composition aims to seamlessly insert a user-specified object into a new scene, but existing models struggle with complex lighting (e.g., accurate shadows, water reflections) and diverse, high-resolution inputs. Modern text-to-image diffusion models (e.g., SD3.5, FLUX) already encode essential physical and resolution priors, yet lack a framework to unleash them without resorting to latent inversion, which often locks object poses into contextually inappropriate orientations, or brittle attention surgery. We propose SHINE, a training-free framework for Seamless, High-fidelity Insertion with Neutralized Errors. SHINE introduces manifold-steered anchor loss, leveraging pretrained customization adapters (e.g., IP-Adapter) to guide latents for faithful subject representation while preserving background integrity. Degradation-suppression guidance and adaptive background blending are proposed to further eliminate low-quality outputs and visible seams. To address the lack of rigorous benchmarks, we introduce ComplexCompo, featuring diverse resolutions and challenging conditions such as low lighting, strong illumination, intricate shadows, and reflective surfaces. Experiments on Compl

In this work, we explore the nature of $z>1$ galactic bars. Once thought to be highly transient, our results demonstrate otherwise. Our sample consists of nine massive ($>10^{10.5}\,\rm M_{\odot}$) star-forming barred-spiral galaxies at $z_{\rm spec} \sim 1.5$. Using rest-frame near-IR (F444W) JWST/NIRCam imaging, we apply ellipse fitting along with 1D and 2D morphological modeling to directly measure bar properties. We find that five galaxies host flat surface brightness profiles (bar Sérsic index $<0.4$), indicative of highly evolved, "mature" bars. By contrast, only two galaxies show exponential profiles, characteristic of young bars, and these are also shorter in absolute length than the flat bars. We therefore conclude that a large fraction of bars at this epoch have already matured, thereby indicating the presence of well-settled disks required to facilitate bar formation and sustained evolution well before $z\sim1.5$. To assess the gravitational impact of the bars, we calculate the maximum transverse-to-radial force ratio ($Q_{b}$). We find that $Q_{b}$ values are comparable to, or weaker than, those of bars in the local Universe, Seven of the nine bars show only a

Mathematical capabilities were previously believed to emerge in common language models only at a very large scale or require extensive math-related pre-training. This paper shows that the LLaMA-2 7B model with common pre-training already exhibits strong mathematical abilities, as evidenced by its impressive accuracy of 97.7% and 72.0% on the GSM8K and MATH benchmarks, respectively, when selecting the best response from 256 random generations. The primary issue with the current base model is the difficulty in consistently eliciting its inherent mathematical capabilities. Notably, the accuracy for the first answer drops to 49.5% and 7.9% on the GSM8K and MATH benchmarks, respectively. We find that simply scaling up the SFT data can significantly enhance the reliability of generating correct answers. However, the potential for extensive scaling is constrained by the scarcity of publicly available math questions. To overcome this limitation, we employ synthetic data, which proves to be nearly as effective as real data and shows no clear saturation when scaled up to approximately one million samples. This straightforward approach achieves an accuracy of 82.6% on GSM8K and 40.6% on MATH

In this work, I report that large fraction of stars detected by Ádám et al. (2023, A&A, 674, A170, arXiv:2304.08394) and noted in that work as new discoveries are in fact known systems. This is especially true for the dense bulge fields with large blending of nearby sources. Among the published 245 stars determined to be doubly eclipsing (i.e. containing two eclipsing signals), I identified 53 blends. In other words, about a quarter of the systems noted by Ádám et al. (2023, A&A, 674, A170) are not actually doubly eclipsing; rather, these are contaminations of known nearby sources that have already been detected by OGLE. Such a high proportion of reported false positives should not be readily ignored and ought to be addressed in future studies.

Generative AI has the potential to transform how public services are delivered by enhancing productivity and reducing time spent on bureaucracy. Furthermore, unlike other types of artificial intelligence, it is a technology that has quickly become widely available for bottom-up adoption: essentially anyone can decide to make use of it in their day to day work. But to what extent is generative AI already in use in the public sector? Our survey of 938 public service professionals within the UK (covering education, health, social work and emergency services) seeks to answer this question. We find that use of generative AI systems is already widespread: 45% of respondents were aware of generative AI usage within their area of work, while 22% actively use a generative AI system. Public sector professionals were positive about both current use of the technology and its potential to enhance their efficiency and reduce bureaucratic workload in the future. For example, those working in the NHS thought that time spent on bureaucracy could drop from 50% to 30% if generative AI was properly exploited, an equivalent of one day per week (an enormous potential impact). Our survey also found a hig

Today's sign language recognition models require large training corpora of laboratory-like videos, whose collection involves an extensive workforce and financial resources. As a result, only a handful of such systems are publicly available, not to mention their limited localization capabilities for less-populated sign languages. Utilizing online text-to-video dictionaries, which inherently hold annotated data of various attributes and sign languages, and training models in a few-shot fashion hence poses a promising path for the democratization of this technology. In this work, we collect and open-source the UWB-SL-Wild few-shot dataset, the first of its kind training resource consisting of dictionary-scraped videos. This dataset represents the actual distribution and characteristics of available online sign language data. We select glosses that directly overlap with the already existing datasets WLASL100 and ASLLVD and share their class mappings to allow for transfer learning experiments. Apart from providing baseline results on a pose-based architecture, we introduce a novel approach to training sign language recognition models in a few-shot scenario, resulting in state-of-the-art

Intracluster light (ICL) is diffuse light from stars that are gravitationally bound not to individual member galaxies, but to the halo of galaxy clusters. Leading theories predict that the ICL fraction, defined by the ratio of the ICL to the total light, rapidly decreases with increasing redshift, to the level of a few per cent at z > 1. However, observational studies have remained inconclusive about the fraction beyond redshift unity because, to date, only two clusters in this redshift regime have been investigated. One shows a much lower fraction than the mean value at low redshift, whereas the other possesses a fraction similar to the low-redshift value. Here we report an ICL study of ten galaxy clusters at 1 \lesssim z \lesssim 2 based on deep infrared imaging data. Contrary to the leading theories, our study finds that ICL is already abundant at z \lesssim 1, with a mean ICL fraction of approximately 17\%. Moreover, no significant correlation between cluster mass and ICL fraction or between ICL color and cluster-centric radius is observed. Our findings suggest that gradual stripping can no longer be the dominant mechanism of ICL formation. Instead, our study supports the sc