If autoresearch is itself a form of research, then autoresearch can be applied to research itself. We present Bilevel Autoresearch, a bilevel framework in which an outer autoresearch loop improves an inner autoresearch loop by reading its code and traces, identifying bottlenecks, and generating injectable Python search mechanisms at runtime. The inner loop optimizes task performance; the outer loop optimizes how the inner loop searches. Both loops use the same LLM, so improvements come from the bilevel architecture rather than a stronger meta-level model, although the outer loop consumes additional inference and wall-clock budget. On Karpathy's GPT pretraining benchmark, the meta-autoresearch outer loop achieves a 5x improvement over the standard inner loop alone (-0.045 vs. -0.009 val_bpb), while parameter-level adjustment without mechanism change yields no reliable gain. The outer loop instantiates mechanisms from adjacent search domains, including combinatorial optimization, multi-armed bandits, and design of experiments, without human specification of the final mechanism design. Trace analysis suggests that these mechanisms break deterministic search patterns and force explorat
Recent segmentation methods leveraging Multi-modal Large Language Models (MLLMs) have shown reliable object-level segmentation and enhanced spatial perception. However, almost all previous methods predominantly rely on specialist mask decoders to interpret masks from generated segmentation-related embeddings and visual features, or incorporate multiple additional tokens to assist. This paper aims to investigate whether and how we can unlock segmentation from MLLM itSELF with 1 segmentation Embedding (SELF1E) while achieving competitive results, which eliminates the need for external decoders. To this end, our approach targets the fundamental limitation of resolution reduction in pixel-shuffled image features from MLLMs. First, we retain image features at their original uncompressed resolution, and refill them with residual features extracted from MLLM-processed compressed features, thereby improving feature precision. Subsequently, we integrate pixel-unshuffle operations on image features with and without LLM processing, respectively, to unleash the details of compressed features and amplify the residual features under uncompressed resolution, which further enhances the resolution
Differential Privacy (DP) provides a rigorous framework for privacy, ensuring the outputs of data-driven algorithms remain statistically indistinguishable across datasets that differ in a single entry. While guaranteeing DP generally requires explicitly injecting noise either to the algorithm itself or to its outputs, the intrinsic randomness of existing algorithms presents an opportunity to achieve DP ``for free''. In this work, we explore the role of regularization in achieving DP across three different decision-making problems: multi-armed bandits, linear contextual bandits, and reinforcement learning from human feedback (RLHF), in offline data settings. We show that adding KL-regularization to the learning objective (a common approach in optimization algorithms) makes the action sampled from the resulting stochastic policy itself differentially private. This offers a new route to privacy guarantees without additional noise injection, while also preserving the inherent advantage of regularization in enhancing performance.
Existing detoxification methods for large language models mainly focus on post-training stage or inference time, while few tackle the source of toxicity, namely, the dataset itself. Such training-based or controllable decoding approaches cannot completely suppress the model's inherent toxicity, whereas detoxifying the pretraining dataset can fundamentally reduce the toxicity that the model learns during training. Hence, we attempt to detoxify directly on raw corpora with SoCD (Soft Contrastive Decoding), which guides an LLM to localize and rewrite toxic spans in raw data while preserving semantics, in our proposed HSPD (Hierarchical Semantic-Preserving Detoxification) pipeline, yielding a detoxified corpus that can drop-in replace the original for fine-tuning or other training. On GPT2-XL, HSPD attains state-of-the-art detoxification, reducing Toxicity Probability (TP) from 0.42 to 0.18 and Expected Maximum Toxicity (EMT) from 0.43 to 0.20. We further validate consistent best-in-class results on LLaMA2-7B, OPT-6.7B, and Falcon-7B. These findings show that semantics-preserving, corpus-level rewriting with HSPD effectively suppresses downstream toxicity while retaining data utility a
Understanding how societies react to epidemic threats requires more than tracking infection curves. Public perception, collective memory and behavioural adaptation interact through feedback loops that can amplify or suppress the spread of fear, vigilance and precaution. In this work we reinterpret the classical Lorenz system in a socioepidemic context, governed by nonlinear interactions between perceived infection, social transmission behaviour and memory of past risk. We provide a qualitative analysis of the model and show that small fluctuations in perception or behaviour can trigger transitions between stable, oscillatory and chaotic collective responses. These results suggest that social reactions to epidemics may evolve according to intrinsic dynamical rules, generating complex patterns of vigilance, fatigue and renewed concern that mirror the irregular rhythms observed in real outbreaks. Our findings highlight the importance of incorporating behavioural feedbacks into epidemic modeling and reveal how chaotic dynamics may arise not only from pathogens but from society itself.
Can language models improve their reasoning performance without external rewards, using only their own sampled responses for training? We show that they can. We propose Self-evolving Post-Training (SePT), a simple post-training method that alternates between self-generation and training on self-generated responses. It repeatedly samples questions, uses the model itself to generate responses under a specified sampling temperature, and then trains the model on the self-generated data. In this self-training loop, we use an online data refresh mechanism, where each new batch is generated by the most recently updated model. Across six math reasoning benchmarks, SePT improves a strong no-training baseline, defined as the untuned base model evaluated at its best swept decoding temperature, on several tested models. Additional ablations demonstrate the importance of online data refresh and temperature dynamics. Overall, our results identify a practical regime where reasoning can be improved using self-generated supervision alone. Our code is available at https://github.com/ElementQi/SePT.
Recent generative models face significant risks of producing harmful content, which has underscored the importance of machine unlearning (MU) as a critical technique for eliminating the influence of undesired data. However, existing MU methods typically assign the same weight to all data to be forgotten, which makes it difficult to effectively forget certain data that is harder to unlearn than others. In this paper, we empirically demonstrate that the loss of data itself can implicitly reflect its varying difficulty. Building on this insight, we introduce Loss-based Reweighting Unlearning (LoReUn), a simple yet effective plug-and-play strategy that dynamically reweights data during the unlearning process with minimal additional computational overhead. Our approach significantly reduces the gap between existing MU methods and exact unlearning in both image classification and generation tasks, effectively enhancing the prevention of harmful content generation in text-to-image diffusion models.
Generative Artificial Intelligence (AI) technologies and large models are producing realistic outputs across various domains, such as images, text, speech, and music. Creating these advanced generative models requires significant resources, particularly large and high-quality datasets. To minimise training expenses, many algorithm developers use data created by the models themselves as a cost-effective training solution. However, not all synthetic data effectively improve model performance, necessitating a strategic balance in the use of real versus synthetic data to optimise outcomes. Currently, the previously well-controlled integration of real and synthetic data is becoming uncontrollable. The widespread and unregulated dissemination of synthetic data online leads to the contamination of datasets traditionally compiled through web scraping, now mixed with unlabeled synthetic data. This trend, known as the AI autophagy phenomenon, suggests a future where generative AI systems may increasingly consume their own outputs without discernment, raising concerns about model performance, reliability, and ethical implications. What will happen if generative AI continuously consumes itself
The primary axes of interest in image-generating diffusion models are image quality, the amount of variation in the results, and how well the results align with a given condition, e.g., a class label or a text prompt. The popular classifier-free guidance approach uses an unconditional model to guide a conditional model, leading to simultaneously better prompt alignment and higher-quality images at the cost of reduced variation. These effects seem inherently entangled, and thus hard to control. We make the surprising observation that it is possible to obtain disentangled control over image quality without compromising the amount of variation by guiding generation using a smaller, less-trained version of the model itself rather than an unconditional model. This leads to significant improvements in ImageNet generation, setting record FIDs of 1.01 for 64x64 and 1.25 for 512x512, using publicly available networks. Furthermore, the method is also applicable to unconditional diffusion models, drastically improving their quality.
The wide-ranging applications of large language models (LLMs), especially in safety-critical domains, necessitate the proper evaluation of the LLM's adversarial robustness. This paper proposes an efficient tool to audit the LLM's adversarial robustness via a prompt-based adversarial attack (PromptAttack). PromptAttack converts adversarial textual attacks into an attack prompt that can cause the victim LLM to output the adversarial sample to fool itself. The attack prompt is composed of three important components: (1) original input (OI) including the original sample and its ground-truth label, (2) attack objective (AO) illustrating a task description of generating a new sample that can fool itself without changing the semantic meaning, and (3) attack guidance (AG) containing the perturbation instructions to guide the LLM on how to complete the task by perturbing the original sample at character, word, and sentence levels, respectively. Besides, we use a fidelity filter to ensure that PromptAttack maintains the original semantic meanings of the adversarial examples. Further, we enhance the attack power of PromptAttack by ensembling adversarial examples at different perturbation leve
Open-vocabulary dense prediction tasks including object detection and image segmentation have been advanced by the success of Contrastive Language-Image Pre-training (CLIP). CLIP models, particularly those incorporating vision transformers (ViTs), have exhibited remarkable generalization ability in zero-shot image classification. However, when transferring the vision-language alignment of CLIP from global image representation to local region representation for the open-vocabulary dense prediction tasks, CLIP ViTs suffer from the domain shift from full images to local image regions. In this paper, we embark on an in-depth analysis of the region-language alignment in CLIP models, which is essential for downstream open-vocabulary dense prediction tasks. Subsequently, we propose an approach named CLIPSelf, which adapts the image-level recognition ability of CLIP ViT to local image regions without needing any region-text pairs. CLIPSelf empowers ViTs to distill itself by aligning a region representation extracted from its dense feature map with the image-level representation of the corresponding image crop. With the enhanced CLIP ViTs, we achieve new state-of-the-art performance on open
In neutron transmutation doped germanium, the thermal neutron fluence of reactor irradiation is as high as 10$^{18}$~n$\cdot$cm$^{-2}$. For radiological safety reasons, general Co or Au neutron flux monitors cannot be easily used. We have experimentally demonstrated the feasibility of measuring the X-rays emitted by the NTD-Ge itself to determine the absolute thermal neutron fluence for the first time. A Micro-Megas Detector (MMD) and a Silicon Drift Detector (SDD) are set up to detect the tagging KX-rays with 9.2 keV and 10.3 keV cascading from the decays of $^{71}$Ge. Combined the detection efficiencies calculated by GEANT4, neutron fluence results given with proper accuracy by MMD and SDD are in a good agreement with each other.
This paper presents EvolveMT for efficiently combining multiple machine translation (MT) engines. The proposed system selects the output from a single engine for each segment by utilizing online learning techniques to predict the most suitable system for every translation request. A neural quality estimation metric supervises the method without requiring reference translations. The online learning capability of this system allows for dynamic adaptation to alterations in the domain or machine translation engines, thereby obviating the necessity for additional training. EvolveMT selects a subset of translation engines to be called based on the source sentence features. The degree of exploration is configurable according to the desired quality-cost trade-off. Results from custom datasets demonstrate that EvolveMT achieves similar translation accuracy at a lower cost than selecting the best translation of each segment from all translations using an MT quality estimator. To our knowledge, EvolveMT is the first meta MT system that adapts itself after deployment to incoming translation requests from the production environment without needing costly retraining on human feedback.
We analyze the puzzle video game This Game Is Not Going To Load Itself, where the player routes data packets of three different colors from given sources to given sinks of the correct color. Given the sources, sinks, and some previously placed arrow tiles, we prove that the game is in Sigma_2^P; in NP for sources of equal period; NP-complete for three colors and six equal-period sources with player input; and even without player input, simulating the game is both NP- and coNP-hard for two colors and many sources with different periods. On the other hand, we characterize which locations for three data sinks admit a perfect placement of arrow tiles that guarantee correct routing no matter the placement of the data sources, effectively solving most instances of the game as it is normally played.
We present a construction of left braces of right nilpotency class at most two based on suitable actions of an abelian group on itself with an invariance condition. This construction allows us to recover the construction of a free right nilpotent one-generated left brace of class two.
Small fluid leaks are common and frequently troublesome. We often consider how to stop a leak, but here we ask a different question: how might a leak stop itself? We experimentally study leaking flow transitions from continuous drainage to spontaneous arrest. High-speed imaging reveals that fluid breakup events generate droplets whose Laplace pressures oppose the leak. Early droplets grow unstably, allowing the leak to continue, but ultimately a final capping droplet equilibrates to a stable spherical cap via lightly damped harmonic oscillations. A total energetic theory incorporating both the potential and kinetic energy of attempted capping droplets shows that inertia plays a key role in the leak-stop mechanism. Further experiments examining the stability of rivulet flow in such a system demonstrate that a transition from continuous to discrete flow is an essential prerequisite in determining when a leak can stop itself.
The impact of possible a-priori "imprinting" effects of general relativity itself on recent attempts to measure the Lense-Thirring precessions with the LAGEOS satellites orbiting the Earth and the terrestrial geopotential models by the dedicated mission GRACE is investigated. It is analytically shown that general relativity, not explicitly solved for in the GRACE-based models, may "imprint" their even zonal harmonic coefficients J_L at a non-negligible level, given the present-day accuracy in recovering them. This translates into a bias of the LAGEOS-based relativistic tests as large as the Lense-Thirring effect itself. Further analyses should include general relativity itself in the GRACE data processing by explicitly solving for it.
We propose Self-Supervised Implicit Attention (SSIA), a new approach that adaptively guides deep neural network models to gain attention by exploiting the properties of the models themselves. SSIA is a novel attention mechanism that does not require any extra parameters, computation, or memory access costs during inference, which is in contrast to existing attention mechanism. In short, by considering attention weights as higher-level semantic information, we reconsidered the implementation of existing attention mechanisms and further propose generating supervisory signals from higher network layers to guide lower network layers for parameter updates. We achieved this by building a self-supervised learning task using the hierarchical features of the network itself, which only works at the training stage. To verify the effectiveness of SSIA, we performed a particular implementation (called an SSIA block) in convolutional neural network models and validated it on several image classification datasets. The experimental results show that an SSIA block can significantly improve the model performance, even outperforms many popular attention methods that require additional parameters and
We give a subfactor construction for a $II_{1}$ factor M which is not anti-isomorphic to itself. The $II_{1}$ factor we consider is essentially the same as the example previously given by Connes. However, our construction uses the recently developed theory of free group factors. We show that there exists an inclusion of $II_{1}$ factors $A\subset B$ which by iteration of the Jones basic construction produces $M$ as the enveloping algebra. Here A is a free group factor and B is isomorphic to the crossed product of A by an action of a finite group. By using a Connes' argument involving the invariant $χ(M)$, we verify that $M$ is not anti--isomorphic to itself. Publication of this manuscript is funded in part by the National Science Foundation. This material is based upon work supported by the National Science Foundation under Grant No. DMS--9810361.
Let $X$ be an algebraic K3 surface, $v=(r,H,s)$ a primitive isotropic Mukai vector on $X$ and $M_X(v)$ the moduli of sheaves over $X$ with $v$. Let $N(X)$ be Picard lattice of $X$. In math.AG/0309348 and math.AG/0606289, all divisors in moduli of $(X,H)$ (i. e. pairs $H\in N(X)$ with $\rk N(X)=2$) implying $M_X(v)\cong X$ were described. They give some Mukai's correspondences of $X$ with itself. Applying these results, we show that there exists $v$ and a codimension 2 submoduli in moduli of $(X,H)$ (i. e. a pair $H\in N(X)$ with $\rk N(X)=3$) implying $M_X(v)\cong X$, but this submoduli cannot be extended to a divisor in moduli with the same property. There are plenty of similar examples. We discuss the general problem of description of all similar submoduli and defined by them Mukai's correspondences of $X$ with itself and their compositions, trying to outline a possible general theory.