搜索结果：Unreasonable

In recommender systems, collecting, storing, and processing large-scale interaction data is increasingly costly in terms of time, energy, and computation, yet it remains unclear when additional data stops providing meaningful gains. This paper investigates how offline recommendation performance evolves as the size of the training dataset increases and whether a saturation point can be observed. We implemented a reproducible Python evaluation workflow with two established toolkits, LensKit and RecBole, included 11 large public datasets with at least 7 million interactions, and evaluated 10 tool-algorithm combinations. Using absolute stratified user sampling, we trained models on nine sample sizes from 100,000 to 100,000,000 interactions and measured NDCG@10. Overall, raw NDCG usually increased with sample size, with no observable saturation point. To make result groups comparable, we applied min-max normalization within each group, revealing a clear positive trend in which around 75% of the points at the largest completed sample size also achieved the group's best observed performance. A late-stage slope analysis over the final 10-30% of each group further supported this upward tren

We demonstrate that the system of fine-tuning constraints for life is overdetermined: the number of inequalities is too large relative to the number of free parameters in the chemical sector and likely not only there. This entails that life-permitting conditions are secured at the level of physics not only by the fine-tuned fundamental constants, but also by a very special form of the laws. In 1960, Eugene Wigner published his essay "The Unreasonable Effectiveness of Mathematics in the Natural Sciences," summarizing his reflections on the miraculous comprehensibility of the mathematical structure of physical laws. The present paper points to another, no less remarkable, aspect of this structure, which may be called as our title suggests.

Last-layer retraining (LLR) methods -- wherein the last layer of a neural network is reinitialized and retrained on a held-out set following ERM training -- have garnered interest as an efficient approach to rectify dependence on spurious correlations and improve performance on minority groups. Surprisingly, LLR has been found to improve worst-group accuracy even when the held-out set is an imbalanced subset of the training set. We initially hypothesize that this ``unreasonable effectiveness'' of LLR is explained by its ability to mitigate neural collapse through the held-out set, resulting in the implicit bias of gradient descent benefiting robustness. Our empirical investigation does not support this hypothesis. Instead, we present strong evidence for an alternative hypothesis: that the success of LLR is primarily due to better group balance in the held-out set. We conclude by showing how the recent algorithms CB-LLR and AFR perform implicit group-balancing to elicit a robustness improvement.

This paper provides an overview of how the determination of absence of unreasonable risk can be operationalized. It complements previous theoretical work published by existing developers of Automated Driving Systems (ADS) on the overall engineering practices and methodologies for readiness determination. Readiness determination is, at its core, a risk assessment process. It is aimed at evaluating the residual risk associated with a new deployment. The paper proposes methodological criteria to ground the readiness review process for an ADS release. While informed by Waymo's experience in this domain, the criteria presented are agnostic of any specific ADS technological solution and/or architectural choice, to support broad implementation by others in the industry. The paper continues with a discussion on governance and decision-making toward approval of a new release candidate for the ADS. The implementation of the presented criteria requires the existence of appropriate safety management practices in addition to many other cultural, procedural, and operational considerations. As such, the paper is concluded by a statement of limitations for those wishing to replicate part or all of

This paper examines the use of Monte Carlo simulations to understand statistical concepts in A/B testing and Randomized Controlled Trials (RCTs). We discuss the applicability of simulations in understanding false positive rates and estimate statistical power, implementing variance reduction techniques and examining the effects of early stopping. By comparing frequentist and Bayesian approaches, we illustrate how simulations can clarify the relationship between p-values and posterior probabilities, and the validity of such approximations. The study also references how Monte Carlo simulations can be used to understand network effects in RCTs on social networks. Our findings show that Monte Carlo simulations are an effective tool for experimenters to deepen their understanding and ensure their results are statistically valid and practically meaningful.

The Tunneling Potential Formalism was introduced to calculate the tunneling actions that control vacuum decay as an alternative to the standard Euclidean Formalism. The new approach sets the problem as a simple variational problem in field space with decay described by a tunneling potential function $V_t$ that extremizes a simple action functional $S[V_t]$ and has a number of appealing properties that have been presented elsewhere. In this note I discuss several instances in which this $V_t$ approach seems to give more than one would have expected a priori, as the following: the $V_t$ describing the decay is a minimum of the new action $S[V_t]$ rather than a saddle point; the decay of AdS, dS or Minkowski vacua are governed by a unique universal $S[V_t]$ which also gives the Hawking-Moss instanton in the appropriate limit; physically relevant solutions beyond the Coleman-De Luccia (CdL) bounce, like pseudo-bounces or bubbles of nothing (BoNs), show up in a straightforward way as generalizations of the CdL bounce, with the correct boundary conditions; in cases for which the Euclidean action calculation requires the inclusion of particular boundary terms (like for BoNs or for the dec

How is knowledge stored in an LLM's weights? We study this via layer pruning: if removing a certain layer does not affect model performance in common question-answering benchmarks, then the weights in that layer are not necessary for storing the knowledge needed to answer those questions. To find these unnecessary parameters, we identify the optimal block of layers to prune by considering similarity across layers; then, to "heal" the damage, we perform a small amount of finetuning. Surprisingly, with this method we find minimal degradation of performance until after a large fraction (up to half) of the layers are removed for some common open-weight models. From a scientific perspective, the robustness of these LLMs to the deletion of layers implies either that current pretraining methods are not properly leveraging the parameters in the deeper layers of the network or that the shallow layers play a critical role in storing knowledge. For our study, we use parameter-efficient finetuning (PEFT) methods, specifically quantization and Low Rank Adapters (QLoRA), such that each of our experiments can be performed on a single 40GB A100 GPU.

In kinetic theory, the classic $n Σv$ approach calculates the rate of particle interactions from local quantities: the number density of particles $n$, the cross-section $Σ$, and the average relative speed $v$. In stellar dynamics, this formula is often applied to problems in collisional (i.e. dense) environments such as globular and nuclear star clusters, where blue stragglers, tidal capture binaries, binary ionizations, and micro-tidal disruptions arise from rare close encounters. The local $n Σv$ approach implicitly assumes the ergodic hypothesis, which is not well motivated for the densest star systems in the Universe. In the centers of globular and nuclear star clusters, orbits close into 1D ellipses because of the degeneracy of the potential (either Keplerian or harmonic). We find that the interaction rate in perfectly Keplerian or harmonic potentials is determined by a global quantity -- the number of orbital intersections -- and that this rate can be far lower or higher than the ergodic $n Σv$ estimate. However, we find that in most astrophysical systems, deviations from a perfectly Keplerian or harmonic potential (due to e.g. granularity or extended mass) trigger sufficien

Recently, Neural Radiance Field (NeRF) has shown great success in rendering novel-view images of a given scene by learning an implicit representation with only posed RGB images. NeRF and relevant neural field methods (e.g., neural surface representation) typically optimize a point-wise loss and make point-wise predictions, where one data point corresponds to one pixel. Unfortunately, this line of research failed to use the collective supervision of distant pixels, although it is known that pixels in an image or scene can provide rich structural information. To the best of our knowledge, we are the first to design a nonlocal multiplex training paradigm for NeRF and relevant neural field methods via a novel Stochastic Structural SIMilarity (S3IM) loss that processes multiple data points as a whole set instead of process multiple inputs independently. Our extensive experiments demonstrate the unreasonable effectiveness of S3IM in improving NeRF and neural surface representation for nearly free. The improvements of quality metrics can be particularly significant for those relatively difficult tasks: e.g., the test MSE loss unexpectedly drops by more than 90% for TensoRF and DVGO over e

Recent work in database query optimization has used complex machine learning strategies, such as customized reinforcement learning schemes. Surprisingly, we show that LLM embeddings of query text contain useful semantic information for query optimization. Specifically, we show that a simple binary classifier deciding between alternative query plans, trained only on a small number of labeled embedded query vectors, can outperform existing heuristic systems. Although we only present some preliminary results, an LLM-powered query optimizer could provide significant benefits, both in terms of performance and simplicity.

Large Language Models (LLMs) have demonstrated remarkable problem-solving and basic mathematics abilities. However, their efficacy is highly contingent on the formulation of the prompt. This study endeavors to quantify the influence of incorporating "positive thinking" into the system message of the prompt, then compare that to systematic prompt optimization. We assess the performance of 60 combinations of system message snippets, tested with and without Chain of Thought prompting, across three models with parameters ranging from 7 to 70 billion on the GSM8K dataset. Our findings reveal that results do not universally generalize across models. In most instances, the inclusion of "positive thinking" prompts positively affected model performance. Notably, however, Llama2-70B exhibited an exception when not utilizing Chain of Thought, as the optimal system message was found to be none at all. Given the combinatorial complexity, and thus computation time, of experimenting with hand-tuning prompts for large black-box models, we then compared the performance of the best "positive thinking" prompt against the output of systematic prompt optimization. We show that employing an automated pr

Source-Free Video Unsupervised Domain Adaptation (SFVUDA) task consists in adapting an action recognition model, trained on a labelled source dataset, to an unlabelled target dataset, without accessing the actual source data. The previous approaches have attempted to address SFVUDA by leveraging self-supervision (e.g., enforcing temporal consistency) derived from the target data itself. In this work, we take an orthogonal approach by exploiting "web-supervision" from Large Language-Vision Models (LLVMs), driven by the rationale that LLVMs contain a rich world prior surprisingly robust to domain-shift. We showcase the unreasonable effectiveness of integrating LLVMs for SFVUDA by devising an intuitive and parameter-efficient method, which we name Domain Adaptation with Large Language-Vision models (DALL-V), that distills the world prior and complementary source model information into a student network tailored for the target. Despite the simplicity, DALL-V achieves significant improvement over state-of-the-art SFVUDA methods.

Existing theory predicts that data heterogeneity will degrade the performance of the Federated Averaging (FedAvg) algorithm in federated learning. However, in practice, the simple FedAvg algorithm converges very well. This paper explains the seemingly unreasonable effectiveness of FedAvg that contradicts the previous theoretical predictions. We find that the key assumption of bounded gradient dissimilarity in previous theoretical analyses is too pessimistic to characterize data heterogeneity in practical applications. For a simple quadratic problem, we demonstrate there exist regimes where large gradient dissimilarity does not have any negative impact on the convergence of FedAvg. Motivated by this observation, we propose a new quantity, average drift at optimum, to measure the effects of data heterogeneity, and explicitly use it to present a new theoretical analysis of FedAvg. We show that the average drift at optimum is nearly zero across many real-world federated training tasks, whereas the gradient dissimilarity can be large. And our new analysis suggests FedAvg can have identical convergence rates in homogeneous and heterogeneous data settings, and hence, leads to better under

This paper discusses, from a mathematician's point of view, the thesis formulated by Israel Gelfand, one of the greatest mathematicians of the 20th century, and one of the pioneers of mathematical biology: "There is only one thing which is more unreasonable than the unreasonable effectiveness of mathematics in physics, and this is the unreasonable ineffectiveness of mathematics in biology."

There is a significant need for principled uncertainty reasoning in machine learning systems as they are increasingly deployed in safety-critical domains. A new approach with uncertainty-aware regression-based neural networks (NNs), based on learning evidential distributions for aleatoric and epistemic uncertainties, shows promise over traditional deterministic methods and typical Bayesian NNs, notably with the capabilities to disentangle aleatoric and epistemic uncertainties. Despite some empirical success of Deep Evidential Regression (DER), there are important gaps in the mathematical foundation that raise the question of why the proposed technique seemingly works. We detail the theoretical shortcomings and analyze the performance on synthetic and real-world data sets, showing that Deep Evidential Regression is a heuristic rather than an exact uncertainty quantification. We go on to discuss corrections and redefinitions of how aleatoric and epistemic uncertainties should be extracted from NNs.

We survey applications of Bridgeland stability conditions in algebraic geometry and discuss open questions for future research.

A function g, with domain the natural numbers, is a quasi-polynomial if there exists a period m and polynomials p_0,p_1,...,p_{m-1} such that g(t)=p_i(t) for t=i mod m. Quasi-polynomials classically -- and "reasonably" -- appear in Ehrhart theory and in other contexts where one examines a family of polyhedra, parametrized by a variable t, and defined by linear inequalities of the form a_1x_1+...+a_dx_d <= b(t). Recent results of Chen, Li, Sam; Calegari, Walker; and Roune, Woods show a quasi-polynomial structure in several problems where the a_i are also allowed to vary with t. We discuss these "unreasonable" results and conjecture a general class of sets that exhibit various (eventual) quasi-polynomial behaviors: sets S_t of d-tuples of natural numbers that are defined with quantifiers ("for all", "there exists"), boolean operations (and, or, not), and statements of the form a_1(t)x_1+...+a_d(t)x_d <= b(t), where a_i(t) and b(t) are polynomials in t. These sets are a generalization of sets defined in the Presburger arithmetic. We prove several relationships between our conjectures, and we prove several special cases of the conjectures. The title is a play on Eugene Wigner's "

Terry Sejnowski's 2020 paper [arXiv:2002.04806] is entitled "The unreasonable effectiveness of deep learning in artificial intelligence". However, the paper doesn't attempt to answer the implied question of why Deep Convolutional Neural Networks (DCNNs) can approximate so many of the mappings that they have been trained to model. While there are detailed mathematical analyses, this short paper attempts to look at the issue differently, considering the way that these networks are used, the subset of these functions that can be achieved by training (starting from some location in the original function space), as well as the functions to which these networks will actually be applied.