搜索结果：Epilogue

4-bit quantization reduces the memory footprint and latency of large language model inference, but its aggressive precision reduction can severely degrade accuracy. Prior methods address this by decomposing each weight matrix into two components (e.g., via singular value decomposition) and quantizing them separately, assigning the bulk of values to a low-precision residual component while handling outliers with a high-precision low-rank component. However, such decompositions are designed to minimize the real-valued energy of the residual, rather than the post-quantization error of the residual and low-rank components. We propose TwinQuant, a 4-bit quantization framework that learns quantization-friendly decomposed subspaces and jointly reshapes both the low-rank and residual components. TwinQuant learns component-specific transformations via a joint optimization over the Stiefel and general linear manifolds, flattening their distributions and reducing dynamic-range imbalance. To enable efficient end-to-end execution, we further design a fused dual-component kernel that pipelines the two-stage low-rank computation on-chip and merges both components with a single epilogue, avoiding

Optimizing GPU kernels with LLM agents is an iterative process over a large design space. Every candidate must be generated, compiled, validated, and profiled, so fewer trials will save both runtime and cost. We make two key observations. First, the abstraction level that agents operate at is important. If it is too low, the LLM wastes reasoning on low-impact details. If it is too high, it may miss important optimization choices. Second, agents cannot easily tell when they reach the point of diminishing returns, wasting resources as they continue searching. These observations motivate two design principles to improve efficiency: (1) a compact domain-specific language (DSL) that can be learned in context and lets the model reason at a higher level while preserving important optimization levers, and (2) Speed-of-Light (SOL) guidance that uses first-principles performance bounds to steer and budget search. We implement these principles in $μ$CUTLASS, a DSL with a compiler for CUTLASS-backed GPU kernels that covers kernel configuration, epilogue fusion, and multi-stage pipelines. We use SOL guidance to estimate headroom and guide optimization trials, deprioritize problems that are near

Post-training INT8 (W8A8) quantization of diffusion transformers is widely deployed as a speed optimization, yet on consumer Ampere GPUs it is frequently slower than the FP8 and NF4 alternatives it is meant to beat. We trace this to a software artifact: the production "INT8" forward quantizes weights and activations only to immediately dequantize them back to bf16 and run a bf16 matrix multiply, never engaging the GPU's INT8 tensor cores, so the hardware's compute advantage is left entirely unrealized. We close this gap with a single fused Triton INT8 GEMM (int8xint8->int32 on Ampere tensor cores, with per-token x per-channel dequantization and bias folded into the epilogue, autotuned per GEMM shape) dropped into the Ideogram 4.0 diffusion transformer's linear layers in place of the dequantize-to-bf16 path. In the kernel, the int8xint8->int32 accumulation is bit-exact against torch._int_mm and the dequantized output matches the reference at cosine similarity 1.0 with no NaNs, running 2.8-4.2x faster than bf16 per GEMM. End to end it delivers a ~1.1x (~9-10%) speedup at 768px, and at 1024px it generates an image in 156.5 s on a single RTX 3090, faster than the single-card NF4

Modern machine learning (ML) workloads increasingly rely on GPUs, yet achieving high end-to-end performance remains challenging due to dependencies on both GPU kernel efficiency and host-side settings. Although LLM-based methods show promise on automated GPU kernel generation, prior works mainly focus on single-kernel optimization and do not extend to end-to-end programs, hindering practical deployment. To address the challenge, in this work, we propose StitchCUDA, a multi-agent framework for end-to-end GPU program generation, with three specialized agents: a Planner to orchestrate whole system design, a Coder dedicated to implementing it step-by-step, and a Verifier for correctness check and performance profiling using Nsys/NCU. To fundamentally improve the Coder's ability in end-to-end GPU programming, StitchCUDA integrates rubric-based agentic reinforcement learning over two atomic skills, task-to-code generation and feedback-driven code optimization, with combined rubric reward and rule-based reward from real executions. Therefore, the Coder learns how to implement advanced CUDA programming techniques (e.g., custom kernel fusion, cublas epilogue), and we also effectively preven

SwiGLU is the dominant MLP activation in modern large language models, yet its intermediate tensor materialization costs 9-37% of MLP execution time. We present two complementary CUTLASS-based SM90 kernels that fuse SwiGLU into GeMM at the tile level. Kernel-1 overlaps Swish computation on the Gate accumulator with Up-tile loading using the Pingpong warp-specialized schedule; Kernel-2 interleaves SwiGLU with tile stores via a custom Epilogue Visitor Tree. Evaluated on Qwen-2.5 models (0.5B-72B) on NVIDIA H100, our kernels achieve up to 2.47x speedup over PyTorch, shifting workloads from memory-bound to compute-bound and reaching 79.5% peak BF16 utilization. We demonstrate that torch.compile cannot replicate this fusion (3-7x slower than our kernels), validating the need for hand-crafted tile-level design. Our fused kernels are also numerically superior, achieving zero mismatches compared to 4.5-11% for cuBLAS.

Sampling from a categorical distribution is mathematically simple, but in large-vocabulary decoding, it often triggers extra memory traffic and extra kernels after the LM head. We present FlashSampling, an exact sampling primitive that fuses sampling into the LM-head matmul and never materializes the logits tensor in HBM. The method is simple: compute logits tile-by-tile on chip, add Gumbel noise, keep only one maximizer per row and per vocabulary tile, and finish with a small reduction over tiles. In tensor-parallel decoding, FlashSampling replaces the all-gather of logits with streaming peer-to-peer writes: This overlaps GPU-to-GPU communication with computation and HBM loads across up to 8 GPUs, with near-ideal scaling at large batch sizes. Our kernel is exact because argmax decomposes over partitions; grouped variants for online and tensor-parallel settings are exact by hierarchical factorization of the categorical distribution. FlashSampling demonstrates kernel-level speedups on decode workloads across 4 different datacenter GPUs (H100, H200, B200, B300), and in end-to-end vLLM experiments, it reduces time per output token by up to $10\%$ on the models we test. These results s

Efficient large language model (LLM) serving is increasingly constrained by deployment cost. Quantization is a key technique for reducing serving cost, yet even state-of-the-art 4-bit quantizers exhibit a noticeable quality gap from FP16, particularly for smaller models where low-bit serving is most beneficial. We identify a fundamental cause of this gap: quantization error is highly input-dependent and varies substantially across tokens, while existing post-quantization compensation methods are static and apply identical corrections to all inputs. As a result, easy tokens are over-corrected while hard tokens remain under-corrected. We present SPEAR, a system for post-quantization error-adaptive recovery that improves low-bit LLM serving. SPEAR introduces lightweight Error Compensators (ECs) modulated by per-token gates and places them only at the most error-sensitive layers identified through a CKA-guided entropy-aware diagnostic. This focuses a small parameter budget where it is most effective. Efficient deployment of ECs presents several systems challenges, including additional computation, tensor-parallel synchronization caused by input-dependent gating, and latency instability

Many have have taken in hand to write a treatise on the Star of Bethlehem, particularly on Kepler's explanation as a stellar birth, triggered by Mars joining a great conjunction (a meeting of Jupiter and Saturn), as he observed it in October 1604. It is the aim of my short treatise to introduce, at a hopefully well digestible level, into underlying antiquity's concepts of physics, astrology, and astronomy around the time of Christ's birth and to demonstrate their essential developments up to the early modern period. In the latter, Kepler played a significant role. His contributions enabled him to calculate back to the year 7 BC with astonishing precision, locating a triple great conjunction in Pisces, which was then complemented by Mars. This is confirmed by both Babylonian tradition and modern astronomy. The following background provided may be specifically helpful for a more theologically oriented audience, also because the controversy surrounding "geocentric" and "heliocentric" world systems (Copernicus) is touched upon. And, because the overall context raises questions for theology, with which the epilogue concludes.

Brownian Boost is a one-parameter family of stochastic differential games played on the real line in which players spend at rates of their choosing in an ongoing effort to influence the drift of a randomly diffusing point particle~$X$. One or other player is rewarded, at time infinity, according to whether~$X$ tends to plus or minus infinity. Each player's net receipt is the final reward (only for the victor) minus the player's total spend. We characterise and explicitly compute the time-homogeneous Markov-perfect Nash equilibria of Brownian Boost, finding the derivatives of the players' expected payoffs to solve a pair of coupled first-order non-linear ODE. Brownian Boost is a high-noise limit of a two-dimensional family of player-funded tug-of-war games, one of which was studied in~\cite{LostPennies}. We analyse the discrete games, finding them, and Brownian Boost, to exemplify key features studied in the economics literature of tug-of-war initiated by~\cite{HarrisVickers87}: a battlefield region where players spend heavily; stakes that decay rapidly but asymmetrically in distance to the battlefield; and an effect of discouragement that makes equilibria fragile under asymmetric p

Large language models (LLMs) enabled dialogue systems have become one of the central modes in human-machine interaction, which bring about vast amounts of conversation logs and increasing demand for dialogue generation. The dialogue's life-cycle spans from $\textit{Prelude}$ through $\textit{Interlocution}$ to $\textit{Epilogue}$, encompassing rich dialogue elements. Despite large volumes of dialogue-related studies, there is a lack of systematic investigation into the dialogue stages to frame benchmark construction that covers comprehensive dialogue elements. This hinders the precise modeling, generation and assessment of LLMs-based dialogue systems. To bridge this gap, in this paper, we introduce a new research task--$\textbf{D}$ialogue $\textbf{E}$lement $\textbf{MO}$deling, including $\textit{Element Awareness}$ and $\textit{Dialogue Agent Interaction}$, and propose a novel benchmark, $\textbf{DEMO}$, designed for a comprehensive dialogue modeling and assessment. On this basis, we further build the DEMO agent with the adept ability to model dialogue elements via imitation learning. Extensive experiments on DEMO indicate that current representative LLMs still have considerable p

The research work in the thesis is focused on the thermodynamic analysis of cosmological models, especially the models that explain late-time cosmic acceleration. According to the cosmological principle, the universe is spatially homogeneous and isotropic. FRW metric is considered to describe it. In the initial chapter, an overview of cosmology is presented. Chapter 2 delves into thermodynamics applied to cosmology, emphasizing the Generalized Second Law. It thoroughly covers Hayward-Kodama temperature and discusses the apparent horizon and the conditions for thermodynamic stability. Chapter 3 presents a thermodynamic comparison of quintessence models, focusing on thawing and freezing scenarios. Chapter 4 examines Brans-Dicke cosmological models in a spatially isotropic and homogeneous universe, assessing their compatibility with the Generalized Second Law of Thermodynamics. Chapter 5 explores the thermodynamic viability of various dark energy models reconstructed using the cosmological jerk parameter. Chapter 6 examines a model in a spatially flat FRW spacetime that mimics the characteristics of the $Λ$CDM model, assessing its thermodynamic stability. Chapter 7 serves as the epilo

Recently, we introduced a symmetry on the structure of angular momentum which interchanges internal and external degrees of freedom. The spin-orbit duality is a holographic map that projects a massive theory in four-dimensional flat spacetime onto the three-dimensional $\mathbb{S}^2\times\mathbb{R}$ null infinity. This cylinder has radius $R\sim1/m$ and, quantum-mechanically, its vacuum state is a fuzzy sphere. Progress shows that, first, this duality realizes the Hopf map, a fact manifest on the superparticle. Secondly, the bulk Poincarè group transforms into the conformal group on the cylinder. In fact, the general version of the duality yields that the dual symmetries include the BMS group, as is appropriate at null infinity. As an example, the Landau levels in $\mathbb{R}^3$ are shown to match those of a Dirac monopole on the dual $\mathbb{S}^2$, in the thermodynamic limit. This dual system is actually identified with a three-dimensional critical Ising model. The map is then realized on $N_f$ massive fermions in flat space which, indeed, are the hologram of $2N_f$ massless fermions on the cylinder. However, the dual space is really the conformal class of $\mathbb{S}^2\times\mat

Building on arXiv:1902.03807, this paper develops a unifying study on the boundedness properties of several representative classes of hybrid operators, i.e. operators that enjoy both zero and non-zero curvature features. Specifically, via the LGC-method, we provide suitable $L^p$ bounds for three classes of operators: (1) Carleson-type operators, (2) Hilbert transform along variable curves, and, taking the center stage, (3) Bilinear Hilbert transform and bilinear maximal operators along curves. All these classes of operators will be studied in the context of hybrid curves with no quadratic resonances. The above study is interposed between two naturally derived topics: i) A prologue providing a first rigorous account on how the presence/absence of a higher order modulation invariance property interacts with and determines the nature of the method employed for treating operators with such a property. ii) An epilogue revealing how several key ingredients within our present study can blend and inspire a short, intuitive new proof of the smoothing inequality that plays the central role in the analysis of the curved version of the triangular Hilbert transform treated in arXiv:2008.10140.