共找到 20 条结果
TriCG is a short-recurrence iterative method recently introduced by Montoison and Orban [SIAM J. Sci. Comput., 43 (2021), pp. A2502--A2525] for solving symmetric quasi-definite (SQD) linear systems. TriCG takes advantage of the inherent block structure of SQD linear systems and performs substantially better than SYMMLQ. However, numerical experiments have revealed that the convergence of TriCG can be notably slow when the off-diagonal block contains a substantial number of large elliptic singular values. To address this limitation, we introduce a deflation strategy tailored for TriCG to improve its convergence behavior. Specifically, we develop a generalized Saunders--Simon--Yip process with deflated restarting to construct the deflation subspaces. Building upon this process, we propose a novel method termed TriCG with deflated restarting. The deflation subspaces can also be utilized to solve SQD linear systems with multiple right-hand sides. Numerical experiments are provided to illustrate the superior performance of the proposed methods.
Communication-efficient distributed optimizers such as DiLoCo reduce synchronization costs by letting workers perform many local updates before aggregating their progress with an outer momentum optimizer. Recent theory suggests that the outer optimizer acts on an effective spectrum induced by the inner optimization loop, and that the choice of outer momentum controls how progress from local updates is accumulated across communication rounds. We study periodic restarting of the outer momentum as a simple complementary mechanism for controlling this outer memory. In a linearized squared-loss model where prediction-space residuals evolve under the empirical NTK, we derive a mode-wise restart contraction showing that resets exploit phase cancellation by discarding stale momentum while preserving inner-loop progress. Toy experiments verify the predicted contraction behavior, and language-model pretraining experiments show that periodic restarts widen the stable range of outer learning rates and momentum values across communication periods.
We develop a sketch-and-restart framework for computing the action of a matrix function on a vector, $f(A) b$, where $A$ is large, sparse, and non-Hermitian. The framework combines quadrature-based restarting with Arnoldi-like decompositions generated by sketched or truncated Arnoldi processes. Within this framework, we develop two classes of restarted algorithms. The first uses a fixed Krylov subspace dimension and is based either on the sketched Arnoldi process or on a new sketched harmonic Arnoldi process proposed in this work. The second class chooses the Krylov subspace dimension adaptively by running the truncated Arnoldi process until the condition number of the generated basis, estimated from its sketch, exceeds a prescribed threshold. We also establish the convergence of the restarted sketched harmonic Arnoldi method for Stieltjes functions under the assumption that $A$ is positive real. Numerical experiments demonstrate the effectiveness of the proposed framework, including the computational savings achieved through sketching, the storage reduction enabled by adaptive truncation, and the acceleration obtained from thick restarting.
Nonlinear optimization methods are typically iterative and make use of gradient information to determine a direction of improvement and function information to effectively check for progress. When this information is corrupted by noise, designing a convergent and practical algorithmic process becomes challenging, as care must be taken to avoid taking bad steps due to erroneous information. For this reason, simple gradient-based schemes have been quite popular, despite being outperformed by more advanced techniques in the noiseless setting. In this paper, we propose a general algorithmic framework based on line search that is endowed with iteration and evaluation complexity guarantees even in a noisy setting. These guarantees are obtained as a result of a restarting condition, that monitors desirable properties for the steps taken at each iteration and can be checked even in the presence of noise. Experiments using a nonlinear conjugate gradient variant and a quasi-Newton variant illustrate that restarting can be performed without compromising practical efficiency and robustness.
We consider the mean first passage time (MFPT) for a diffusive particle in a potential landscape with the extra condition that the particle is reset to its original position with some rate r. We study non-smooth and non-convex potentials and focus on the case where the restart rate depends on the space coordinate. There, we show that it is beneficial to restart at a lower rate once you are closer to your intended target.
In this article, I review past, current, and future advances on the study of radio-loud AGN (RLAGN; radio-loud quasars and radio galaxies) lifecycles exclusively in the remnant and restarting phases. I focus on their dynamics and energetics as inferred from radio observations while discussing their radiative lifetimes, population statistics, and trends in their physical characteristics. I briefly summarize multi-wavelength observations, particularly X-rays, that have enabled studies of the large-scale environments of RLAGN in order to understand their role in feedback. Furthermore, I discuss analytic and numerical simulations that predict key properties of remnant and restarting sources as found in wide-area surveys, and discuss the prospects of future surveys that may shed further light on these elusive subpopulations of RLAGN.
We present a new Krylov subspace recycling method for solving a linear system of equations, or a sequence of slowly changing linear systems. Our approach is to reduce the computational overhead of recycling techniques while still benefiting from the acceleration afforded by such techniques. As such, this method augments an unprojected Krylov subspace. Furthermore, it combines randomized sketching and deflated restarting in a way that avoids orthogononalizing a full Krylov basis. We call this new method GMRES-SDR (sketched deflated restarting). With this new method, we provide new theory, which initially characterizes unaugmented sketched GMRES as a projection method for which the projectors involve the sketching operator. We demonstrate that sketched GMRES and its sibling method sketched FOM are an MR/OR pairing, just like GMRES and FOM. We furthermore obtain residual convergence estimates. Building on this, we characterize GMRES-SDR also in terms of sketching-based projectors. Compression of the augmented Krylov subspace for recycling is performed using a sketched version of harmonic Ritz vectors. We present results of numerical experiments demonstrating the effectiveness of GMRES
We propose new restarting strategies for accelerated gradient and accelerated coordinate descent methods. Our main contribution is to show that the restarted method has a geometric rate of convergence for any restarting frequency, and so it allows us to take profit of restarting even when we do not know the strong convexity coefficient. The scheme can be combined with adaptive restarting, leading to the first provable convergence for adaptive restarting schemes with accelerated gradient methods. Finally, we illustrate the properties of the algorithm on a regularized logistic regression problem and on a Lasso problem.
In this paper a new restarting method for Krylov subspace matrix exponential evaluations is proposed. Since our restarting technique essentially employs the residual, some convergence results for the residual are given. We also discuss how the restart length can be adjusted after each restart cycle, which leads to an adaptive restarting procedure. Numerical tests are presented to compare our restarting with three other restarting methods. Some of the algorithms described in this paper are a part of the Octave/Matlab package expmARPACK available at http://team.kiam.ru/botchev/expm/.
An accurate residual--time (AccuRT) restarting for computing matrix exponential actions of nonsymmetric matrices by the shift-and-invert (SAI) Krylov subspace method is proposed. The proposed restarting method is an extension of the recently proposed RT (residual--time) restarting and it is designed to avoid a possible accuracy loss in the conventional RT restarting. An expensive part of the SAI Krylov method is solution of linear systems with the shifted matrix. Since the AccuRT algorithm adjusts the shift value, we discuss how the proposed restarting can be implemented with just a single LU~factorization (or a preconditioner setup) of the shifted matrix. Numerical experiments demonstrate an improved accuracy and efficiency of the approach.
Several radio galaxies are known that show radio morphological signatures that are best interpreted as restarting of nuclear activity after a period of quiescence. The conditions surrounding the phenomenon of nuclear recurrence are not understood. In this paper we have attempted to address this question by examining the nuclear fuelling characteristics in a sample of restarting radio galaxies. We have examined the detection rate for molecular gas in a representative sample of nine restarting radio galaxies, for seven of which we present new upper limits to the molecular gas mass derived from CO line observations we made with the IRAM 30-m telescope. We derive a low CO detection rate for the relatively young restarted radio galaxies suggesting that the cessation of the nuclear activity and its subsequent restarting may be a result of instabilities in the fuelling process rather than a case of depletion of fuel followed by a recent fuel acquisition. It appears that abundant molecular gas content at the level of few 10^8 to 10^9 solar masses does not necessarily accompany the nuclear restarting phenomenon. For comparison we also discuss the molecular gas properties of five normal gian
Following some previous studies on restarting automata, we introduce a refined model - the h-lexicalized restarting automaton (h-RLWW). We argue that this model is useful for expressing lexicalized syntax in computational linguistics. We compare the input languages, which are the languages traditionally considered in automata theory, to the so-called basic and h-proper languages, which are (implicitly) used by categorial grammars, the original tool for the description of lexicalized syntax. The basic and h-proper languages allow us to stress several nice properties of h-lexicalized restarting automata, and they are suitable for modeling the analysis by reduction and, subsequently, for the development of categories of a lexicalized syntax. Based on the fact that a two-way deterministic monotone restarting automaton can be transformed into an equivalent deterministic monotone RL-automaton in (Marcus) contextual form, we obtain a transformation from monotone RLWW-automata that recognize the class CFL of context-free languages as their input languages to deterministic monotone h-RLWW-automata that recognize CFL through their h-proper languages. Through this transformation we obtain aut
We propose algorithms for efficient time integration of large systems of oscillatory second order ordinary differential equations (ODEs) whose solution can be expressed in terms of trigonometric matrix functions. Our algorithms are based on a residual notion for second order ODEs, which allows to extend the ``residual-time restarting'' Krylov subspace framework -- which was recently introduced for exponential and $\varphi$-functions occurring in time integration of first order ODEs -- to our setting. We then show that the computational cost can be further reduced in many cases by using our restarting in the Gautschi cosine scheme. We analyze residual convergence in terms of Faber and Chebyshev series and supplement these theoretical results by numerical experiments illustrating the efficiency of the proposed methods.
An efficient and robust restart strategy is important for any Krylov-based method for eigenvalue problems. The tensor infinite Arnoldi method (TIAR) is a Krylov-based method for solving nonlinear eigenvalue problems (NEPs). This method can be interpreted as an Arnoldi method applied to a linear and infinite dimensional eigenvalue problem where the Krylov basis consists of polynomials. We propose new restart techniques for TIAR and analyze efficiency and robustness. More precisely, we consider an extension of TIAR which corresponds to generating the Krylov space using not only polynomials but also structured functions that are sums of exponentials and polynomials, while maintaining a memory efficient tensor representation. We propose two restarting strategies, both derived from the specific structure of the infinite dimensional Arnoldi factorization. One restarting strategy, which we call semi-explicit TIAR restart, provides the possibility to carry out locking in a compact way. The other strategy, which we call implicit TIAR restart, is based on the Krylov-Schur restart method for linear eigenvalue problem and preserves its robustness. Both restarting strategies involve approximati
Anderson Acceleration (AA) is a popular acceleration technique to enhance the convergence of fixed-point iterations. The analysis of AA approaches typically focuses on the convergence behavior of a corresponding fixed-point residual, while the behavior of the underlying objective function values along the accelerated iterates is currently not well understood. In this paper, we investigate local properties of AA with restarting applied to a basic gradient scheme in terms of function values. Specifically, we show that AA with restarting is a local descent method and that it can decrease the objective function faster than the gradient method. These new results theoretically support the good numerical performance of AA when heuristic descent conditions are used for globalization and they provide a novel perspective on the convergence analysis of AA that is more amenable to nonconvex optimization problems. Numerical experiments are conducted to illustrate our theoretical findings.
We present a study on lookahead hierarchies for restarting automata with auxiliary symbols and small lookahead. In particular, we show that there are just two different classes of languages recognised RRWW automata, through the restriction of lookahead size. We also show that the respective (left-) monotone restarting automaton models characterise the context-free languages and that the respective right-left-monotone restarting automata characterise the linear languages both with just lookahead length 2.
It is well-known that first-order methods can offer accelerated convergence rates in the presence of growth structures. Restarting schemes provide a general tool for such speed-ups. These schemes typically either require unrealistic problem knowledge, incur logarithmic overhead factors in oracle complexity, and/or have a nontrivial initial burn-in phase. We present a parameter-free approach for restarting any first-order method, avoiding these three drawbacks. Our approach dynamically deploys parallel instances of a given first-order method communicating progress in the style of Renegar and Grimmer. Our optimized scheme avoids expensive burn-ins and only requires $O(\log\log(1/ε))$ parallel processes when the accelerated rate is sublinear.
Recent evidence on the sustainment of wall-normal-velocity bursts in wall-bounded turbulence challenges the classical streak-dependent picture, suggesting that the problem should be approached relying on no a priori knowledge regarding other flow structures. This paper discusses the restarts of bursts in a log-minimal channel within the framework of a linearised Navier-Stokes system with forcing terms encapsulating the nonlinear effects of all other structures. Two generic issues are addressed. The first concerns the conditions for burst-restart-like solutions for the forced linearised system itself. We formulate optimisation problems to understand the 'minimal requirements' for burst restarting. The solutions illustrate three conceptual periods in a typical restarting process, distinguished by the behaviour of spanwise vorticity structures: breakup, counter-rotating catch-up, and co-rotating catch-up. External forces promote this process by breaking up forward-inclined vortices and merging co-rotating, catching-up vortices. A quantity termed linearly available energy (LAE) is accordingly proposed to parameterise the restarting process. The second issue concerns the contributory fe
Learning rate scheduling plays a critical role in the optimization of deep neural networks, directly influencing convergence speed, stability, and generalization. While existing schedulers such as cosine annealing, cyclical learning rates, and warm restarts have shown promise, they often rely on fixed or periodic triggers that are agnostic to the training dynamics, such as stagnation or convergence behavior. In this work, we propose a simple yet effective strategy, which we call Stochastic Gradient Descent with Escalating Restarts (SGD-ER). It adaptively increases the learning rate upon convergence. Our method monitors training progress and triggers restarts when stagnation is detected, linearly escalating the learning rate to escape sharp local minima and explore flatter regions of the loss landscape. We evaluate SGD-ER across CIFAR-10, CIFAR-100, and TinyImageNet on a range of architectures including ResNet-18/34/50, VGG-16, and DenseNet-101. Compared to standard schedulers, SGD-ER improves test accuracy by 0.5-4.5%, demonstrating the benefit of convergence-aware escalating restarts for better local optima.
We introduce a new restarting scheme for a continuous inertial dynamics with Hessian driven-damping, and establish a linear convergence rate for the function values along the restarted trajectories. The proposed routine is implemented without knowing the strong convexity parameter, and is a generalization of existing speed restart schemes. It interpolates between speed and function value restarts, considerably delaying the restarting time, while preserving convergence and function value decrease. Numerical experiments show an improvement in the convergence rates for both continuous-time dynamical systems, and the associated accelerated first-order algorithms derived via time discretization.