This paper introduces Growing Networks with Autonomous Pruning (GNAP) for image classification. Unlike traditional convolutional neural networks, GNAP change their size, as well as the number of parameters they are using, during training, in order to best fit the data while trying to use as few parameters as possible. This is achieved through two complementary mechanisms: growth and pruning. GNAP start with few parameters, but their size is expanded periodically during training to add more expressive power each time the network has converged to a saturation point. Between these growing phases, model parameters are trained for classification and pruned simultaneously, with complete autonomy by gradient descent. Growing phases allow GNAP to improve their classification performance, while autonomous pruning allows them to keep as few parameters as possible. Experimental results on several image classification benchmarks show that our approach can train extremely sparse neural networks with high accuracy. For example, on MNIST, we achieved 99.44% accuracy with as few as 6.2k parameters, while on CIFAR10, we achieved 92.2\ accuracy with 157.8k parameters.
It is a celebrated fact that a simple random walk on an infinite $k$-ary tree for $k \geq 2$ returns to the initial vertex at most finitely many times during infinitely many transitions; it is called transient. This work points out the fact that a simple random walk on an infinitely growing $k$-ary tree can return to the initial vertex infinitely many times, it is called recurrent, depending on the growing speed of the tree. Precisely, this paper is concerned with a simple specific model of a random walk on a growing graph (RWoGG), and shows a phase transition between the recurrence and transience of the random walk regarding the growing speed of the graph. To prove the phase transition, we develop a coupling argument, introducing the notion of less homesick as graph growing (LHaGG). We also show some other examples, including a random walk on $\{0,1\}^n$ with infinitely growing $n$, of the phase transition between the recurrence and transience. We remark that some graphs concerned in this paper have infinitely growing degrees.
Loewner chains are ubiquitous in the theory of slit mappings, and hence in the study of bounded conformal maps. They have attracted new interest in the past decades through their applications to statistical physics and fractal geometry, particularly in contexts involving randomness. In this article, we delve into topological features of the growing hulls obtained from Loewner chains with a general local growth property, inspired by the classical works of Loewner and Pommerenke. We first revisit Loewner's theorem, associating to each locally growing collection of hulls a real-valued driving function W, possibly discontinuous. We then investigate the points chronologically added to the growing hulls, which may be part of a simply connected swallowed ``bubble'', or a compact connected boundary set. For continuous driving functions, the Loewner chain can often be associated with a continuous curve (dubbed ``generating curve''). Motivated by this, we introduce a more general notion of a ``generating function'' for the Loewner chain, and characterize when there exists such a function η (which can be continuous, càdlàg, càglàd, or neither). We then investigate the necessity of left and ri
We consider growing open chemical reaction systems (CRSs), in which autocatalytic chemical reactions are encapsulated in a finite volume and its size can change in conjunction with the reactions. The thermodynamics of growing CRSs is indispensable for understanding biological cells and designing protocells by clarifying the physical conditions and costs for their growing states. In this work, we establish a thermodynamic theory of growing CRSs by extending the Hessian geometric structure of non-growing CRSs. The theory provides the environmental conditions to determine the fate of the growing CRSs; growth, shrinking or equilibration. We also identify thermodynamic constraints; one to restrict the possible states of the growing CRSs and the other to further limit the region where a nonequilibrium steady growing state can exist. Moreover, we evaluate the entropy production rate in the steady growing state. The growing nonequilibrium state has its origin in the extensivity of thermodynamics, which is different from the conventional nonequilibrium states with constant volume. These results are derived from general thermodynamic considerations without assuming any specific thermodynamic
While the physics of disordered packing in non-growing systems is well understood, unexplored phenomena can emerge when packing takes place in growing domains. We study the arrangements of pigment cells (chromatophores) on squid skin as a biological example of a packed system on an expanding surface. We find that relative density fluctuations in cell numbers grow with spatial scale. We term this behavior ``hyperdisordered'', in contrast with hyperuniform behavior in which relative fluctuations tend to zero at large scale. We find that hyperdisordered scaling, akin to that of a critical system, is quantitatively reproduced by a model in which hard disks are randomly inserted in a homogeneously growing surface. In addition, we find that chromatophores increase in size during animal development, but maintain a stationary size distribution. The physical mechanisms described in our work may apply to a broad class of growing dense systems.
Recently, there has been increasing interest in efficient pretraining paradigms for training Transformer-based models. Several recent approaches use smaller models to initialize larger models in order to save computation (e.g., stacking and fusion). In this work, we study the fundamental question of how to select the best growing strategy from a given pool of growing strategies. Prior works have extensively focused on loss- and/or function-preserving behavior at initialization or simply performance at the end of training. Instead, we identify that behavior at initialization can be misleading as a predictor of final performance and present an alternative perspective based on early training dynamics, which we call "landscape-aware growing (LAG)". We perform extensive analysis of correlation of the final performance with performance in the initial steps of training and find early and more accurate predictions of the optimal growing strategy (i.e., with only a small "lag" after initialization). This perspective also motivates an adaptive strategy for gradual stacking.
Models of growing networks are a central topic in network science. In these models, vertices are usually labeled by their arrival time, distinguishing even those node pairs whose structural roles are identical. In contrast, unlabeled networks encode only structure, so unlabeled growth rules must be defined in terms of structurally distinguishable outcomes; network symmetries therefore play a key role in unlabeled growth dynamics. Here, we introduce and study models of growing unlabeled trees, defined in analogy to widely-studied labeled growth models such as uniform and preferential attachment. We develop a theoretical formalism to analyze these trees via tracking their leaf-based statistics. We find that while many characteristics of labeled network growth are retained, numerous critical differences arise, caused primarily by symmetries among leaves in common neighborhoods. In particular, degree heterogeneity is enhanced, with the strength of this enhancement depending on details of growth dynamics: mild enhancement for uniform attachment, and extreme enhancement for preferential attachment. These results and the developed analytical formalism may be of interest beyond the setting
We introduce the Aurellion Function, a novel recursively defined fast-growing hierarchy based on Knuth's up-arrow notation, defined by $A_1 = 10 \uparrow\uparrow\uparrow 10$, $A_{n+1} = 10 \uparrow^{A_n} 10$, where the number of arrows in the operation increases superexponentially with $n$. We analyze its growth rate relative to classical hierarchies such as the fast-growing hierarchy $(f_α)_{α< \varepsilon_0}$, and discuss its provability status in formal arithmetic. We provide formal bounds showing $A_n$ dominates all functions provably total in Peano Arithmetic, situating the Aurellion Function near the proof-theoretic ordinal $Γ_0$ due to its ability to majorize all functions $f_α$ for $α< \varepsilon_0$. We also outline possible transfinite extensions indexed by countable ordinals, thus bridging symbolic large-number constructions and ordinal analysis.
We consider analogues of Grigorchuk-Gupta-Sidki (GGS-)groups acting on trees of growing degree; the so-called growing GGS-groups. These groups are not just infinite and do not possess the congruence subgroup property, but many of them are branch and have the $p$-congruence subgroup property, for a prime $p$. Among them, we find groups with maximal subgroups only of finite index, and with infinitely many such maximal subgroups. These give the first examples of finitely generated branch groups with infinitely many finite-index maximal subgroups. Additionally, we prove that congruence quotients of growing GGS-groups associated to a defining vector of zero sum give rise to Beauville groups.
In this work, we introduce Progressive Growing of Patch Size, an automatic curriculum learning approach for 3D medical image segmentation. Our approach progressively increases the patch size during model training, resulting in an improved class balance for smaller patch sizes and accelerated convergence of the training process. We evaluate our curriculum approach in two settings: a resource-efficient mode and a performance mode, both regarding Dice score performance and computational costs across 15 diverse and popular 3D medical image segmentation tasks. The resource-efficient mode matches the Dice score performance of the conventional constant patch size sampling baseline with a notable reduction in training time to only 44%. The performance mode improves upon constant patch size segmentation results, achieving a statistically significant relative mean performance gain of 1.28% in Dice Score. Remarkably, across all 15 tasks, our proposed performance mode manages to surpass the constant patch size baseline in Dice Score performance, while simultaneously reducing training time to only 89%. The benefits are particularly pronounced for highly imbalanced tasks such as lesion segmentat
Most real systems are growing. In order to model the evolution of real systems, many growing network models have been proposed to reproduce some specific topology properties. As the structure strongly influences the network function, designing the function-aimed growing strategy is also a significant task with many potential applications. In this letter, we focus on synchronization in the growing networks. In order to enhance the synchronizability during the network evolution, we propose the Spectral-Based Growing (SBG) strategy. Based on the linear stability analysis of synchronization, we show that our growing mechanism yields better synchronizability than the existing topology-aimed growing strategies in both artificial and real-world networks. We also observe that there is an optimal degree of new added nodes, which means adding nodes with neither too large nor too low degree could enhance the synchronizability. Furthermore, some topology measurements are considered in the resultant networks. The results show that the degree, node betweenness centrality from SBG strategy are more homogenous than those from other growing strategies. Our work highlights the importance of the func
This paper studies the bounded confidence model on growing fully-mixed populations. In this model, in addition to the usual opinion clusters, significant secondary clusters of smaller size appear systematically, while those secondary clusters appear erratically and include much fewer agents when the population is fixed. Through simulations, we derive the bifurcation diagram of the growing population model and compare it to the diagram obtained with an evolving probability density instead of agents, and with their equivalent with a fixed population. Our tests when changing the usual bounded confidence function into a smooth bounded confidence function suggest that these secondary clusters are mainly generated by a different mechanism when the population is growing than when it is fixed.
An amoeba is a tree together with instructions how to iteratively grow trees by adding paths of a fixed length $\ell$. This paper analyses such a growth process. An amoeba is mortal if all versions of the process are finite, and it is immortal if they are all infinite. We obtain some necessary and some sufficient conditions for mortality. In particular, for growing caterpillars in the case $\ell=1$ we characterize mortal amoebas. We discuss variations of the mortality concept, conjecture that some of them are equivalent, and support this conjecture for $\ell\in\{1,2\}$.
New entropy measures have been recently introduced for the quantification of the complexity of networks. Most of these entropy measures apply to static networks or to dynamical processes defined on static complex networks. In this paper we define the entropy rate of growing network models. This entropy rate quantifies how many labeled networks are typically generated by the growing network models. We analytically evaluate the difference between the entropy rate of growing tree network models and the entropy of tree networks that have the same asymptotic degree distribution. We find that the growing networks with linear preferential attachment generated by dynamical models are exponentially less than the static networks with the same degree distribution for a large variety of relevant growing network models. We study the entropy rate for growing network models showing structural phase transitions including models with non-linear preferential attachment. Finally, we bring numerical evidence that the entropy rate above and below the structural phase transitions follow a different scaling with the network size.
In this paper, we abstract a kind of stochastic processes from evolving processes of growing networks, this process is called growing network Markov chains. Thus the existence and the formulas of degree distribution are transformed to the corresponding problems of growing network Markov chains. First we investigate the growing network Markov chains, and obtain the condition in which the steady degree distribution exists and get its exact formulas. Then we apply it to various growing networks. With this method, we get a rigorous, exact and unified solution of the steady degree distribution for growing networks.
The determination of collision-free shortest paths among growing discs has previously been studied for discs with fixed growing rates. Here, we study a more general case of this problem, where: (1) the speeds at which the discs are growing are polynomial functions of degree $\dd$, and (2) the source and destination points are given as query points. We show how to preprocess the $n$ growing discs so that, for two given query points $s$ and $d$, a shortest path from $s$ to $d$ can be found in $O(n^2 \log (\dd n))$ time. The preprocessing time of our algorithm is $O(n^2 \log n + k \log k)$ where $k$ is the number of intersections between the growing discs and the tangent paths (straight line paths which touch the boundaries of two growing discs). We also prove that $k \in O(n^3\dd)$.
Soft, growing inflated beam robots, also known as everting vine robots, have previously been shown to navigate confined spaces with ease. Less is known about their ability to navigate three-dimensional open spaces where they have the potential to collapse under their own weight as they attempt to move through a space. Previous work has studied collapse of inflated beams and vine robots due to purely transverse or purely axial external loads. Here, we extend previous models to predict the length at which straight vine robots will collapse under their own weight at arbitrary launch angle relative to gravity, inflated diameter, and internal pressure. Our model successfully predicts the general trends of collapse behavior of straight vine robots. We find that collapse length increases non-linearly with the robot's launch angle magnitude, linearly with the robot's diameter, and with the square root of the robot's internal pressure. We also demonstrate the use of our model to determine the robot parameters required to grow a vine robot across a gap in the floor. This work forms the foundation of an approach for modeling the collapse of vine robots and inflated beams in arbitrary shapes.
We study the organization and dynamics of growing directed networks. These networks are built by adding nodes successively in such a way that each new node has $K$ directed links to the existing ones. The organization of a growing directed network is analyzed in terms of the number of ``descendants'' of each node in the network. We show that the distribution $P(S)$ of the size, $S$, of the descendant cluster is described generically by a power-law, $P(S) \sim S^{-η}$, where the exponent $η$ depends on the value of $K$ as well as the strength of preferential attachment. We determine that, in the case of growing random directed networks without any preferential attachment, $η$ is given by $1+1/K$. We also show that the Boolean dynamics of these networks is stable for any value of $K$. However, with a small fraction of reversal in the direction of the links, the dynamics of growing directed networks appears to operate on ``the edge of chaos'' with a power-law distribution of the cycle lengths. We suggest that the growing directed network may serve as another paradigm for the emergence of the scale-free features in network organization and dynamics.
Patterns and forms adopted by Nature, such as the shape of living cells, the geometry of shells and the branched structure of plants, are often the result of simple dynamical paradigms. Here we show that a growing self-interacting string attached to a tracking origin, modeled to resemble nascent polypeptides in vivo, develops helical structures which are more pronounced at the growing end. We also show that the dynamic growth ensemble shares several features of an equilibrium ensemble in which the growing end of the polymer is under an effective stretching force. A statistical analysis of native states of proteins shows that the signature of this non-equilibrium phenomenon has been fixed by evolution at the C-terminus, the growing end of a nascent protein. These findings suggest that a generic non-equilibrium growth process might have provided an additional evolutionary advantage for nascent proteins by favoring the preferential selection of helical structures.
In this paper, a novel cognitive architecture for action recognition is developed by applying layers of growing grid neural networks.Using these layers makes the system capable of automatically arranging its representational structure. In addition to the expansion of the neural map during the growth phase, the system is provided with a prior knowledge of the input space, which increases the processing speed of the learning phase. Apart from two layers of growing grid networks the architecture is composed of a preprocessing layer, an ordered vector representation layer and a one-layer supervised neural network. These layers are designed to solve the action recognition problem. The first-layer growing grid receives the input data of human actions and the neural map generates an action pattern vector representing each action sequence by connecting the elicited activation of the trained map. The pattern vectors are then sent to the ordered vector representation layer to build the time-invariant input vectors of key activations for the second-layer growing grid. The second-layer growing grid categorizes the input vectors to the corresponding action clusters/sub-clusters and finally the