搜索结果：Connections

Residual connections are central to modern deep learning architectures, enabling the training of very deep networks by mitigating gradient vanishing. Hyper-Connections recently generalized residual connections by introducing multiple connection strengths at different depths, thereby addressing the seesaw effect between gradient vanishing and representation collapse. However, Hyper-Connections increase memory access costs by expanding the width of hidden states. In this paper, we propose Frac-Connections, a novel approach that divides hidden states into multiple parts rather than expanding their width. Frac-Connections retain partial benefits of Hyper-Connections while reducing memory consumption. To validate their effectiveness, we conduct large-scale experiments on language tasks, with the largest being a 7B MoE model trained on up to 3T tokens, demonstrating that Frac-Connections significantly outperform residual connections.

For Hermitian connections on a Hermitian complex line bundle over a Riemannian manifold $(X,g)$, we can define the ``volume", which can be considered to be the ``mirror" of the standard volume for submanifolds. We call the critical points minimal connections. In this paper, (1) we prove monotonicity formulas for minimal connections with respect to some versions of volume functionals under certain conditions on $\dim X$ and the curvature of $g$. These formulas would be important in bubbling analysis. As a corollary, we obtain the vanishing theorem for minimal connections on the odd dimensional Euclidean space. (2) We see that the formal ``large radius limit" of the defining equation of minimal connections is that of Yang--Mills connections. Then the existence theorem of minimal connections is proved for a ``sufficiently large" metric. (3) We can consider deformed Donaldson--Thomas (dDT) connections on $G_2$-manifolds as ``mirrors" of calibrated (associative) submanifolds. We show that dDT connections are minimal connections, just as calibrated submanifolds are minimal submanifolds. By the argument specific to dDT connections, we obtain the stronger monotonicity formulas and vanishin

Given a base manifold $M$ and a Lie group $G$, we define $\bar{\cal A}^H_M$ a space of generalized $G$-connections on $M$ with the following properties: - The space of smooth connections ${\cal A}^\infty_M = \sqcup_π{\cal A}^\infty_π$ is densely embedded in $\bar{\cal A}^H_M = \sqcup_π\bar{\cal A}^H_{cc(π)}$; moreover, in contrast with the usual space of generalized connections, the embedding preserves topological sectors. - It is a homogeneous covering space for the standard space of generalized connections of loop quantization $\bar{\cal A}_M$. - It is a measurable space constructed as an inverse limit of of spaces of connections with a cutoff, much like $\bar{\cal A}_M$. At each level of the cutoff, a Haar measure, a BF measure and heat kernel measures can be defined. - The topological charge of generalized connections on closed manifolds $Q= \int Tr(F)$ in 2d, $Q= \int Tr(F \wedge F)$ in 4d, etc, is defined. - On a subdivided manifold, it can be calculated in terms of the spaces of generalized connections associated to its pieces. Thus, spaces of boundary connections can be computed from spaces associated to faces. - The soul of our generalized connections is a notion of higher

We present hyper-connections, a simple yet effective method that can serve as an alternative to residual connections. This approach specifically addresses common drawbacks observed in residual connection variants, such as the seesaw effect between gradient vanishing and representation collapse. Theoretically, hyper-connections allow the network to adjust the strength of connections between features at different depths and dynamically rearrange layers. We conduct experiments focusing on the pre-training of large language models, including dense and sparse models, where hyper-connections show significant performance improvements over residual connections. Additional experiments conducted on vision tasks also demonstrate similar improvements. We anticipate that this method will be broadly applicable and beneficial across a wide range of AI problems.

We examine the heap of linear connections on anchored vector bundles and Lie algebroids. Naturally, this covers the example of affine connections on a manifold. We present some new interpretations of classical results via this ternary structure of connections. Endomorphisms of linear connections are studied, and their ternary structure, in particular the endomorphism truss, is explicitly presented. We remark that the use of ternary structures in differential geometry is novel and that the endomorphism truss of linear connections provides a concrete geometric example of a truss.

We study the relation between torsion tensors of principal connections on G-structures and characteristic conic connections on associated cone structures. We formulate sufficient conditions under which the existence of a characteristic conic connection implies the existence of a torsion-free principal connection. We verify these conditions for adjoint varieties of simple Lie algebras, excluding those of type $\textsf{A}_{\ell eq 2}$ or $\textsf{C}_{\ell}$. As an application, we give a complete classification of the germs of minimal rational curves whose VMRT at a general point is such an adjoint variety: nontrivial ones come from lines on hyperplane sections of certain Grassmannians or minimal rational curves on wonderful group compactifications.

We find an explicit formula for the gamma vector in terms of the input polynomial in a way that extends it to arbitrary polynomials. More specifically, we find explicit linear combination in terms of coefficients of the input polynomial (using Catalan numbers and binomial coefficients) and an expression involving the derivative of the input polynomial. The first expression suggests connections to common Coxeter group/noncrossing partition structures in existing gamma positivity examples. In the case where the input is the $h$-polynomial of a simplicial complex, this gives an interpretation of the gamma vector as a measure of differences in local and global contributions. We also apply them to connect signs/inequalities of (shifts of) the gamma vector to upper/lower bound conditions on coefficients of the input polynomial. Finally, we make use of the shape of the sums used to make these estimates and connections with intersection numbers to relate these properties of the gamma vector to algebraic structures (e.g. characteristic classes involved in existing log concavity and Schur positivity properties).

We introduce the notion of a regular integrable connection on a smooth log scheme over $\mathbf{C}$ and construct an equivalence between the category of such connections and the category of integrable connections on its analytification, compatible with de Rham cohomology. This extends the work of Deligne (when the log structure is trivial), and combined with the work of Ogus yields a topological description of the category of regular connections in terms of certain constructible sheaves on the Kato--Nakayama space. The key ingredients are the notion of a canonical extension in this context and the existence of good compactifications of log schemes obtained recently by Włodarczyk.

Galois connections are a foundational tool for structuring abstraction in semantics and their use lies at the heart of the theory of abstract interpretation. Yet, mechanization of Galois connections using proof assistants remains limited to restricted modes of use, preventing their general application in mechanized metatheory and certified programming. This paper presents constructive Galois connections, a variant of Galois connections that is effective both on paper and in proof assistants; is complete with respect to a large subset of classical Galois connections; and enables more general reasoning principles, including the "calculational" style advocated by Cousot. To design constructive Galois connections we identify a restricted mode of use of classical ones which is both general and amenable to mechanization in dependently-typed functional programming languages. Crucial to our metatheory is the addition of monadic structure to Galois connections to control a "specification effect." Effectful calculations may reason classically, while pure calculations have extractable computational content. Explicitly moving between the worlds of specification and implementation is enabled by

Around 1923, Elie Cartan introduced affine connections on manifolds and definedthe main related concepts: torsion, curvature, holonomy groups. He discussed applications of these concepts in Classical and Relativistic Mechanics; in particular he explained how parallel transport with respect to a connection can be related to the principle of inertia in Galilean Mechanics and, more generally, can be used to model the motion of a particle in a gravitational field. In subsequent papers, Elie Cartan extended these concepts for other types of connections on a manifold: Euclidean, Galilean and Minkowskian connections which can be considered as special types of affine connections, the group of affine transformations of the affine tangent space being replaced by a suitable subgroup; and more generally, conformal and projective connections, associated to a group which is no more a subgroup of the affine group. Around 1950, Charles Ehresmann introduced connections on a fibre bundle and, when the bundle has a Lie group as structure group, connection forms on the associated principal bundle, with values in the Lie algebra of the structure group. He called Cartan connections the various types of

For bounded lattices, we introduce certain Galois connections, called (cyclically) essential, retractable and UC Galois connections, which behave well with respect to concepts of module-theoretic nature involving essentiality. We show that essential retractable Galois connections preserve uniform dimension, whereas essential retractable UC Galois connections induce a bijective correspondence between sets of closed elements. Our results are applied to suitable Galois connections between submodule lattices. Cyclically essential Galois connections unify semi-projective and semi-injective modules, while retractable Galois connections unify retractable and coretractable modules.

We develop a new perspective on principal bundles with connection as morphisms from the tangent bundle of the underlying manifold to a classifying dg-Lie groupoid. This groupoid can be identified with a lift of the inner homomorphisms groupoid arising in Ševera's differentiation procedure of Lie quasi-groupoids. Our new perspective readily extends to principal groupoid bundles, but requires an adjustment, an additional datum familiar from higher gauge theory. We show that for Lie groupoids, the additional adjustment data amounts to a Cartan connection. The resulting adjusted connections naturally provide a global formulation of the kinematical data of curved Yang-Mills-Higgs theories as described by Kotov-Strobl (arXiv:1510.07654) and Fischer (arXiv:2104.02175).

We define and study multiplicative connections in the tangent bundle of a Lie groupoid. Multiplicative connections are linear connections satisfying an appropriate compatibility with the groupoid structure. Our definition is natural in the sense that a linear connection on a Lie groupoid is multiplicative if and only if its torsion is a multiplicative tensor in the sense of Bursztyn-Drummond [5] and its geodesic spray is a multiplicative vector field. We identify the obstruction to the existence of a multiplicative connection. We also discuss the infinitesimal version of multiplicative connections in the tangent bundle, that we call infinitesimally multiplicative (IM) connections and we prove an integration theorem for IM connections. Finally, we present a few toy examples.

We consider smooth families of Lie groups (group bundles) and connections that are compatible with the group operation. We characterize the space of group connections on a group bundle as an affine space modeled over the vector space of $1$-forms with values cocycles in the Lie algebra bundle of the aforementioned group bundle. We show that group connections satisfy the Ambrose-Singer theorem and that group bundles can be seen as a particular case of associated bundles realizing group connections as associated connections. We give a construction of the Moduli space of group connections with fixed base and fiber, as an space of representations of the fundamental group of the base.

This article is an overview of the results obtained in recent years on symplectic connections. We present what is known about preferred connections (critical points of a variational principle). The class of Ricci-type connections (for which the curvature is entirely determined by the Ricci tensor) is described in detail, as well as its far reaching generalization to special connections. A twistorial construction shows a relation between Ricci-type connections and complex geometry. We give a construction of Ricci-flat symplectic connections. We end up by presenting, through an explicit example, an approach to noncommutative symplectic symmetric spaces.

We shall give a twisted Dirac structure on the space of irreducible connections on a SU(n)-bundle over a three-manifold, and give a family of twisted Dirac structures on the space of irreducible connections on the trivial SU(n)-bundle over a four-manifold. The twist is described by the Cartan 3-form on the space of connections. It vanishes over the subspace of flat connections. So the spaces of flat connections are endowed with ( non-twisted ) Dirac structures. The Dirac structure on the space of flat connections over the three-manifold is obtained as the boundary restriction of a corresponding Dirac structure over the four-manifold. We discuss also the action of the group of gauge transformations over these Dirac structures.

A gauge invariant notion of a strong connection is presented and characterized. It is then used to justify the way in which a global curvature form is defined. Strong connections are interpreted as those that are induced from the base space of a quantum bundle. Examples of both strong and non-strong connections are provided. In particular, such connections are constructed on a quantum deformation of the fibration $S^2 -> RP^2$. A certain class of strong $U_q(2)$-connections on a trivial quantum principal bundle is shown to be equivalent to the class of connections on a free module that are compatible with the q-dependent hermitian metric. A particular form of the Yang-Mills action on a trivial $U\sb q(2)$-bundle is investigated. It is proved to coincide with the Yang-Mills action constructed by A.Connes and M.Rieffel. Furthermore, it is shown that the moduli space of critical points of this action functional is independent of q.

Recently the present authors introduced a general class of Finsler connections which leads to a smart representation of connection theory in Finsler geometry and yields to a classification of Finsler connections into the three classes. Here the properties of one of these classes namely the Berwald-type connections which contains Berwald and Chern(Rund) connections as a special case is studied. It is proved among the other that the hv-curvature of these connections vanishes if and only if the Finsler space is a Berwald one. Some applications of this connection is discussed.

The notion of an odd quasi-connection on a supermanifold, which is loosely an affine connection that carries non-zero Grassmann parity, is examined. Their torsion and curvature are defined, however, in general, they are not tensors. A special class of such generalised connections, referred to as odd connections in this paper, have torsion and curvature tensors. Part of the structure is an odd involution of the tangent bundle of the supermanifold and this puts drastic restrictions on the supermanifolds that admit odd connections. In particular, they must have equal number of even and odd dimensions. Amongst other results, we show that an odd connection is defined, up to an odd tensor field of type $(1,2)$, by an affine connection and an odd endomorphism of the tangent bundle. Thus, the theory of odd connections and affine connections are not completely separate theories. As an example relevant to physics, it is shown that $N= 1$ super-Minkowski spacetime admits a natural odd connection.