Search

We study the concentration of the Neural Tangent Kernel (NTK) $K_\theta : \mathbb{R}^{m_0} \times \mathbb{R}^{m_0} \to \mathbb{R}^{m_l \times m_l}$ of $l$-layer Multilayer Perceptrons (MLPs) $N : \mathbb{R}^{m_0} \times \Theta \to \mathbb{R}^{m_l}$ equipped with activation functions $\phi(s) = a s + b \vert s \vert$ for some $a,b \in \mathbb{R}$ with the parameter $\theta \in \Theta$ being initialized at the Edge Of Chaos (EOC). Without relying on the gradient independence assumption that has only been shown to hold asymptotically in the infinitely wide limit, we prove that an approximate version of gradient independence holds at finite width. Showing that the NTK entries $K_\theta(x_{i_1},x_{i_2})$ for $i_1,i_2 \in [1:n]$ over a dataset $\{x_1,\cdots,x_n\} \subset \mathbb{R}^{m_0}$ concentrate simultaneously via maximal inequalities, we prove that the NTK matrix $K(\theta) = [\frac{1}{n} K_\theta(x_{i_1},x_{i_2}) : i_1,i_2 \in [1:n]] \in \mathbb{R}^{nm_l \times nm_l}$ concentrates around its infinitely wide limit $\overset{\scriptscriptstyle\infty}{K} \in \mathbb{R}^{nm_l \times nm_l}$ without the need for linear overparameterization. Our results imply that in order to accurately approximate the limit, hidden layer widths have to grow quadratically as $m_k = k^2 m$ for some $m \in \mathbb{N}+1$ for sufficient concentration. For such MLPs, we obtain the concentration bound $\mathbb{P}( \Vert K(\theta) - \overset{\scriptscriptstyle\infty}{K} \Vert \leq O((\Delta_\phi^{-2} + m_l^{\frac{1}{2}} l) \kappa_\phi^2 m^{-\frac{1}{2}})) \geq 1-O(m^{-1})$ modulo logarithmic terms, where we denoted $\Delta_\phi = \frac{b^2}{a^2+b^2}$ and $\kappa_\phi = \frac{\vert a \vert + \vert b \vert}{\sqrt{a^2 + b^2}}$. This reveals in particular that the absolute value ($\Delta_\phi=1$, $\kappa_\phi=1$) beats the ReLU ($\Delta_\phi=\frac{1}{2}$, $\kappa_\phi=\sqrt{2}$) in terms of the concentration of the NTK., Comment: 36 pages, 1 figure

We study the properties of the Neural Tangent Kernel (NTK) $\overset{\scriptscriptstyle\infty}{K} : \mathbb{R}^{m_0} \times \mathbb{R}^{m_0} \to \mathbb{R}^{m_l \times m_l}$ corresponding to infinitely wide $l$-layer Multilayer Perceptrons (MLPs) taking inputs from $\mathbb{R}^{m_0}$ to outputs in $\mathbb{R}^{m_l}$ equipped with activation functions $\phi(s) = a s + b \vert s \vert$ for some $a,b \in \mathbb{R}$ and initialized at the Edge Of Chaos (EOC). We find that the entries $\overset{\scriptscriptstyle\infty}{K}(x_1,x_2)$ can be approximated by the inverses of the cosine distances of the activations corresponding to $x_1$ and $x_2$ increasingly better as the depth $l$ increases. By quantifying these inverse cosine distances and the spectrum of the matrix containing them, we obtain tight spectral bounds for the NTK matrix $\overset{\scriptscriptstyle\infty}{K} = [\frac{1}{n} \overset{\scriptscriptstyle\infty}{K}(x_{i_1},x_{i_2}) : i_1, i_2 \in [1:n]]$ over a dataset $\{x_1,\cdots,x_n\} \subset \mathbb{R}^{m_0}$, transferred from the inverse cosine distance matrix via our approximation result. Our results show that $\Delta_\phi = \frac{b^2}{a^2+b^2}$ determines the rate at which the condition number of the NTK matrix converges to its limit as depth increases, implying in particular that the absolute value ($\Delta_\phi=1$) is better than the ReLU ($\Delta_\phi=\frac{1}{2}$) in this regard., Comment: 18 pages, 1 figure

Our goal is to provide simple and practical algorithms in higher-order Fourier analysis which are based on spectral decompositions of operators. We propose a general framework for such algorithms and provide a detailed analysis of the quadratic case. Our results reveal new spectral aspects of the theory underlying higher-order Fourier analysis. Along these lines, we prove new inverse and regularity theorems for the Gowers norms based on higher-order character decompositions. Using these results, we prove a spectral inverse theorem and a spectral regularity theorem in quadratic Fourier analysis., Comment: 71 pages

Reinforcement learning often requires extensive training data. Simulation-to-real transfer offers a promising approach to address this challenge in robotics. While differentiable simulators offer improved sample efficiency through exact gradients, they can be unstable in contact-rich environments and may lead to poor generalization. This paper introduces a novel approach integrating sharpness-aware optimization into gradient-based reinforcement learning algorithms. Our simulation results demonstrate that our method, tested on contact-rich environments, significantly enhances policy robustness to environmental variations and action perturbations while maintaining the sample efficiency of first-order methods. Specifically, our approach improves action noise tolerance compared to standard first-order methods and achieves generalization comparable to zeroth-order methods. This improvement stems from finding flatter minima in the loss landscape, associated with better generalization. Our work offers a promising solution to balance efficient learning and robust sim-to-real transfer in robotics, potentially bridging the gap between simulation and real-world performance., Comment: 9 pages, 6 figures

The anions C$_7$N$^-$ and C$_{10}$H$^-$ are the two longest of the linear (C,N)-bearing and (C,H)-bearing chains which have so far been detected in the Interstellar Medium. In order to glean information on their collision-induced rotational state-changing processes, we analyse the general features of new ab initio potentials describing the interaction of both linear anions with H$_2$, one of the most abundant partners in their ISM environment. We employ an artificial neural network fit of the reduced-dimensionality potential energy surface for C$_7$N$^-$...H$_2$ interaction and discuss in detail the spatial features in terms of multipolar radial coefficients. For the C$_{10}$H$^-$...H$_2$ interaction we use the initial grid of two dimensional raw points to generate by quadrature the Legendre expansion directly, further including the long-range terms as discussed in the main text. Quantum scattering calculations are employed to obtain rotationally inelastic cross sections, for collision energies in the range of 10$^{-4}$ to 400 cm$^{-1}$. From them we generate the corresponding inelastic rate coefficients as a function of temperature covering the range from 10 to 50 K. The results for the rate coefficients for the longest cyanopolyyne are compared with the earlier results obtained for the smaller terms of the same series, also in collision with H$_2$. We obtain that the inelastic rate coefficients for the long linear anions are all fairly large compared with the earlier systems. The consequences of such findings on their non-equilibrium rotational populations in interstellar environments are illustrated in our conclusions.

In this paper, we explore when the Betti numbers of the coordinate rings of a projective monomial curve and one of its affine charts are identical. Given an infinite field $k$ and a sequence of relatively prime integers $a_0 = 0 < a_1 < \cdots < a_n = d$, we consider the projective monomial curve $\mathcal{C}\subset\mathbb{P}_k^{\,n}$ of degree $d$ parametrically defined by $x_i = u^{a_i}v^{d-a_i}$ for all $i \in \{0,\ldots,n\}$ and its coordinate ring $k[\mathcal{C}]$. The curve $\mathcal{C}_1 \subset \mathbb A_k^n$ with parametric equations $x_i = t^{a_i}$ for $i \in \{1,\ldots,n\}$ is an affine chart of $\mathcal{C}$ and we denote by $k[\mathcal{C}_1]$ its coordinate ring. The main contribution of this paper is the introduction of a novel (Gr\"obner-free) combinatorial criterion that provides a sufficient condition for the equality of the Betti numbers of $k[\mathcal{C}]$ and $k[\mathcal{C}_1]$. Leveraging this criterion, we identify infinite families of projective curves satisfying this property. Also, we use our results to study the so-called shifted family of monomial curves, i.e., the family of curves associated to the sequences $j+a_1 < \cdots < j+a_n$ for different values of $j \in \mathbb N$. In this context, Vu proved that for large enough values of $j$, one has an equality between the Betti numbers of the corresponding affine and projective curves. Using our results, we improve Vu's upper bound for the least value of $j$ such that this occurs., Comment: To appear in Ric. Mat

Network telemetry based on data models is expected to become the standard mechanism for collecting operational data from network devices efficiently. But the wide variety of standard and proprietary data models along with the different implementations of telemetry protocols offered by network vendors, become a barrier when monitoring heterogeneous network infrastructures. To facilitate the integration and sharing of context information related to model-driven telemetry, this work proposes a semantic network inventory that integrates new information models specifically developed to capture context information in a vendor-agnostic fashion using current standards defined for context management. To automate the integration of this context information within the network inventory, a reference architecture is designed. Finally, a prototype of the solution is implemented and validated through a case study that illustrates how the network inventory can ease the operation of model-driven telemetry in multi-vendor networks., Comment: 8 pages, 5 figures

Water is a key ingredient for the emergence of life as we know it. Yet, its destruction and reformation in space remains unprobed in warm gas. Here, we detect the hydroxyl radical (OH) emission from a planet-forming disk exposed to external far-ultraviolet (FUV) radiation with the James Webb Space Telescope. The observations are confronted with the results of quantum dynamical calculations. The highly excited OH infrared rotational lines are the tell-tale signs of H2O destruction by FUV. The OH infrared ro-vibrational lines are attributed to chemical excitation via the key reaction O+H=OH+H which seeds the formation of water in the gas-phase. We infer that the equivalent of the Earth ocean's worth of water is destroyed per month and replenished. These results show that under warm and irradiated conditions water is destroyed and efficiently reformed via gas-phase reactions. This process, assisted by diffusive transport, could reduce the HDO/H2O ratio in the warm regions of planet-forming disks., Comment: Version submitted to Nature Astronomy

In recent work, Jamneshan, Shalom and Tao proved an inverse theorem for the Gowers $U^{k+1}$-norm on finite abelian groups of fixed torsion $m$, where the final correlating harmonic is a polynomial phase function of degree at most $C(k,m)$. They also posed a related central question, namely, whether the bound $C$ can be reduced to the optimal value $k$ for every $m$. We make progress on this question using nilspace theory. First we connect the question to the study of finite nilspaces whose structure groups have torsion $m$. Then we prove one of the main results of this paper: a primary decomposition theorem for finite nilspaces, extending the Sylow decomposition in group theory. Thus we give an analogue for nilspaces of an ergodic-theoretic Sylow decomposition in the aforementioned work of Jamneshan-Shalom-Tao. We deduce various consequences which illustrate the following general idea: the primary decomposition enables a reduction of higher-order Fourier analysis in the $m$-torsion setting to the case of abelian $p$-groups. These consequences include a positive answer to the question of Jamneshan-Shalom-Tao when $m$ is squarefree, and also a new relation between uniformity norms and certain generalized cut norms on products of abelian groups of coprime orders. Another main result in this paper is a positive answer to the above central question for the $U^3$-norm, proving that $C(2,m)=2$ for all $m$. Finally, we give a partial answer to the question for all $k$ and $m$, proving an inverse theorem involving extensions of polynomial phase functions which were introduced by the third-named author, known as projected phase polynomials of degree $k$. A notable aspect is that this inverse theorem implies that of Jamneshan-Shalom-Tao, while involving projected phase polynomials of degree $k$, which are genuine obstructions to having small $U^{k+1}$-norm., Comment: 26 pages. Comments from Jamneshan, Shalom and Tao incorporated

We report new results from quantum calculations of energy-transfer processes taking place in interstellar environments and involving two newly observed molecular species: C$_5$N$^-$ and C$_7$N$^-$ in collision with He atoms and the p-H$_2$ molecules. These species are part of the anionic molecular chains labeled as cyanopolyynes which have been observed over the years in molecule-rich Circumstellar Envelopes and in molecular clouds. In the present work, we first carry out new $ab$ $initio$ calculations for the C$_7$N$^-$ interaction potential with He atom and then obtain state-to-state rotationally inelastic cross sections and rate coefficients involving the same transitions which have been observed experimentally by emission in the interstellar medium (ISM) from both of these linear species. For the C$_5$N$^-$/He system we extend the calculations already published in our earlier work (see reference below) to compare more directly the two molecular anions. We extend further the quantum calculations by also computing in this work collision rate coefficients for the hydrogen molecule interacting with C5N$^-$, using our previously computed interaction potential. Additionally, we obtain the same rate coefficients for the C$_7$N$^-$/H$_2$ system by using a scaling procedure that makes use of the new C$_7$N$^-$/He rate coefficients, as discussed in detail in the present paper. Their significance in affecting internal state populations in ISM environments where the title anions have been found is analyzed by using the concept of critical density indicators. Finally, similarities and differences between such species and the comparative efficiency of their collision rate coefficients are discussed. These new calculations suggest that, at least for the case of these longer chains, the rotational populations could reach local thermal equilibrium conditions within their observational environments.

Assuming a delta pulse excitation, quantum wavepackets are propagated on the excited state manifold in the energy range from 3.4-5.0 eV for coronene and 2.4-3.5 eV for circumcoronene to study the time evolution of the states as well as their lifetimes. The full-dimensional (102 and 210 degrees of freedom for coronene and circumcoronene respectively) non-adiabatic dynamics simulated with the ML-MCTDH method on twelve coupled singlet electronic states show that the different absorption spectra are only due to electronic delocalisation effects that change the excited state energies, but the structural dynamics in both compounds are identical. Breathing and tilting motions drive the decay dynamics of the electronic states away from the Frank-Condon region independently of the size of the aromatic system. This promising result allows the use of coronene as a model system for the dynamics of larger polycyclic aromatic hydrocarbons (PAHs) and graphene one dimensional sheets or nanoflakes.

Building on previous work in the nilspace-theoretic approach to the study of Host-Kra factors of measure-preserving systems, we prove that every ergodic $\mathbb{F}_p^\omega$-system of order $k$ is a factor of an Abramov $\mathbb{F}_p^\omega$-system of order $k$. This answers a question of Jamneshan, Shalom and Tao., Comment: 15 pages

Compact finite-rank nilspaces have become central in the nilspace approach to higher-order Fourier analysis, notably through their role in a general form of the inverse theorem for the Gowers norms. This paper studies these nilspaces per se, and in connection with further refinements of this inverse theorem that have been conjectured recently. Our first main result states that every compact finite-rank nilspace is obtained by taking a free nilspace (a nilspace based on an abelian group of the form $\mathbb{Z}^{r}\times \mathbb{R}^s$) and quotienting this by a discrete group action of a specific type, describable in terms of polynomials. We call these group actions "higher-order lattice actions", as they generalize actions of lattices in $\mathbb{Z}^r\times \mathbb{R}^s$. The second main result (which relies on the first one) represents every compact finite-rank nilspace as a double-coset space $K\backslash G / \Gamma$ where $G$ is a nilpotent Lie group of a specific kind. Our third main result extends the aforementioned results to $k$-step compact nilspaces (not necessarily of finite rank), by representing any such nilspace as a quotient of infinite products of free nilspaces and also as double coset spaces $K\backslash G/\Gamma$ where $G$ is a degree-$k$ nilpotent pro-Lie group. These results open the study of compact nilspaces to areas more classical than nilspace theory, such as the theory of topological group actions. The results also require developing the theory of topological non-compact nilspaces, for which we provide groundwork in this paper. Applications include new inverse theorems for Gowers norms on any finite abelian group. These theorems are purely group theoretic in that the correlating harmonics are based on double-coset spaces. This yields progress towards the Jamneshan-Tao conjecture., Comment: 81 pages

Let $A=\{a_0,\ldots,a_{n-1}\}$ be a finite set of $n\geq 4$ non-negative relatively prime integers such that $0=a_0

A candidate explanation of the good empirical performance of deep neural networks is the implicit regularization effect of first order optimization methods. Inspired by this, we prove a convergence theorem for nonconvex composite optimization, and apply it to a general learning problem covering many machine learning applications, including supervised learning. We then present a deep multilayer perceptron model and prove that, when sufficiently wide, it $(i)$ leads to the convergence of gradient descent to a global optimum with a linear rate, $(ii)$ benefits from the implicit regularization effect of gradient descent, $(iii)$ is subject to novel bounds on the generalization error, $(iv)$ exhibits the lazy training phenomenon and $(v)$ enjoys learning rate transfer across different widths. The corresponding coefficients, such as the convergence rate, improve as width is further increased, and depend on the even order moments of the data generating distribution up to an order depending on the number of layers. The only non-mild assumption we make is the concentration of the smallest eigenvalue of the neural tangent kernel at initialization away from zero, which has been shown to hold for a number of less general models in contemporary works. We present empirical evidence supporting this assumption as well as our theoretical claims., Comment: 31 pages, 5 figures

For any standard Borel space $B$, let $\mathcal{P}(B)$ denote the space of Borel probability measures on $B$. In relation to a difficult problem of Aldous in exchangeability theory, and in connection with arithmetic combinatorics, Austin raised the question of describing the structure of affine-exchangeable probability measures on product spaces indexed by the vector space $\mathbb{F}_2^\omega$, i.e., the measures in $\mathcal{P}(B^{\mathbb{F}_2^\omega})$ that are invariant under the coordinate permutations on $B^{\mathbb{F}_2^\omega}$ induced by all affine automorphisms of $\mathbb{F}_2^{\omega}$. We answer this question by describing the extreme points of the space of such affine-exchangeable measures. We prove that there is a single structure underlying every such measure, namely, a random infinite-dimensional cube (sampled using Haar measure adapted to a specific filtration) on a group that is a countable power of the 2-adic integers. Indeed, every extreme affine-exchangeable measure in $\mathcal{P}(B^{\mathbb{F}_2^\omega})$ is obtained from a $\mathcal{P}(B)$-valued function on this group, by a vertex-wise composition with this random cube. The consequences of this result include a description of the convex set of affine-exchangeable measures in $\mathcal{P}(B^{\mathbb{F}_2^\omega})$ equipped with the vague topology (when $B$ is a compact metric space), showing that this convex set is a Bauer simplex. We also obtain a correspondence between affine-exchangeability and limits of convergent sequences of (compact-metric-space valued) functions on vector spaces $\mathbb{F}_2^n$ as $n\to\infty$. Via this correspondence, we establish the above-mentioned group as a general limit domain valid for any such sequence., Comment: 62 pages