Search

Polynomial neural networks have been implemented in a range of applications and present an advantageous framework for theoretical machine learning. A polynomial neural network of fixed architecture and activation degree gives an algebraic map from the network's weights to a set of polynomials. The image of this map is the space of functions representable by the network. Its Zariski closure is an affine variety known as a neurovariety. The dimension of a polynomial neural network's neurovariety provides a measure of its expressivity. In this work, we introduce the notion of the activation threshold of a network architecture which expresses when the dimension of a neurovariety achieves its theoretical maximum. In addition, we prove expressiveness results for polynomial neural networks with equi-width~architectures., Comment: 13 pages

In the last quarter of a century, algebraic statistics has established itself as an expanding field which uses multilinear algebra, commutative algebra, computational algebra, geometry, and combinatorics to tackle problems in mathematical statistics. These developments have found applications in a growing number of areas, including biology, neuroscience, economics, and social sciences. Naturally, new connections continue to be made with other areas of mathematics and statistics. This paper outlines three such connections: to statistical models used in educational testing, to a classification problem for a family of nonparametric regression models, and to phase transition phenomena under uniform sampling of contingency tables. We illustrate the motivating problems, each of which is for algebraic statistics a new direction, and demonstrate an enhancement of related methodologies., Comment: This research was performed while the authors were visiting the Institute for Mathematical and Statistical Innovation (IMSI), which is supported by the National Science Foundation (Grant No. DMS-1929348). We participated in the long program "Algebraic Statistics and Our Changing World"

We survey recent applications of topology and singularity theory in the study of the algebraic complexity of concrete optimization problems in applied algebraic geometry and algebraic statistics., Comment: comments are very welcome!

The linear optimization degree gives an algebraic measure of complexity of optimizing a linear objective function over an algebraic model. Geometrically, it can be interpreted as the degree of a projection map on the {affine} conormal variety. Fixing an affine variety, our first result shows that the geometry of {this} conormal variety, expressed in terms of bidegrees, completely determines the Chern-Mather classes of the given variety. We also show that these bidegrees coincide with the linear optimization degrees of generic affine sections., Comment: v2: A new Section 6 and several references are added, v3: A new Subsection 1.1 is added to clarify relations to other works

We implement a real polyhedral homotopy method using three functions. The first function provides a certificate that our real polyhedral homotopy is applicable to a given system; the second function generates binomial systems for a start system; the third function outputs target solutions from the start system obtained by the second function. This work realizes the theoretical contributions in \cite{ergur2019polyhedral} as easy to use functions, allowing for further investigation into real homotopy algorithms., Comment: 13 pages, 1 figure, Version to appear in Journal of Software for Algebra and Geometry

We develop a new method that improves the efficiency of equation-by-equation algorithms for solving polynomial systems. Our method is based on a novel geometric construction, and reduces the total number of homotopy paths that must be numerically continued. These improvements may be applied to the basic algorithms of numerical algebraic geometry in the settings of both projective and multiprojective varieties. Our computational experiments demonstrate significant savings obtained on several benchmark systems. We also present an extended case study on maximum likelihood estimation for rank-constrained symmetric $n\times n$ matrices, in which multiprojective $u$-generation allows us to complete the list of ML degrees for $n\le 6.$, Comment: 25 pages

Using compactifications in the logarithmic cotangent bundle, we obtain a formula for the Chern classes of the pushforward of Lagrangian cycles under an open embedding with normal crossing complement. This generalizes earlier results of Aluffi and Wu-Zhou. The first application of our formula is a geometric description of Chern-Mather classes of an arbitrary very affine variety, generalizing earlier results of Huh which held under the smooth and schon assumptions. As the second application, we confirm an involution formula relating sectional maximum likelihood (ML) degrees and ML bidegrees, which was conjectured by Huh and Sturmfels in 2013.

The aim of the present work is to introduce a lens whose faces are a conical surface and a spherical surface. We illuminate this lens by a plane wavefront and its associated refracted wavefronts, light rays and caustic are computed. We find that the caustic has two branches. The first is constituted by two segments of a line, one part of this caustic is real and the other one virtual. The second branch of the caustic is a two-dimensional surface with a singularity of the cusp ridge type. It is important to remark that the two branches of the caustic are disconnected. Because of this property we believe that using this optical element one could generate a scalar optical accelerating beam in the region where the caustic is a two-dimensional surface of revolution, and at the same time a scalar optical beam with similar properties to the Bessel beam of zero order in the region were the real caustic is a segment of a line along the optical axis., Comment: 19 pages, 7 figures

The method of moments is a classical statistical technique for density estimation that solves a system of moment equations to estimate the parameters of an unknown distribution. A fundamental question critical to understanding identifiability asks how many moment equations are needed to get finitely many solutions and how many solutions there are. We answer this question for classes of Gaussian mixture models using the tools of polyhedral geometry. In addition, we show that a generic Gaussian $k$-mixture model is identifiable from its first $3k+2$ moments. Using these results, we present a homotopy algorithm that performs parameter recovery for high dimensional Gaussian mixture models where the number of paths tracked scales linearly in the dimension., Comment: 39 pages

We consider statistical models arising from the common set of solutions to a sparse polynomial system with general coefficients. The maximum likelihood degree counts the number of critical points of the likelihood function restricted to the model. We prove the maximum likelihood degree of a sparse polynomial system is determined by its Newton polytopes and equals the mixed volume of a related Lagrange system of equations., Comment: 13 pages

In this article we provide examples, methods and algorithms to determine conditions on the parameters of certain type of parametric optimization problems, such that among the resulting local minima and maxima there is at least one which satisfies given polynomial conditions (for example it is singular or symmetric).

The Macaulay2 package DecomposableSparseSystems implements methods for studying and numerically solving decomposable sparse polynomial systems. We describe the structure of decomposable sparse systems and explain how the methods in this package may be used to exploit this structure, with examples., Comment: 7 pages, software available at https://www.math.tamu.edu/~thomasjyahl/research/DSS/DSSsite.html

Let $X$ be a complex affine variety in $\mathbb{C}^N$, and let $f:\mathbb{C}^N\to \mathbb{C}$ be a polynomial function whose restriction to $X$ is nonconstant. For $g:\mathbb{C}^N \to \mathbb{C}$ a general linear function, we study the limiting behavior of the critical points of the one-parameter family of $f_t: =f-tg$ as $t\to 0$. Our main result gives an expression of this limit in terms of critical sets of the restrictions of $g$ to the singular strata of $(X,f)$. We apply this result in the context of optimization problems. For example, we consider nearest point problems (e.g., Euclidean distance degrees) for affine varieties and a possibly nongeneric data point., Comment: 24 pages

Amendola et al. proposed a method for solving systems of polynomial equations lying in a family which exploits a recursive decomposition into smaller systems. A family of systems admits such a decomposition if and only if the corresponding Galois group is imprimitive. When the Galois group is imprimitive we consider the problem of computing an explicit decomposition. A consequence of Esterov's classification of sparse polynomial systems with imprimitive Galois groups is that this decomposition is obtained by inspection. This leads to a recursive algorithm to solve decomposable sparse systems, which we present and give evidence for its efficiency., Comment: 20 pages

We demonstrate our implementation of a continuation method as described in \cite{HR2015} for solving polynomials systems. Given a sequence of (multi)homogeneous polynomials, the software "multiregeneration" outputs the respective (multi)degree in a wide range of cases and partial multidegree in all others. We use Python for the file processing, while Bertini is needed for the continuation. Moreover, parallelization options and several strategies for solving structured polynomial systems are available.

A numerical description of an algebraic subvariety of projective space is given by a general linear section, called a witness set. For a subvariety of a product of projective spaces (a multiprojective variety), the corresponding numerical description is given by a witness collection, whose structure is more involved. We build on recent work to develop a toolkit for the numerical manipulation of multiprojective varieties that operates on witness collections, and use this toolkit in an algorithm for numerical irreducible decomposition of multiprojective varieties. The toolkit and decomposition algorithm are illustrated throughout in a series of examples., Comment: 28 pages

Two well studied invariants of a complex projective variety are the unit Euclidean distance degree and the generic Euclidean distance degree. These numbers give a measure of the algebraic complexity for "nearest" point problems of the algebraic variety. It is well known that the latter is an upper bound for the former. While this bound may be tight, many varieties appearing in optimization, engineering, statistics, and data science, have a significant gap between these two numbers. We call this difference the defect of the ED degree of an algebraic variety. In this paper we compute this defect by classical techniques in Singularity Theory, thereby deriving a new method for computing ED degrees of smooth complex projective varieties., Comment: 14 pages

We give a positive answer to a conjecture of Aluffi-Harris on the computation of the Euclidean distance degree of a possibly singular projective variety in terms of the local Euler obstruction function., Comment: 10 pages. Key words: Euclidean distance degree, Euler characteristic, local Euler obstruction function

A finite unit norm tight frame is a collection of $r$ vectors in $\mathbb{R}^n$ that generalizes the notion of orthonormal bases. The affine finite unit norm tight frame variety is the Zariski closure of the set of finite unit norm tight frames. Determining the fiber of a projection of this variety onto a set of coordinates is called the algebraic finite unit norm tight frame completion problem. Our techniques involve the algebraic matroid of an algebraic variety, which encodes the dimensions of fibers of coordinate projections. This work characterizes the bases of the algebraic matroid underlying the variety of finite unit norm tight frames in $\mathbb{R}^3$. Partial results towards similar characterizations for finite unit norm tight frames in $\mathbb{R}^n$ with $n \ge 4$ are also given. We provide a method to bound the degree of the projections based off of combinatorial~data., Comment: 15 pages

The Euclidean distance degree of an algebraic variety is a well-studied topic in applied algebra and geometry. It has direct applications in geometric modeling, computer vision, and statistics. We use non-proper Morse theory to give a topological interpretation of the Euclidean distance degree of an affine variety in terms of Euler characteristics. As a concrete application, we solve the open problem in computer vision of determining the Euclidean distance degree of the affine multiview variety., Comment: Euclidean distance degree, multiview variety, triangulation problem, non-proper Morse theory, Euler-Poincare characteristic, local Euler obstruction

We develop a new homotopy method for solving multiparameter eigenvalue problems (MEPs) called the fiber product homotopy method. For a $k$-parameter eigenvalue problem with matrices of sizes $n_1,\dots ,n_k = O(n)$, fiber product homotopy method requires deformation of $O(1)$ linear equations, while existing homotopy methods for MEPs require $O(n)$ nonlinear equations. We show that the fiber product homotopy method theoretically finds all eigenpairs of an MEP with probability one. It is especially well-suited for dimension-deficient singular MEPs, a weakness of all other existing methods, as the fiber product homotopy method is provably convergent with probability one for such problems as well, a fact borne out by numerical experiments. More generally, our numerical experiments indicate that the fiber product homotopy method significantly outperforms the standard Delta method in terms of accuracy, with consistent backward errors on the order of $10^{-16}$, even for dimension-deficient singular problems, and without any use of extended precision. In terms of speed, it significantly outperforms previous homotopy-based methods on all problems and outperforms the Delta method on larger problems, and is also highly parallelizable. We show that the fiber product MEP that we solve in the fiber product homotopy method, although mathematically equivalent to a standard MEP, is typically a much better conditioned problem., Comment: 27 pages, 8 figures

Macpherson defined Chern-Schwartz-Macpherson (CSM) classes by introducing the (local) Euler obstruction function, which is an integer valued function on the variety that is constant on each stratum of a Whitney stratification of an algebraic variety. By understanding the Euler obstruction function, one gains insights about a singular algebraic variety. It was recently shown by the author and B. Wang, how to compute these functions using maximum likelihood degrees. This paper discusses a symbolic and a numerical implementation of algorithms to compute the Euler obstruction at a point. Macaulay2 and Bertini are used in the implementations., Comment: 8 pages

In this article, we give a numerical algorithm to compute braid groups of curves, hyperplane arrangements, and parameterized system of polynomial equations. Our main result is an algorithm that determines the cross-locus and the generators of the braid group., Comment: 15 pages

Quadratic eigenvalue problems (QEP) and more generally polynomial eigenvalue problems (PEP) are among the most common types of nonlinear eigenvalue problems. Both problems, especially the QEP, have extensive applications. A typical approach to solve QEP and PEP is to use a linearization method to reformulate the problem as a higher dimensional linear eigenvalue problem. In this article, we use homotopy continuation to solve these nonlinear eigenvalue problems without passing to higher dimensions. Our main contribution is to show that our method produces substantially more accurate results, and finds all eigenvalues with a certificate of correctness via Smale's $\alpha$-theory. To explain the superior accuracy, we show that the nonlinear eigenvalue problem we solve is better conditioned than its reformulated linear eigenvalue problem, and our homotopy continuation algorithm is more stable than QZ algorithm - theoretical findings that are borne out by our numerical experiments. Our studies provide yet another illustration of the dictum in numerical analysis that, for reasons of conditioning and stability, it is sometimes better to solve a nonlinear problem directly even when it could be transformed into a linear problem with the same solution mathematically.

We give a numerical algorithm computing Euler obstruction functions using maximum likelihood degrees. The maximum likelihood degree is a well-studied property of a variety in algebraic statistics and computational algebraic geometry. In this article we use this degree to give a new way to compute Euler obstruction functions. We define the maximum likelihood obstruction function and show how it coincides with the Euler obstruction function. With this insight, we are able to bring new tools of computational algebraic geometry to study Euler obstruction functions., Comment: 11 pages

We study the maximum likelihood degree (ML degree) of toric varieties, known as discrete exponential models in statistics. By introducing scaling coefficients to the monomial parameterization of the toric variety, one can change the ML degree. We show that the ML degree is equal to the degree of the toric variety for generic scalings, while it drops if and only if the scaling vector is in the locus of the principal $A$-determinant. We also illustrate how to compute the ML estimate of a toric variety numerically via homotopy continuation from a scaled toric variety with low ML degree. Throughout, we include examples motivated by algebraic geometry and statistics. We compute the ML degree of rational normal scrolls and a large class of Veronese-type varieties. In addition, we investigate the ML degree of scaled Segre varieties, hierarchical loglinear models, and graphical models., Comment: 21 pages, 5 figures

The Galois group of a parameterized polynomial system of equations encodes the structure of the solutions. This monodromy group acts on the set of solutions for a general set of parameters, that is, on the fiber of a projection from the incidence variety of parameters and solutions onto the space of parameters. When this projection is decomposable, the Galois group is imprimitive, and we show that the structure can be exploited for computational improvements. Furthermore, we develop a new algorithm for solving these systems based on a suitable trace test. We illustrate our method on examples in statistics, kinematics, and benchmark problems in computational algebra. In particular, we resolve a conjecture on the number of solutions of the moment system associated to a mixture of Gaussian distributions., Comment: 18 pages, 2 figures, 2 tables

The trace test in numerical algebraic geometry verifies the completeness of a witness set of an irreducible variety in affine or projective space. We give a brief derivation of the trace test and then consider it for subvarieties of products of projective spaces using multihomogeneous witness sets. We show how a dimension reduction leads to a practical trace test in this case involving a curve in a low-dimensional affine space., Comment: 12 pages, 6 figures

The Galois/monodromy group of a family of geometric problems or equations is a subtle invariant that encodes the structure of the solutions. Computing monodromy permutations using numerical algebraic geometry gives information about the group, but can only determine it when it is the full symmetric group. We give numerical methods to compute the Galois group and study it when it is not the full symmetric group. One algorithm computes generators while the other gives information on its structure as a permutation group. We illustrate these algorithms with examples using a Macaulay2 package we are developing that relies upon Bertini to perform monodromy computations.

An algebraic approach to the maximum likelihood estimation problem is to solve a very structured parameterized polynomial system called likelihood equations that have finitely many complex (real or non-real) solutions. The only solutions that are statistically meaningful are the real solutions with positive coordinates. In order to classify the parameters (data) according to the number of real/positive solutions, we study how to efficiently compute the discriminants, say data-discriminants (DD), of the likelihood equations. We develop a probabilistic algorithm with three different strategies for computing DDs. Our implemented probabilistic algorithm based on Maple and FGb is more efficient than our previous version presented in ISSAC2015, and is also more efficient than the standard elimination for larger benchmarks. By applying RAGlib to a DD we compute, we give the real root classification of 3 by 3 symmetric matrix model., Comment: 4 tables. arXiv admin note: substantial text overlap with arXiv:1501.00334. authors' note: The paper the extended and improved version of our paper presented in ISSAC2015 (arXiv:1501.00334) and the paper is accepted by Journal of Symbolic Computation Special Issue of ISSAC2015

For general data, the number of complex solutions to the likelihood equations is constant and this number is called the (maximum likelihood) ML-degree of the model. In this article, we describe the special locus of data for which the likelihood equations have a solution in the model's singular locus.

In the field of numerical algebraic geometry, positive-dimensional solution sets of systems of polynomial equations are described by witness sets. In this paper, we define multiprojective witness sets which encode the multidegree information of an irreducible multiprojective variety. Our main results generalize the regeneration solving procedure, a trace test, and numerical irreducible decomposition to the multiprojective case. Examples are included to demonstrate this new approach.

The maximum likelihood degree (ML degree) measures the algebraic complexity of a fundamental optimization problem in statistics: maximum likelihood estimation. In this problem, one maximizes the likelihood function over a statistical model. The ML degree of a model is an upper bound to the number of local extrema of the likelihood function and can be expressed as a weighted sum of Euler characteristics. The independence model (i.e. rank one matrices over the probability simplex) is well known to have an ML degree of one, meaning their is a unique local maxima of the likelihood function. However, for mixtures of independence models (i.e. rank two matrices over the probability simplex), it was an open question as to how the ML degree behaved. In this paper, we use Euler characteristics to prove an outstanding conjecture by Hauenstein, the first author, and Sturmfels; we give recursions and closed form expressions for the ML degree of mixtures of independence models.

In many areas of applied mathematics and statistics, it is a fundamental problem to find the best representative of a model by optimizing an objective function. This can be done by determining critical points of the objective function restricted to the model. We compile ideas arising from numerical algebraic geometry to compute the critical points of an objective function. Our method consists of using numerical homotopy continuation and a monodromy action on the total critical space to compute all of the complex critical points of an objective function. To illustrate the relevance of our method, we apply it to the Euclidean distance function to compute ED-degrees and the likelihood function to compute maximum likelihood degrees., Comment: 14 pages

Maximum likelihood estimation (MLE) is a fundamental computational problem in statistics. The problem is to maximize the likelihood function with respect to given data on a statistical model. An algebraic approach to this problem is to solve a very structured parameterized polynomial system called likelihood equations. For general choices of data, the number of complex solutions to the likelihood equations is finite and called the ML-degree of the model. The only solutions to the likelihood equations that are statistically meaningful are the real/positive solutions. However, the number of real/positive solutions is not characterized by the ML-degree. We use discriminants to classify data according to the number of real/positive solutions of the likelihood equations. We call these discriminants data-discriminants (DD). We develop a probabilistic algorithm for computing DDs. Experimental results show that, for the benchmarks we have tried, the probabilistic algorithm is more efficient than the standard elimination algorithm. Based on the computational results, we discuss the real root classification problem for the 3 by 3 symmetric matrix~model., Comment: 2 tables

Maximum likelihood estimation (MLE) is a fundamental computational problem in statistics. In this paper, MLE for statistical models with discrete data is studied from an algebraic statistics viewpoint. A reformulation of the MLE problem in terms of dual varieties and conormal varieties will be given. With this description, the dual likelihood equations and the dual MLE problem are defined. We show that solving the dual MLE problem yields solutions to the MLE problem, so we can solve the MLE problem without ever determining the defining equations of the model.

Given a statistical model, the maximum likelihood degree is the number of complex solutions to the likelihood equations for generic data. We consider discrete algebraic statistical models and study the solutions to the likelihood equations when the data contain zeros and are no longer generic. Focusing on sampling and model zeros, we show that, in these cases, the solutions to the likelihood equations are contained in a previously studied variety, the likelihood correspondence. The number of these solutions give a lower bound on the ML degree, and the problem of finding critical points to the likelihood function can be partitioned into smaller and computationally easier problems involving sampling and model zeros. We use this technique to compute a lower bound on the ML degree for $2 \times 2 \times 2 \times 2$ tensors of border rank $\leq 2$ and $3 \times n$ tables of rank $\leq 2$ for $n=11, 12, 13, 14$, the first four values of $n$ for which the ML degree was previously unknown.

Numerical algebraic geometry is the field of computational mathematics concerning the numerical solution of polynomial systems of equations. Bertini, a popular software package for computational applications of this field, includes implementations of a variety of algorithms based on polynomial homotopy continuation. The Macaulay2 package Bertini.m2 provides an interface to Bertini, making it possible to access the core run modes of Bertini in Macaulay2. With these run modes, users can find approximate solutions to zero-dimensional systems and positive-dimensional systems, test numerically whether a point lies on a variety, sample numerically from a variety, and perform parameter homotopy runs.