Search

The dually flat structure of statistical manifolds can be derived in a non-parametric way from a particular case of affine space defined on a qualified set of probability measures. The statistically natural displacement mapping of the affine space depends on the notion of Fisher's score. The model space must be carefully defined if the state space is not finite. Among various options, we discuss how to use Orlicz-Sobolev spaces with Gaussian weight. Such a fully non-parametric set-up provides tools to discuss intrinsically infinite-dimensional evolution problems., Comment: Submitted review paper. arXiv admin note: text overlap with arXiv:2002.12871

In this chapter, we study Information Geometry from a particular non-parametric or functional point of view. The basic model is a probabilities subset usually specified by regularity conditions. For example, probability measures mutually absolutely continuous or probability densities with a given degree of smoothness. We construct a manifold structure by giving an atlas of charts as mappings from probabilities to a Banach space. The charts we use are quite peculiar in that we consider only instances where the transition mappings are affine. We chose a particular expression of the tangent and cotangent bundles in this affine setting., Comment: Submitted as a contributed chapter in a collective handbook. Added a section on the geometry of the unit sphere

This is a revised version of a tutorial lecture that I presented at the \`Ecole de Physique des Houches on July 26-31 2020. Topics include Non-parametric Information Geometry, the Statistical bundle, exponential Orlicz spaces, and Gaussian Orlicz-Sobolev spaces., Comment: Minor revision of version 1. Submitted

We provide an Information-Geometric formulation of Classical Mechanics on the Riemannian manifold of probability distributions, which is an affine manifold endowed with a dually-flat connection. In a non-parametric formalism, we consider the full set of positive probability functions on a finite sample space, and we provide a specific expression for the tangent and cotangent spaces over the statistical manifold, in terms of a Hilbert bundle structure that we call the Statistical Bundle. In this setting, we compute velocities and accelerations of a one-dimensional statistical model using the canonical dual pair of parallel transports and define a coherent formalism for Lagrangian and Hamiltonian mechanics on the bundle. Finally, in a series of examples, we show how our formalism provides a consistent framework for accelerated natural gradient dynamics on the probability simplex, paving the way for direct applications in optimization, game theory and neural networks., Comment: 39 pages, 5 figures; revised version published on IJGMMP

We derive bounds for the Orlicz norm of the deviation of a random variable defined on $\mathbb{R}^n$ from its Gaussian mean value. The random variables are assumed to be smooth and the bound itself depends on the Orlicz norm of the gradient. Applications to non-parametric Information Geometry are discussed., Comment: Accepted. Minor corrections

In Optimal Transport (OT) on a finite metric space, one defines a distance on the probability simplex that extends the distance on the ground space. The distance is the value of a Linear Programming (LP) problem on the set of non-negative-valued 2-way tables with assigned probability functions as margins. We apply to this case the methodology of moves from Algebraic Statistics (AS) and use it to derive a Monte Carlo Markov Chain (MCMC) solution algorithm., Comment: 25 pages; a proof has been added and some notational issues have been fixed

This set of notes is intended for a short course aiming to provide an (almost) self-contained and (almost) elementary introduction to the topic of Information Geometry (IG) of the probability simplex. Such a course can be considered an introduction to the original monograph by Amari and Nagaoka (2000), and to the recent monographs by Amari (2016} and by Ay et al. (2017). The focus is on a non-parametric approach, that is, I consider the geometry of the full probability simplex and compare the IG formalism with what is classically done in Statistical Physics.

Kantorovich distance (or 1-Wasserstein distance) on the probability simplex of a finite metric space is the value of a Linear Programming problem for which a closed-form expression is known in some cases. When the ground distance is defined by a graph, a few examples have already been studied. In the present paper, after re-deriving, with different tools, the result for trees, we prove that, for an arbitrary weighted graph, the K-distance is the minimum of the K-distances over all the spanning trees associated with the graph. We work in the dual LP-problem by using Arens-Eells norm associated with the metric space. Finally, we introduce new norms that are naturally related to $\ell_1$-embeddable distances and allows for a partial extension of our results to this new setting., Comment: This is a substantial revision of v4 of the paper. Title has been changed to reflect new developments

We discuss the Pistone-Sempi exponential manifold on the finite-dimensional Gaussian space. We consider the role of the entropy, the continuity of translations, Poincar\'e-type inequalities, the generalized differentiability of probability densities of the Gaussian space., Comment: Extendexd version of the paper presented at the IGAIA IV Conference 2016. Submitted for publication in the conference proceedings

The Wasserstein distance on multivariate non-degenerate Gaussian densities is a Riemannian distance. After reviewing the properties of the distance and the metric geodesic, we present an explicit form of the Riemannian metrics on positive-definite matrices and compute its tensor form with respect to the trace inner product. The tensor is a matrix which is the solution to a Lyapunov equation. We compute the explicit formula for the Riemannian exponential, the normal coordinates charts and the Riemannian gradient. Finally, the Levi-Civita covariant derivative is computed in matrix form together with the differential equation for the parallel transport. While all computations are given in matrix form, nonetheless we discuss also the use of a special moving frame., Comment: Revised version. Introduction extended. References added

In the present paper we consider modal propositional logic and look for the constraints that are imposed to the propositions of the special type $\Box a$ by the structure of the relevant finite Kripke frame. We translate the usual language of modal propositional logic in terms of notions of commutative algebra, namely polynomial rings, ideals, and bases of ideals. We use extensively the perspective obtained in previous works in Algebraic Statistics. We prove that the constraints on $\Box a$ can be derived through a binomial ideal containing a toric ideal and we give sufficient conditions under which the toric ideal fully describes the constraints.

We study the class on non-parametric deformed statistical models where the deformed exponential has linear growth at infinity and is sub-exponential at zero. This class generalizes the class introduced by N.J.~Newton. We discuss the convexity and regularity of the normalization operator, the form of the deformed statistical divergences and their convex duality, the properties of the escort densities, and the affine manifold structure of the statistical bundle., Comment: Revised submission to proceedings GSI 2017 Conference, Paris

We study the continuity of space translations on non-parametric exponential families based on the exponential Orlicz space with Gaussian reference density., Comment: Submitted to GSI 2017, Paris

We study optimal finite dimensional approximations of the generally infinite-dimensional Fokker-Planck-Kolmogorov (FPK) equation, finding the curve in a given finite-dimensional family that best approximates the exact solution evolution. For a first local approximation we assign a manifold structure to the family and a metric. We then project the vector field of the partial differential equation (PDE) onto the tangent space of the chosen family, thus obtaining an ordinary differential equation for the family parameter. A second global approximation will be based on projecting directly the exact solution from its infinite dimensional space to the chosen family using the nonlinear metric projection. This will result in matching expectations with respect to the exact and approximating densities for particular functions associated with the chosen family, but this will require knowledge of the exact solution of FPK. A first way around this is a localized version of the metric projection based on the assumed density approximation. While the localization will remove global optimality, we will show that the somewhat arbitrary assumed density approximation is equivalent to the mathematically rigorous vector field projection. More interestingly we study the case where the approximating family is defined based on a number of eigenfunctions of the exact equation. In this case we show that the local vector field projection provides also the globally optimal approximation in metric projection, and for some families this coincides with a Galerkin method., Comment: Extended revision of previous arXiv version with change of title. The previous version was entitled "Maximum likelihood eigenfunctions of the Fokker--Planck equation and Hellinger projection"

We propose a dimensionality reduction method for infinite-dimensional measure-valued evolution equations such as the Fokker-Planck partial differential equation or the Kushner-Stratonovich resp. Duncan-Mortensen-Zakai stochastic partial differential equations of nonlinear filtering, with potential applications to signal processing, quantitative finance, heat flows and quantum theory among many other areas. Our method is based on the projection coming from a duality argument built in the exponential statistical manifold structure developed by G. Pistone and co-authors. The choice of the finite dimensional manifold on which one should project the infinite dimensional equation is crucial, and we propose finite dimensional exponential and mixture families. This same problem had been studied, especially in the context of nonlinear filtering, by D. Brigo and co-authors but the $L^2$ structure on the space of square roots of densities or of densities themselves was used, without taking an infinite dimensional manifold environment space for the equation to be projected. Here we re-examine such works from the exponential statistical manifold point of view, which allows for a deeper geometric understanding of the manifold structures at play. We also show that the projection in the exponential manifold structure is consistent with the Fisher Rao metric and, in case of finite dimensional exponential families, with the assumed density approximation. Further, we show that if the sufficient statistics of the finite dimensional exponential family are chosen among the eigenfunctions of the backward diffusion operator then the statistical-manifold or Fisher-Rao projection provides the maximum likelihood estimator for the Fokker Planck equation solution. We finally try to clarify how the finite dimensional and infinite dimensional terminology for exponential and mixture spaces are related., Comment: Added maximum likelihood theorem and projection approximation analysis. Updated version to appear in: Nielsen, F., Critchley, F., & Dodson, K. (Eds), Computational Information Geometry for Image and Signal Processing, Springer Verlag, 2016

The basic Kriging's model assumes a Gaussian distribution with stationary mean and stationary variance. In such a setting, the joint distribution of the spatial process is characterized by the common variance and the correlation matrix or, equivalently, by the common variance and the variogram matrix. We discuss in in detail the option to actually use the variogram as a parameterization., Comment: This paper is the final revision of arXiv:1503.07686 [math.ST] with a new title

Information Geometry generalizes to infinite dimension by modeling the tangent space of the relevant manifold of probability densities with exponential Orlicz spaces. We review here several properties of the exponential manifold on a suitable set $\mathcal E$ of mutually absolutely continuous densities. We study in particular the fine properties of the Kullback-Liebler divergence in this context. We also show that this setting is well-suited for the study of the spatially homogeneous Boltzmann equation if $\mathcal E$ is a set of positive densities with finite relative entropy with respect to the Maxwell density. More precisely, we analyse the Boltzmann operator in the geometric setting from the point of its Maxwell's weak form as a composition of elementary operations in the exponential manifold, namely tensor product, conditioning, marginalization and we prove in a geometric way the basic facts i.e., the H-theorem. We also illustrate the robustness of our method by discussing, besides the Kullback-Leibler divergence, also the property of Hyv\"arinen divergence. This requires to generalise our approach to Orlicz-Sobolev spaces to include derivatives.%, Comment: 39 pages, 1 figure. Expanded version of a paper presente at the conference SigmaPhi 2014 Rhodes GR. Under revision for Entropy

The polarization measure is the probability that among 3 individuals chosen at random from a finite population exactly 2 come from the same class. This index is maximum at the midpoints of the edges of the probability simplex. We compute the gradient flow of this index that is the differential equation whose solutions are the curves of steepest ascent. Tools from Information Geometry are extensively used. In a time series, a comparison of the estimated velocity of variation with the direction of the gradient field should be a better index than the simple variation of the index., Comment: 22 pages with 6 figures and 1 table. Extended version of the poster presented at the SIS conference Cagliari 2014. The main sections (not the appendixes) have been submitted to the journal STATISTICA. Typos corrected in v2

Information Geometry has been used to inspire efficient algorithms for stochastic optimization, both in the combinatorial and the continuous case. We give an overview of the authors' research program and some specific contributions to the underlying theory., Comment: 7th IWS Conference, Rimini Italy

We review a nonparametric version of Amari's Information Geometry in which the set of positive probability densities on a given sample space is endowed with an atlas of charts to form a differentiable manifold modeled on Orlicz Banach spaces. This nonparametric setting is used to discuss the setting of typical problems in Machine Learning and Statistical Physics, such as relaxed optimization, Kullback-Leibler divergence, Boltzmann entropy, Boltzmann equation, Comment: 24 pages

The differential-geometric structure of the set of positive densities on a given measure space has raised the interest of many mathematicians after the discovery by C.R. Rao of the geometric meaning of the Fisher information. Most of the research is focused on parametric statistical models. In series of papers by author and coworkers a particular version of the nonparametric case has been discussed. It consists of a minimalistic structure modeled according the theory of exponential families: given a reference density other densities are represented by the centered log likelihood which is an element of an Orlicz space. This mappings give a system of charts of a Banach manifold. It has been observed that, while the construction is natural, the practical applicability is limited by the technical difficulty to deal with such a class of Banach spaces. It has been suggested recently to replace the exponential function with other functions with similar behavior but polynomial growth at infinity in order to obtain more tractable Banach spaces, e.g. Hilbert spaces. We give first a review of our theory with special emphasis on the specific issues of the infinite dimensional setting. In a second part we discuss two specific topics, differential equations and the metric connection. The position of this line of research with respect to other approaches is briefly discussed., Comment: Submitted for publication in the Proceedings od GSI2013 Aug 28-30 2013 Paris

We consider the classical problem of computing the expected value of a real function $f$ of the $d$-variate random variable $X$ using cubature formul\ae. We use in synergy tools from Commutative Algebra for cubature rul\ae, from elementary orthogonal polynomial theory and from Probability.

A deformed logarithm function called $q$-logarithm has received considerable attention by physicist after its introduction by C. Tsallis. J. Naudts has proposed a generalization called $\phi$-logarithm and he has derived the basic properties of $\phi$-exponential families. In this paper we study the related notion of marginal polytope in the case of a finite state space., Comment: Submitted

This is a review of current research in Markov chains as toric statistical models. Its content is mixture of background information, results from the relevant recent literature, new results, and work in progress., Comment: 26 pages. Submitted Oct 10, 2011

Limits of densities belonging to an exponential family appear in many applications, {e.g.} Gibbs models in Statistical Physics, relaxed combinatorial optimization, coding theory, critical likelihood computations, Bayes priors with singular support, random generation of factorial designs. We discuss the problem from the methodological point of view in the case of a finite state space. We prove two characterizations of the limit distributions, both based on a suitable description of the marginal polytope (convex hull of canonical statistics' values). First, the set of limit densities is equal to the set of conditional densities given a face of the marginal polytope. Second, in the lattice case there exists a parametric presentation, in monomial form, of the closure of the statistical model., Comment: 13 pages

For a Markov chain both the detailed balance condition and the cycle Kolmogorov condition are algebraic binomials. This remark suggests to study reversible Markov chains with the tool of Algebraic Statistics, such as toric statistical models. One of the results of this study in an algebraic parameterization of reversible Markov transitions and their invariant probability.

The joint use of counting functions, Hilbert basis and Markov basis allows to define a procedure to generate all the fractions that satisfy a given set of constraints in terms of orthogonality. The general case of mixed level designs, without restrictions on the number of levels of each factor (like primes or power of primes) is studied. This new methodology has been experimented on some significant classes of fractional factorial designs, including mixed level orthogonal arrays., Comment: 27 pages

We discuss the use of Kaniadakis' $\kappa$-exponential in the construction of a statistical manifold modelled on Lebesgue spaces of real random variables. Some algebraic features of the deformed exponential models are considered. A chart is defined for each strictly positive densities; every other strictly positive density in a suitable neighborhood of the reference probability is represented by the centered $\Kln$ likelihood

In the present paper we discuss problems concerning evolutions of densities related to Ito diffusions in the framework of the statistical exponential manifold. We develop a rigorous approach to the problem, and we particularize it to the orthogonal projection of the evolution of the density of a diffusion process onto a finite dimensional exponential manifold. It has been shown by D. Brigo (1996) that the projected evolution can always be interpreted as the evolution of the density of a different diffusion process. We give also a compactness result when the dimension of the exponential family increases, as a first step towards a convergence result to be investigated in the future. The infinite dimensional exponential manifold structure introduced by G. Pistone and C. Sempi is used and some examples are given.

Every fraction is a union of points, which are trivial regular fractions. To characterize non trivial decomposition, we derive a condition for the inclusion of a regular fraction as follows. Let $F = \sum_\alpha b_\alpha X^\alpha$ be the indicator polynomial of a generic fraction, see Fontana et al, JSPI 2000, 149-172. Regular fractions are characterized by $R = \frac 1l \sum_{\alpha \in \mathcal L} e_\alpha X^\alpha$, where $\alpha \mapsto e_\alpha$ is an group homeomorphism from $\mathcal L \subset \mathbb Z_2^d$ into $\{-1,+1\}$. The regular $R$ is a subset of the fraction $F$ if $FR = R$, which in turn is equivalent to $\sum_t F(t)R(t) = \sum_t R(t)$. If $\mathcal H = \{\alpha_1 >... \alpha_k\}$ is a generating set of $\mathcal L$, and $R = \frac1{2^k}(1 + e_1X^{\alpha_1}) ... (1 + e_kX^{\alpha_k})$, $e_j = \pm 1$, $j=1 ... k$, the inclusion condition in term of the $b_\alpha$'s is % \begin{equation}b_0 + e_1 b_{\alpha_1} + >... + e_1 ... e_k b_{\alpha_1 + ... + \alpha_k} = 1. \tag{*}\end{equation} % The last part of the paper will discuss some examples to investigate the practical applicability of the previous condition (*). This paper is an offspring of the Alcotra 158 EU research contract on the planning of sequential designs for sample surveys in tourism statistics., Comment: Presented by R. Fontana at the DAE 2007 Conference, The University of Memphis, November 2, 2007

In a general fractional factorial design, the $n$-levels of a factor are coded by the $n$-th roots of the unity. This device allows a full generalization to mixed-level designs of the theory of the polynomial indicator function which has already been introduced for two level designs by Fontana and the Authors (2000). the properties of orthogonal arrays and regular fractions are discussed.

In this paper we consider a Bayesian analysis of contingency tables allowing for the possibility that cells may have probability zero. In this sense we depart from standard log-linear modeling that implicitly assumes a positivity constraint. Our approach leads us to consider mixture models for contingency tables, where the components of the mixture, which we call model-instances, have distinct support. We rely on ideas from polynomial algebra in order to identify the various model instances. We also provide a method to assign prior probabilities to each instance of the model, as well as describing methods for constructing priors on the parameter space of each instance. We illustrate our methodology through a $5 \times 2$ table involving two structural zeros, as well as a zero count. The results we obtain show that our analysis may lead to conclusions that are substantively different from those that would obtain in a standard framework, wherein the possibility of zero-probability cells is not explicitly accounted for.

In this paper, we relate the problem of generating all 2-level orthogonal arrays of given dimension and force, i.e. elements in OA$(n,m)$, where $n$ is the number of factors and $m$ the force, to the solution of an Integer Programming problem involving rational convex cones. We do not restrict the number of points in the array, i.e. we admit any number of replications. This problem can be theoretically solved by means of Hilbert bases which form a finite generating set for all the elements in in the infinite set OA$(n,m)$. We discuss some examples which are explicitly solved with a software performing Hilbert bases computation.