Search

Consider the Monge-Kantorovich optimal transport problem where the cost function is given by a Bregman divergence. The associated transport cost, which we call the Bregman-Wasserstein divergence, presents a natural asymmetric extension of the squared $2$-Wasserstein metric and has recently found applications in statistics and machine learning. On the other hand, Bregman divergence is a fundamental object in information geometry and induces a dually flat geometry on the underlying manifold. Using the Bregman-Wasserstein divergence, we lift this dualistic geometry to the space of probability measures, thus extending Otto's weak Riemannian structure of the Wasserstein space to statistical manifolds. We do so by generalizing Lott's formal geometric computations on the Wasserstein space. In particular, we define primal and dual connections on the space of probability measures and show that they are conjugate with respect to Otto's metric. We also define primal and dual displacement interpolations which satisfy the corresponding geodesic equations. As applications, we study displacement convexity and the Bregman-Wasserstein barycenter., Comment: 46 pages, 3 figures

Spaces of convex and concave functions appear naturally in theory and applications. For example, convex regression and log-concave density estimation are important topics in nonparametric statistics. In stochastic portfolio theory, concave functions on the unit simplex measure the concentration of capital, and their gradient maps define novel investment strategies. The gradient maps may also be regarded as optimal transport maps on the simplex. In this paper we construct and study probability measures supported on spaces of concave functions. These measures may serve as prior distributions in Bayesian statistics and Cover's universal portfolio, and induce distribution-valued random variables via optimal transport. The random concave functions are constructed on the unit simplex by taking a suitably scaled (mollified, or soft) minimum of random hyperplanes. Depending on the regime of the parameters, we show that as the number of hyperplanes tends to infinity there are several possible limiting behaviors. In particular, there is a transition from a deterministic almost sure limit to a non-trivial limiting distribution that can be characterized using convex duality and Poisson point processes., Comment: 42 pages, 8 figures. Substantially revised. To appear in The Annals of Applied Probability

Convex PCA, which was introduced by Bigot et al., is a dimension reduction methodology for data with values in a convex subset of a Hilbert space. This setting arises naturally in many applications, including distributional data in the Wasserstein space of an interval, and ranked compositional data under the Aitchison geometry. Our contribution in this paper is threefold. First, we present several new theoretical results including consistency as well as continuity and differentiability of the objective function in the finite dimensional case. Second, we develop a numerical implementation of finite dimensional convex PCA when the convex set is polyhedral, and show that this provides a natural approximation of Wasserstein geodesic PCA. Third, we illustrate our results with two financial applications, namely distributions of stock returns ranked by size and the capital distribution curve, both of which are of independent interest in stochastic portfolio theory., Comment: 40 pages, 9 figures

Tsallis and R\'{e}nyi entropies, which are monotone transformations of each other, are deformations of the celebrated Shannon entropy. Maximization of these deformed entropies, under suitable constraints, leads to the $q$-exponential family which has applications in non-extensive statistical physics, information theory and statistics. In previous information-geometric studies, the $q$-exponential family was analyzed using classical convex duality and Bregman divergence. In this paper, we show that a generalized $\lambda$-duality, where $\lambda = 1 - q$ is the constant information-geometric curvature, leads to a generalized exponential family which is essentially equivalent to the $q$-exponential family and has deep connections with R\'{e}nyi entropy and optimal transport. Using this generalized convex duality and its associated logarithmic divergence, we show that our $\lambda$-exponential family satisfies properties that parallel and generalize those of the exponential family. Under our framework, the R\'{e}nyi entropy and divergence arise naturally, and we give a new proof of the Tsallis/R\'{e}nyi entropy maximizing property of the $q$-exponential family. We also introduce a $\lambda$-mixture family which may be regarded as the dual of the $\lambda$-exponential family, and connect it with other mixture-type families. Finally, we discuss a duality between the $\lambda$-exponential family and the $\lambda$-logarithmic divergence, and study its statistical consequences., Comment: 41 pages, 7 figures. Revised

In information geometry, generalized exponential families and statistical manifolds with curvature are under active investigation in recent years. In this paper we consider the statistical manifold induced by a logarithmic $L^{(\alpha)}$-divergence which generalizes the Bregman divergence. It is known that such a manifold is dually projectively flat with constant negative sectional curvature, and is closely related to the $\mathcal{F}^{(\alpha)}$-family, a generalized exponential family introduced by the second author. Our main result constructs a dual foliation of the statistical manifold, i.e., an orthogonal decomposition consisting of primal and dual autoparallel submanifolds. This decomposition, which can be naturally interpreted in terms of primal and dual projections with respect to the logarithmic divergence, extends the dual foliation of a dually flat manifold studied by Amari. As an application, we formulate a new $L^{(\alpha)}$-PCA problem which generalizes the exponential family PCA., Comment: 9 pages, 2 figures. To appear in GSI2021

Tsallis and R��nyi entropies, which are monotone transformations of each other, are deformations of the celebrated Shannon entropy. Maximization of these deformed entropies, under suitable constraints, leads to the $q$-exponential family which has applications in non-extensive statistical physics, information theory and statistics. In previous information-geometric studies, the $q$-exponential family was analyzed using classical convex duality and Bregman divergence. In this paper, we show that a generalized $��$-duality, where $��= 1 - q$ is the constant information-geometric curvature, leads to a generalized exponential family which is essentially equivalent to the $q$-exponential family and has deep connections with R��nyi entropy and optimal transport. Using this generalized convex duality and its associated logarithmic divergence, we show that our $��$-exponential family satisfies properties that parallel and generalize those of the exponential family. Under our framework, the R��nyi entropy and divergence arise naturally, and we give a new proof of the Tsallis/R��nyi entropy maximizing property of the $q$-exponential family. We also introduce a $��$-mixture family which may be regarded as the dual of the $��$-exponential family, and connect it with other mixture-type families. Finally, we discuss a duality between the $��$-exponential family and the $��$-logarithmic divergence, and study its statistical consequences., 41 pages, 7 figures. Revised

The adjoint sensitivity method scalably computes gradients of solutions to ordinary differential equations. We generalize this method to stochastic differential equations, allowing time-efficient and constant-memory computation of gradients with high-order adaptive solvers. Specifically, we derive a stochastic differential equation whose solution is the gradient, a memory-efficient algorithm for caching noise, and conditions under which numerical solutions converge. In addition, we combine our method with gradient-based stochastic variational inference for latent stochastic differential equations. We use our method to fit stochastic dynamics defined by neural networks, achieving competitive performance on a 50-dimensional motion capture dataset., Comment: AISTATS 2020; 25 pages, 6 figures in main text; clarify notation in appendix

We study the logarithmic $L^{(\alpha)}$-divergence which extrapolates the Bregman divergence and corresponds to solutions to novel optimal transport problems. We show that this logarithmic divergence is equivalent to a conformal transformation of the Bregman divergence, and, via an explicit affine immersion, is equivalent to Kurose's geometric divergence. In particular, the $L^{(\alpha)}$-divergence is a canonical divergence of a statistical manifold with constant sectional curvature $-\alpha$. For such a manifold, we give a geometric interpretation of its sectional curvature in terms of how the divergence between a pair of primal and dual geodesics differ from the dually flat case. Further results can be found in our follow-up paper [27] which uncovers a novel relation between optimal transport and information geometry., Comment: 10 pages, International Conference on Geometric Science of Information. Springer, 2019. arXiv admin note: text overlap with arXiv:1906.00030

Optimal transport and information geometry both study geometric structures on spaces of probability distributions. Optimal transport characterizes the cost-minimizing movement from one distribution to another, while information geometry originates from coordinate-invariant properties of statistical inference. Their connections and applications in statistics and machine learning have started to gain more attention. In this paper we give a new differential geometric connection between the two fields. Namely, the pseudo-Riemannian framework of Kim and McCann, a geometric perspective on the fundamental Ma-Trudinger-Wang (MTW) condition in the regularity theory of optimal transport maps, encodes the dualistic structure of statistical manifold. This general relation is described using the natural framework of $c$-divergence, a divergence defined by an optimal transport map. As a by-product, we obtain a new information-geometric interpretation of the MTW tensor. This connection sheds light on old and new aspects of information geometry. The dually flat geometry of Bregman divergence corresponds to the quadratic cost and the pseudo-Euclidean space, and the $L^{(\alpha)}$-divergence introduced by Pal and the first author has constant sectional curvature in a sense to be made precise. In these cases we give a geometric interpretation of the information-geometric curvature in terms of the divergence between a primal-dual pair of geodesics., Comment: 28 pages, 2 figures. Substantially revised. [Originally titled "Optimal transport and information geometry.]

We consider an optimal transport problem on the unit simplex whose solutions are given by gradients of exponentially concave functions and prove two main results. First, we show that the optimal transport is the large deviation limit of a particle system of Dirichlet processes transporting one probability measure on the unit simplex to another by coordinatewise multiplication and normalizing. The structure of our Lagrangian and the appearance of the Dirichlet process relate our problem closely to the entropic measure on the Wasserstein space as defined by von-Renesse and Sturm in the context of Wasserstein diffusion. The limiting procedure is a triangular limit where we allow simultaneously the number of particles to grow to infinity while the `noise' tends to zero. The method, which generalizes easily to many other cost functions, including the squared Euclidean distance, provides a novel combination of the Schr\"odinger problem approach due to C. L\'eonard and the related Brownian particle systems by Adams et al.which does not require gamma convergence. Second, we analyze the behavior of entropy along the paths of transport. The reference measure on the simplex is taken to be the Dirichlet measure with all zero parameters which relates to the finite-dimensional distributions of the entropic measure. The interpolating curves are not the usual McCann lines. Nevertheless we show that entropy plus a multiple of the transport cost remains convex, which is reminiscent of the semiconvexity of entropy along lines of McCann interpolations in negative curvature spaces. We also obtain, under suitable conditions, dimension-free bounds of the optimal transport cost in terms of entropy., Comment: 40 pages. To appear in Probability Theory and Related Fields

We extend our study of random walks and induced Dirichlet forms on self-similar sets [arXiv:1604.05440, 1612.01708] to compact spaces of homogeneous type $(K, \rho ,\mu)$. A successive partition on $K$ brings a natural augmented tree structure $(X, E)$ that is Gromov hyperbolic, and the hyperbolic boundary is H\"older equivalent to $K$. We then introduce a class of transient reversible random walks on $(X, E)$ with return ratio $\lambda$. Using Silverstein's theory of Markov chains, we prove that the random walk induces an energy form on $K$ with $$ {\mathcal E}_K [u] \asymp \iint_{K\times K \setminus \Delta} \frac{|u(\xi) - u(\eta)|^2}{V(\xi, \eta)\rho (\xi, \eta)^\beta} d\mu(\xi) d\mu(\eta), $$ where $V(\xi, \eta)$ is the $\mu$-volume of the ball centered at $\xi$ with radius $\rho (\xi, \eta)$, $\Delta$ is the diagonal, and $\beta$ depends on $\lambda$. In particular, for an $\alpha$-set in ${\mathbb R}^d$, the kernel of the energy form is of order $\frac{1}{|\xi-\eta|^{\alpha +\beta}}$. We also discuss conditions for this energy form to be a non-local regular Dirichlet form., Comment: 21 pages, no figures

We consider an optimal transport problem on the unit simplex whose solutions are given by gradients of exponentially concave functions and prove two main results. First, we show that the optimal transport is the large deviation limit of a particle system of Dirichlet processes transporting one probability measure on the unit simplex to another by coordinatewise multiplication and normalizing. The structure of our Lagrangian and the appearance of the Dirichlet process relate our problem closely to the entropic measure on the Wasserstein space as defined by von-Renesse and Sturm in the context of Wasserstein diffusion. The limiting procedure is a triangular limit where we allow simultaneously the number of particles to grow to infinity while the `noise' tends to zero. The method, which generalizes easily to many other cost functions, including the squared Euclidean distance, provides a novel combination of the Schr��dinger problem approach due to C. L��onard and the related Brownian particle systems by Adams et al.which does not require gamma convergence. Second, we analyze the behavior of entropy along the paths of transport. The reference measure on the simplex is taken to be the Dirichlet measure with all zero parameters which relates to the finite-dimensional distributions of the entropic measure. The interpolating curves are not the usual McCann lines. Nevertheless we show that entropy plus a multiple of the transport cost remains convex, which is reminiscent of the semiconvexity of entropy along lines of McCann interpolations in negative curvature spaces. We also obtain, under suitable conditions, dimension-free bounds of the optimal transport cost in terms of entropy., 40 pages. To appear in Probability Theory and Related Fields

Divergences, also known as contrast functions, are distance-like quantities defined on manifolds of non-negative or probability measures. Using the duality in optimal transport, we introduce and study the one-parameter family of $L^{(\pm \alpha)}$-divergences. It includes the Bregman divergence corresponding to the Euclidean quadratic cost, and the $L$-divergence introduced by Pal and the author in connection with portfolio theory and a logarithmic cost function. They admit natural generalizations of exponential family that are closely related to the $\alpha$-family and $q$-exponential family. In particular, the $L^{(\pm \alpha)}$-divergences of the corresponding potential functions are R\'{e}nyi divergences. Using this unified framework we prove that the induced geometries are dually projectively flat with constant sectional curvatures, and a generalized Pythagorean theorem holds true. Conversely, we show that if a statistical manifold is dually projectively flat with constant curvature $\pm \alpha$ with $\alpha > 0$, then it is locally induced by an $L^{(\mp \alpha)}$-divergence. We define in this context a canonical divergence which extends the one for dually flat manifolds., Comment: 39 pages. Revised

First introduced by Fernholz in stochastic portfolio theory, functionally generated portfolio allows its investment performance to be attributed to directly observable and easily interpretable market quantities. In previous works we showed that Fernholz's multiplicatively generated portfolio has deep connections with optimal transport and the information geometry of exponentially concave functions. Recently, Karatzas and Ruf introduced a new additive portfolio generation whose relation with optimal transport was studied by Vervuurt. We show that additively generated portfolio can be interpreted in terms of the well-known dually flat information geometry of Bregman divergence. Moreover, we characterize, in a sense to be made precise, all possible forms of functional portfolio constructions that contain additive and multiplicative generations as special cases. Each construction involves a divergence functional on the unit simplex measuring the market volatility captured, and admits a pathwise decomposition for the portfolio value. We illustrate with an empirical example., 19 pages, 4 figures. Revised

Divergences, also known as contrast functions, are distance-like quantities defined on manifolds of non-negative or probability measures. Using the duality in optimal transport, we introduce and study the one-parameter family of $L^{(\pm ��)}$-divergences. It includes the Bregman divergence corresponding to the Euclidean quadratic cost, and the $L$-divergence introduced by Pal and the author in connection with portfolio theory and a logarithmic cost function. They admit natural generalizations of exponential family that are closely related to the $��$-family and $q$-exponential family. In particular, the $L^{(\pm ��)}$-divergences of the corresponding potential functions are R��nyi divergences. Using this unified framework we prove that the induced geometries are dually projectively flat with constant sectional curvatures, and a generalized Pythagorean theorem holds true. Conversely, we show that if a statistical manifold is dually projectively flat with constant curvature $\pm ��$ with $��> 0$, then it is locally induced by an $L^{(\mp ��)}$-divergence. We define in this context a canonical divergence which extends the one for dually flat manifolds., 39 pages. Revised

Consider a family of portfolio strategies with the aim of achieving the asymptotic growth rate of the best one. The idea behind Cover's universal portfolio is to build a wealth-weighted average which can be viewed as a buy-and-hold portfolio of portfolios. When an optimal portfolio exists, the wealth-weighted average converges to it by concentration of wealth. Working under a discrete time and pathwise setup, we show under suitable conditions that the distribution of wealth in the family satisfies a pathwise large deviation principle as time tends to infinity. Our main result extends Cover's portfolio to the nonparametric family of functionally generated portfolios in stochastic portfolio theory and establishes its asymptotic universality., 25 pages; revised

We introduce a pathwise approach to analyze the relative performance of an equity portfolio with respect to a benchmark market portfolio. In this energy-entropy framework, the relative performance is decomposed into three components: a volatility term, a relative entropy term measuring the distance between the portfolio weights and the market capital distribution, and another entropy term that can be controlled by the investor by adopting a suitable rebalancing strategy. This framework leads to a class of portfolio strategies that allows one to outperform, in the long run, a market that is diverse and sufficiently volatile in the sense of stochastic portfolio theory. The framework is illustrated with several empirical examples., Comment: 21 pages, 7 figures. Substantially revised

In \cite{[K]}, Kaimanovich defined an augmented rooted tree $(X, E)$ corresponding to the Sierpinski gasket $K$, and showed that the Martin boundary of the simple random walk $\{Z_n\}$ on it is homeomorphic to $K$. It is of interest to determine the hitting distributions $v_{{\bf x}}(\cdot) = {\mathbb{P}}_{{\bf x}}\{\lim_{n \rightarrow \infty} Z_n \in \cdot\}$ induced on $K$. Using a reflection principle based on the symmetries of $K$, we show that if the walk starts at the root of $(X, E)$, the hitting distribution is exactly the normalized Hausdorff measure $\mu$ on $K$. In particular, each $v_{{\bf x}}$, ${\bf x} \in X$, is absolutely continuous with respect to $\mu$. This answers a question of Kaimanovich [K, Problem 4.14]. The argument can be generalized to other symmetric self-similar sets., Comment: 19 pages, 10 figures