Search

We derive explicitly the adapted $2$-Wasserstein distance between non-degenerate Gaussian distributions on $\mathbb{R}^N$ and characterize the optimal bicausal coupling(s). This leads to an adapted version of the Bures-Wasserstein distance on the space of positive definite matrices., Comment: 14 pages, 2 figures. Revised

We modify the JKO scheme, which is a time discretization of Wasserstein gradient flows, by replacing the Wasserstein distance with more general transport costs on manifolds. We show when the cost function has a mixed Hessian which defines a Riemannian metric, our modified JKO scheme converges under suitable conditions to the corresponding Riemannian Fokker--Planck equation. Thus on a Riemannian manifold one may replace the (squared) Riemannian distance with any cost function which induces the metric. Of interest is when the Riemannian distance is computationally intractable, but a suitable cost has a simple analytic expression. We consider the Fokker--Planck equation on compact submanifolds with the Neumann boundary condition and on complete Riemannian manifolds with a finite drift condition. As an application we consider Hessian manifolds, taking as a cost the Bregman divergence., Comment: 27 pages

Information geometry is a study of statistical manifolds, that is, spaces of probability distributions from a geometric perspective. Its classical information-theoretic applications relate to statistical concepts such as Fisher information, sufficient statistics, and efficient estimators. Today, information geometry has emerged as an interdisciplinary field that finds applications in diverse areas such as radar sensing, array signal processing, quantum physics, deep learning, and optimal transport. This article presents an overview of essential information geometry to initiate an information theorist, who may be unfamiliar with this exciting area of research. We explain the concepts of divergences on statistical manifolds, generalized notions of distances, orthogonality, and geodesics, thereby paving the way for concrete applications and novel theoretical investigations. We also highlight some recent information-geometric developments, which are of interest to the broader information theory community., Comment: 12 pages, 3 figures, 1 table

Consider the Monge-Kantorovich optimal transport problem where the cost function is given by a Bregman divergence. The associated transport cost, which we call the Bregman-Wasserstein divergence, presents a natural asymmetric extension of the squared $2$-Wasserstein metric and has recently found applications in statistics and machine learning. On the other hand, Bregman divergence is a fundamental object in information geometry and induces a dually flat geometry on the underlying manifold. Using the Bregman-Wasserstein divergence, we lift this dualistic geometry to the space of probability measures, thus extending Otto's weak Riemannian structure of the Wasserstein space to statistical manifolds. We do so by generalizing Lott's formal geometric computations on the Wasserstein space. In particular, we define primal and dual connections on the space of probability measures and show that they are conjugate with respect to Otto's metric. We also define primal and dual displacement interpolations which satisfy the corresponding geodesic equations. As applications, we study displacement convexity and the Bregman-Wasserstein barycenter., Comment: 46 pages, 3 figures

Convex PCA, which was introduced by Bigot et al., is a dimension reduction methodology for data with values in a convex subset of a Hilbert space. This setting arises naturally in many applications, including distributional data in the Wasserstein space of an interval, and ranked compositional data under the Aitchison geometry. Our contribution in this paper is threefold. First, we present several new theoretical results including consistency as well as continuity and differentiability of the objective function in the finite dimensional case. Second, we develop a numerical implementation of finite dimensional convex PCA when the convex set is polyhedral, and show that this provides a natural approximation of Wasserstein geodesic PCA. Third, we illustrate our results with two financial applications, namely distributions of stock returns ranked by size and the capital distribution curve, both of which are of independent interest in stochastic portfolio theory., Comment: 40 pages, 9 figures, sample code available at: https://github.com/stevenacampbell/ConvexPCA

The logarithmic divergence is an extension of the Bregman divergence motivated by optimal transport and a generalized convex duality, and satisfies many remarkable properties. Using the geometry induced by the logarithmic divergence, we introduce a generalization of continuous time mirror descent that we term the conformal mirror descent. We derive its dynamics under a generalized mirror map, and show that it is a time change of a corresponding Hessian gradient flow. We also prove convergence results in continuous time. We apply the conformal mirror descent to online estimation of a generalized exponential family, and construct a family of gradient flows on the unit simplex via the Dirichlet optimal transport problem.

Tsallis and R\'{e}nyi entropies, which are monotone transformations of each other, are deformations of the celebrated Shannon entropy. Maximization of these deformed entropies, under suitable constraints, leads to the $q$-exponential family which has applications in non-extensive statistical physics, information theory and statistics. In previous information-geometric studies, the $q$-exponential family was analyzed using classical convex duality and Bregman divergence. In this paper, we show that a generalized $\lambda$-duality, where $\lambda = 1 - q$ is the constant information-geometric curvature, leads to a generalized exponential family which is essentially equivalent to the $q$-exponential family and has deep connections with R\'{e}nyi entropy and optimal transport. Using this generalized convex duality and its associated logarithmic divergence, we show that our $\lambda$-exponential family satisfies properties that parallel and generalize those of the exponential family. Under our framework, the R\'{e}nyi entropy and divergence arise naturally, and we give a new proof of the Tsallis/R\'{e}nyi entropy maximizing property of the $q$-exponential family. We also introduce a $\lambda$-mixture family which may be regarded as the dual of the $\lambda$-exponential family, and connect it with other mixture-type families. Finally, we discuss a duality between the $\lambda$-exponential family and the $\lambda$-logarithmic divergence, and study its statistical consequences., Comment: 41 pages, 7 figures. Revised

In information geometry, generalized exponential families and statistical manifolds with curvature are under active investigation in recent years. In this paper we consider the statistical manifold induced by a logarithmic $L^{(\alpha)}$-divergence which generalizes the Bregman divergence. It is known that such a manifold is dually projectively flat with constant negative sectional curvature, and is closely related to the $\mathcal{F}^{(\alpha)}$-family, a generalized exponential family introduced by the second author. Our main result constructs a dual foliation of the statistical manifold, i.e., an orthogonal decomposition consisting of primal and dual autoparallel submanifolds. This decomposition, which can be naturally interpreted in terms of primal and dual projections with respect to the logarithmic divergence, extends the dual foliation of a dually flat manifold studied by Amari. As an application, we formulate a new $L^{(\alpha)}$-PCA problem which generalizes the exponential family PCA., Comment: 9 pages, 2 figures. To appear in GSI2021

In this paper we develop a concrete and fully implementable approach to the optimization of functionally generated portfolios in stochastic portfolio theory. The main idea is to optimize over a family of rank-based portfolios parameterized by an exponentially concave function on the unit interval. This choice can be motivated by the long term stability of the capital distribution observed in large equity markets, and allows us to circumvent the curse of dimensionality. The resulting optimization problem, which is convex, allows for various regularizations and constraints to be imposed on the generating function. We prove an existence and uniqueness result for our optimization problem and provide a stability estimate in terms of a Wasserstein metric of the input measure. Then, we formulate a discretization which can be implemented numerically using available software packages and analyze its approximation error. Finally, we present empirical examples using CRSP data from the US stock market, including the performance of the portfolios allowing for dividends, defaults, and transaction costs., Comment: 41 pages, 7 figures, 1 table. Revised version

The adjoint sensitivity method scalably computes gradients of solutions to ordinary differential equations. We generalize this method to stochastic differential equations, allowing time-efficient and constant-memory computation of gradients with high-order adaptive solvers. Specifically, we derive a stochastic differential equation whose solution is the gradient, a memory-efficient algorithm for caching noise, and conditions under which numerical solutions converge. In addition, we combine our method with gradient-based stochastic variational inference for latent stochastic differential equations. We use our method to fit stochastic dynamics defined by neural networks, achieving competitive performance on a 50-dimensional motion capture dataset., Comment: AISTATS 2020; 25 pages, 6 figures in main text; clarify notation in appendix

Spaces of convex and concave functions appear naturally in theory and applications. For example, convex regression and log-concave density estimation are important topics in nonparametric statistics. In stochastic portfolio theory, concave functions on the unit simplex measure the concentration of capital, and their gradient maps define novel investment strategies. The gradient maps may also be regarded as optimal transport maps on the simplex. In this paper we construct and study probability measures supported on spaces of concave functions. These measures may serve as prior distributions in Bayesian statistics and Cover's universal portfolio, and induce distribution-valued random variables via optimal transport. The random concave functions are constructed on the unit simplex by taking a suitably scaled (mollified, or soft) minimum of random hyperplanes. Depending on the regime of the parameters, we show that as the number of hyperplanes tends to infinity there are several possible limiting behaviors. In particular, there is a transition from a deterministic almost sure limit to a non-trivial limiting distribution that can be characterized using convex duality and Poisson point processes., Comment: 42 pages, 8 figures. Substantially revised. To appear in The Annals of Applied Probability

We study the logarithmic $L^{(\alpha)}$-divergence which extrapolates the Bregman divergence and corresponds to solutions to novel optimal transport problems. We show that this logarithmic divergence is equivalent to a conformal transformation of the Bregman divergence, and, via an explicit affine immersion, is equivalent to Kurose's geometric divergence. In particular, the $L^{(\alpha)}$-divergence is a canonical divergence of a statistical manifold with constant sectional curvature $-\alpha$. For such a manifold, we give a geometric interpretation of its sectional curvature in terms of how the divergence between a pair of primal and dual geodesics differ from the dually flat case. Further results can be found in our follow-up paper [27] which uncovers a novel relation between optimal transport and information geometry., Comment: 10 pages, International Conference on Geometric Science of Information. Springer, 2019. arXiv admin note: text overlap with arXiv:1906.00030

Optimal transport and information geometry both study geometric structures on spaces of probability distributions. Optimal transport characterizes the cost-minimizing movement from one distribution to another, while information geometry originates from coordinate-invariant properties of statistical inference. Their connections and applications in statistics and machine learning have started to gain more attention. In this paper we give a new differential geometric connection between the two fields. Namely, the pseudo-Riemannian framework of Kim and McCann, a geometric perspective on the fundamental Ma-Trudinger-Wang (MTW) condition in the regularity theory of optimal transport maps, encodes the dualistic structure of statistical manifold. This general relation is described using the natural framework of $c$-divergence, a divergence defined by an optimal transport map. As a by-product, we obtain a new information-geometric interpretation of the MTW tensor. This connection sheds light on old and new aspects of information geometry. The dually flat geometry of Bregman divergence corresponds to the quadratic cost and the pseudo-Euclidean space, and the $L^{(\alpha)}$-divergence introduced by Pal and the first author has constant sectional curvature in a sense to be made precise. In these cases we give a geometric interpretation of the information-geometric curvature in terms of the divergence between a primal-dual pair of geodesics., Comment: 28 pages, 2 figures. Substantially revised. [Originally titled "Optimal transport and information geometry.]

We introduce a new notion of conditional nonlinear expectation under probability distortion. Such a distorted nonlinear expectation is not sub-additive in general, so it is beyond the scope of Peng's framework of nonlinear expectations. A more fundamental problem when extending the distorted expectation to a dynamic setting is time-inconsistency, that is, the usual "tower property" fails. By localizing the probability distortion and restricting to a smaller class of random variables, we introduce a so-called distorted probability and construct a conditional expectation in such a way that it coincides with the original nonlinear expectation at time zero, but has a time-consistent dynamics in the sense that the tower property remains valid. Furthermore, we show that in the continuous time model this conditional expectation corresponds to a parabolic differential equation whose coefficient involves the law of the underlying diffusion. This work is the first step towards a new understanding of nonlinear expectations under probability distortion, and will potentially be a helpful tool for solving time-inconsistent stochastic optimization problems., Comment: 38 pages, 4 figures. To appear in Mathematics of Operations Research

We consider an optimal transport problem on the unit simplex whose solutions are given by gradients of exponentially concave functions and prove two main results. First, we show that the optimal transport is the large deviation limit of a particle system of Dirichlet processes transporting one probability measure on the unit simplex to another by coordinatewise multiplication and normalizing. The structure of our Lagrangian and the appearance of the Dirichlet process relate our problem closely to the entropic measure on the Wasserstein space as defined by von-Renesse and Sturm in the context of Wasserstein diffusion. The limiting procedure is a triangular limit where we allow simultaneously the number of particles to grow to infinity while the `noise' tends to zero. The method, which generalizes easily to many other cost functions, including the squared Euclidean distance, provides a novel combination of the Schr\"odinger problem approach due to C. L\'eonard and the related Brownian particle systems by Adams et al.which does not require gamma convergence. Second, we analyze the behavior of entropy along the paths of transport. The reference measure on the simplex is taken to be the Dirichlet measure with all zero parameters which relates to the finite-dimensional distributions of the entropic measure. The interpolating curves are not the usual McCann lines. Nevertheless we show that entropy plus a multiple of the transport cost remains convex, which is reminiscent of the semiconvexity of entropy along lines of McCann interpolations in negative curvature spaces. We also obtain, under suitable conditions, dimension-free bounds of the optimal transport cost in terms of entropy., Comment: 40 pages. To appear in Probability Theory and Related Fields

We extend our study of random walks and induced Dirichlet forms on self-similar sets [arXiv:1604.05440, 1612.01708] to compact spaces of homogeneous type $(K, \rho ,\mu)$. A successive partition on $K$ brings a natural augmented tree structure $(X, E)$ that is Gromov hyperbolic, and the hyperbolic boundary is H\"older equivalent to $K$. We then introduce a class of transient reversible random walks on $(X, E)$ with return ratio $\lambda$. Using Silverstein's theory of Markov chains, we prove that the random walk induces an energy form on $K$ with $$ {\mathcal E}_K [u] \asymp \iint_{K\times K \setminus \Delta} \frac{|u(\xi) - u(\eta)|^2}{V(\xi, \eta)\rho (\xi, \eta)^\beta} d\mu(\xi) d\mu(\eta), $$ where $V(\xi, \eta)$ is the $\mu$-volume of the ball centered at $\xi$ with radius $\rho (\xi, \eta)$, $\Delta$ is the diagonal, and $\beta$ depends on $\lambda$. In particular, for an $\alpha$-set in ${\mathbb R}^d$, the kernel of the energy form is of order $\frac{1}{|\xi-\eta|^{\alpha +\beta}}$. We also discuss conditions for this energy form to be a non-local regular Dirichlet form., Comment: 21 pages, no figures

Divergences, also known as contrast functions, are distance-like quantities defined on manifolds of non-negative or probability measures. Using the duality in optimal transport, we introduce and study the one-parameter family of $L^{(\pm \alpha)}$-divergences. It includes the Bregman divergence corresponding to the Euclidean quadratic cost, and the $L$-divergence introduced by Pal and the author in connection with portfolio theory and a logarithmic cost function. They admit natural generalizations of exponential family that are closely related to the $\alpha$-family and $q$-exponential family. In particular, the $L^{(\pm \alpha)}$-divergences of the corresponding potential functions are R\'{e}nyi divergences. Using this unified framework we prove that the induced geometries are dually projectively flat with constant sectional curvatures, and a generalized Pythagorean theorem holds true. Conversely, we show that if a statistical manifold is dually projectively flat with constant curvature $\pm \alpha$ with $\alpha > 0$, then it is locally induced by an $L^{(\mp \alpha)}$-divergence. We define in this context a canonical divergence which extends the one for dually flat manifolds., Comment: 39 pages. Revised

First introduced by Fernholz in stochastic portfolio theory, functionally generated portfolio allows its investment performance to be attributed to directly observable and easily interpretable market quantities. In previous works we showed that Fernholz's multiplicatively generated portfolio has deep connections with optimal transport and the information geometry of exponentially concave functions. Recently, Karatzas and Ruf introduced a new additive portfolio generation whose relation with optimal transport was studied by Vervuurt. We show that additively generated portfolio can be interpreted in terms of the well-known dually flat information geometry of Bregman divergence. Moreover, we characterize, in a sense to be made precise, all possible forms of functional portfolio constructions that contain additive and multiplicative generations as special cases. Each construction involves a divergence functional on the unit simplex measuring the market volatility captured, and admits a pathwise decomposition for the portfolio value. We illustrate with an empirical example., Comment: 19 pages, 4 figures. Revised

Cover's celebrated theorem states that the long run yield of a properly chosen "universal" portfolio is as good as the long run yield of the best retrospectively chosen constant rebalanced portfolio. The "universality" pertains to the fact that this result is model-free, i.e., not dependent on an underlying stochastic process. We extend Cover's theorem to the setting of stochastic portfolio theory as initiated by R. Fernholz: the rebalancing rule need not to be constant anymore but may depend on the present state of the stock market. This model-free result is complemented by a comparison with the log-optimal numeraire portfolio when fixing a stochastic model of the stock market. Roughly speaking, under appropriate assumptions, the optimal long run yield coincides for the three approaches mentioned in the title of this paper. We present our results in discrete and continuous time.

A function is exponentially concave if its exponential is concave. We consider exponentially concave functions on the unit simplex. In a previous paper we showed that gradient maps of exponentially concave functions provide solutions to a Monge-Kantorovich optimal transport problem and give a better gradient approximation than those of ordinary concave functions. The approximation error, called L-divergence, is different from the usual Bregman divergence. Using tools of information geometry and optimal transport, we show that L-divergence induces a new information geometry on the simplex consisting of a Riemannian metric and a pair of dually coupled affine connections which defines two kinds of geodesics. We show that the induced geometry is dually projectively flat but not flat. Nevertheless, we prove an analogue of the celebrated generalized Pythagorean theorem from classical information geometry. On the other hand, we consider displacement interpolation under a Lagrangian integral action that is consistent with the optimal transport problem and show that the action minimizing curves are dual geodesics. The Pythagorean theorem is also shown to have an interesting application of determining the optimal trading frequency in stochastic portfolio theory., Comment: 49 pages, 3 figures; revised. To appear in the Annals of Probability

Let $K$ be a self-similar set satisfying the open set condition. Following Kaimanovich's elegant idea, it has been proved that on the symbolic space $X$ of $K$ a natural augmented tree structure ${\mathfrak E}$ exists; it is hyperbolic, and the hyperbolic boundary $\partial_HX$ with the Gromov metric is H\"older equivalent to $K$. In this paper we consider certain reversible random walks with return ratio $0< \lambda <1$ on $(X, {\mathfrak E})$. We show that the Martin boundary ${\mathcal M}$ can be identified with $\partial_H X$ and $K$. With this setup and a device of Silverstein, we obtain precise estimates of the Martin kernel and the Na\"{i}m kernel in terms of the Gromov product. Moreover, the Na\"{i}m kernel turns out to be a jump kernel satisfying the estimate $\Theta (\xi, \eta) \asymp |\xi-\eta|^{-(\alpha+ \beta)}$, where $\alpha$ is the Hausdorff dimension of $K$ and $\beta$ depends on $\lambda$. For suitable $\beta$, the kernel defines a regular non-local Dirichlet form on $K$. This extends the results of Kigami concerning random walks on certain trees with Cantor-type sets as boundaries., Comment: 33 pages with 2 figures

Consider a family of portfolio strategies with the aim of achieving the asymptotic growth rate of the best one. The idea behind Cover's universal portfolio is to build a wealth-weighted average which can be viewed as a buy-and-hold portfolio of portfolios. When an optimal portfolio exists, the wealth-weighted average converges to it by concentration of wealth. Working under a discrete time and pathwise setup, we show under suitable conditions that the distribution of wealth in the family satisfies a pathwise large deviation principle as time tends to infinity. Our main result extends Cover's portfolio to the nonparametric family of functionally generated portfolios in stochastic portfolio theory and establishes its asymptotic universality., Comment: 25 pages; revised

In stochastic portfolio theory, a relative arbitrage is an equity portfolio which is guaranteed to outperform a benchmark portfolio over a finite horizon. When the market is diverse and sufficiently volatile, and the benchmark is the market or a buy-and-hold portfolio, functionally generated portfolios introduced by Fernholz provide a systematic way of constructing relative arbitrages. In this paper we show that if the market portfolio is replaced by the equal or entropy weighted portfolio among many others, no relative arbitrages can be constructed under the same conditions using functionally generated portfolios. We also introduce and study a shaped-constrained optimization problem for functionally generated portfolios in the spirit of maximum likelihood estimation of a log-concave density., Comment: 33 pages, 5 figures, 2 tables; revised version

Consider an equity market with $n$ stocks. The vector of proportions of the total market capitalizations that belong to each stock is called the market weight. The market weight defines the market portfolio which is a buy-and-hold portfolio representing the performance of the entire stock market. Consider a function that assigns a portfolio vector to each possible value of the market weight, and we perform self-financing trading using this portfolio function. We study the problem of characterizing functions such that the resulting portfolio will outperform the market portfolio in the long run under the conditions of diversity and sufficient volatility. No other assumption on the future behavior of stock prices is made. We prove that the only solutions are functionally generated portfolios in the sense of Fernholz. A second characterization is given as the optimal maps of a remarkable optimal transport problem. Both characterizations follow from a novel property of portfolios called multiplicative cyclical monotonicity., Comment: 31 pages, 5 figures; substantially revised; Section 4 illustrates the optiaml transport approach with empirical examples

We introduce a pathwise approach to analyze the relative performance of an equity portfolio with respect to a benchmark market portfolio. In this energy-entropy framework, the relative performance is decomposed into three components: a volatility term, a relative entropy term measuring the distance between the portfolio weights and the market capital distribution, and another entropy term that can be controlled by the investor by adopting a suitable rebalancing strategy. This framework leads to a class of portfolio strategies that allows one to outperform, in the long run, a market that is diverse and sufficiently volatile in the sense of stochastic portfolio theory. The framework is illustrated with several empirical examples., Comment: 21 pages, 7 figures. Substantially revised

In \cite{[K]}, Kaimanovich defined an augmented rooted tree $(X, E)$ corresponding to the Sierpinski gasket $K$, and showed that the Martin boundary of the simple random walk $\{Z_n\}$ on it is homeomorphic to $K$. It is of interest to determine the hitting distributions $v_{{\bf x}}(\cdot) = {\mathbb{P}}_{{\bf x}}\{\lim_{n \rightarrow \infty} Z_n \in \cdot\}$ induced on $K$. Using a reflection principle based on the symmetries of $K$, we show that if the walk starts at the root of $(X, E)$, the hitting distribution is exactly the normalized Hausdorff measure $\mu$ on $K$. In particular, each $v_{{\bf x}}$, ${\bf x} \in X$, is absolutely continuous with respect to $\mu$. This answers a question of Kaimanovich [K, Problem 4.14]. The argument can be generalized to other symmetric self-similar sets., Comment: 19 pages, 10 figures