Search

Samples of phylogenetic trees arise in a variety of evolutionary and biomedical applications, and the Fr\'echet mean in Billera-Holmes-Vogtmann tree space is a summary tree shown to have advantages over other mean or consensus trees. However, use of the Fr\'echet mean raises computational and statistical issues which we explore in this paper. The Fr\'echet sample mean is known often to contain fewer internal edges than the trees in the sample, and in this circumstance calculating the mean by iterative schemes can be problematic due to slow convergence. We present new methods for identifying edges which must lie in the Fr\'echet sample mean and apply these to a data set of gene trees relating organisms from the apicomplexa which cause a variety of parasitic infections. When a sample of trees contains a significant level of heterogeneity in the branching patterns, or topologies, displayed by the trees then the Fr\'echet mean is often a star tree, lacking any internal edges. Not only in this situation, the population Fr\'echet mean is affected by a non-Euclidean phenomenon called stickness which impacts upon asymptotics, and we examine two data sets for which the mean tree is a star tree. The first consists of trees representing the physical shape of artery structures in a sample of medical images of human brains in which the branching patterns are very diverse. The second consists of gene trees from a population of baboons in which there is evidence of substantial hybridization. We develop hypothesis tests which work in the presence of stickiness. The first is a test for the presence of a given edge in the Fr\'echet population mean; the second is a two-sample test for differences in two distributions which share the same sticky population mean., Comment: 37 pages, 16 figures

The Fr\'echet mean, a generalization to a metric space of the expectation of a random variable in a vector space, can exhibit unexpected behavior for a wide class of random variables. For instance, it can stick to a point (more generally to a closed set) under resampling: sample stickiness. It can stick to a point for topologically nearby distributions: topological stickiness, such as total variation or Wasserstein stickiness. It can stick to a point for slight but arbitrary perturbations: perturbation stickiness. Here, we explore these and various other flavors of stickiness and their relationship in varying scenarios, for instance on CAT($\kappa$) spaces, $\kappa\in \mathbb{R}$. Interestingly, modulation stickiness (faster asymptotic rate than $\sqrt{n}$) and directional stickiness (a generalization of moment stickiness from the literature) allow for the development of new statistical methods building on an asymptotic fluctuation, where, due to stickiness, the mean itself features no asymptotic fluctuation. Also, we rule out sticky flavors on manifolds in scenarios with curvature bounds.

It has been observed that the sample mean of certain probability distributions in Billera-Holmes-Vogtmann (BHV) phylogenetic spaces is confined to a lower-dimensional subspace for large enough sample size. This non-standard behavior has been called stickiness and poses difficulties in statistical applications when comparing samples of sticky distributions. We extend previous results on stickiness to show the equivalence of this sampling behavior to topological conditions in the special case of BHV spaces. Furthermore, we propose to alleviate statistical comparision of sticky distributions by including the directional derivatives of the Fr\'echet function: the degree of stickiness., Comment: 8 Pages, 1 Figure, conference submission to GSI 2023