Search

The notion of Fr\'echet mean (also known as "barycenter") network is the workhorse of most machine learning algorithms that require the estimation of a "location" parameter to analyse network-valued data. In this context, it is critical that the network barycenter inherits the topological structure of the networks in the training dataset. The metric - which measures the proximity between networks - controls the structural properties of the barycenter. This work is significant because it provides for the first time analytical estimates of the sample Fr\'echet mean for the stochastic blockmodel, which is at the cutting edge of rigorous probabilistic analysis of random networks. We show that the mean network computed with the Hamming distance is unable to capture the topology of the networks in the training sample, whereas the mean network computed using the effective resistance distance recovers the correct partitions and associated edge density. From a practical standpoint, our work informs the choice of metrics in the context where the sample Fr\'echet mean network is used to characterise the topology of networks for network-valued machine learning, Comment: 23 pages

This work addresses the rising demand for novel tools in statistical and machine learning for "graph-valued random variables" by proposing a fast algorithm to compute the sample Frechet mean, which replaces the concept of sample mean for graphs (or networks). We use convolutional neural networks to learn the morphology of the graphs in a set of graphs. Our experiments on several ensembles of random graphs demonstrate that our method can reliably recover the sample Frechet mean.

For graph-valued data sampled iid from a distribution $\mu$, the sample moments are computed with respect to a choice of metric. In this work, we equip the set of graphs with the pseudo-metric defined by the $\ell_2$ norm between the eigenvalues of the respective adjacency matrices. We use this pseudo metric and the respective sample moments of a graph valued data set to infer the parameters of a distribution $\hat{\mu}$ and interpret this distribution as an approximation of $\mu$. We verify experimentally that complex distributions $\mu$ can be approximated well taking this approach.

We address the following foundational question: what is the population, and sample, Frechet mean (or median) graph of an ensemble of inhomogeneous Erdos-Renyi random graphs? We prove that if we use the Hamming distance to compute distances between graphs, then the Frechet mean (or median) graph of an ensemble of inhomogeneous random graphs is obtained by thresholding the expected adjacency matrix of the ensemble. We show that the result also holds for the sample mean (or median) when the population expected adjacency matrix is replaced with the sample mean adjacency matrix. Consequently, the Frechet mean (or median) graph of inhomogeneous Erdos-Renyi random graphs exhibits a sharp threshold: it is either the empty graph, or the complete graph. This novel theoretical result has some significant practical consequences; for instance, the Frechet mean of an ensemble of sparse inhomogeneous random graphs is always the empty graph.

To characterize the location (mean, median) of a set of graphs, one needs a notion of centrality that is adapted to metric spaces, since graph sets are not Euclidean spaces. A standard approach is to consider the Frechet mean. In this work, we equip a set of graphs with the pseudometric defined by the norm between the eigenvalues of their respective adjacency matrix. Unlike the edit distance, this pseudometric reveals structural changes at multiple scales, and is well adapted to studying various statistical problems for graph-valued data. We describe an algorithm to compute an approximation to the sample Frechet mean of a set of undirected unweighted graphs with a fixed size using this pseudometric., Comment: arXiv admin note: text overlap with arXiv:2105.04062

The availability of large datasets composed of graphs creates an unprecedented need to invent novel tools in statistical learning for graph-valued random variables. To characterize the average of a sample of graphs, one can compute the sample Frechet mean and median graphs. In this paper, we address the following foundational question: does a mean or median graph inherit the structural properties of the graphs in the sample? An important graph property is the edge density; we establish that edge density is an hereditary property, which can be transmitted from a graph sample to its sample Frechet mean or median graphs, irrespective of the method used to estimate the mean or the median. Because of the prominence of the Frechet mean in graph-valued machine learning, this novel theoretical result has some significant practical consequences., Comment: 14 pages

To characterize the location (mean, median) of a set of graphs, one needs a notion of centrality that is adapted to metric spaces, since graph sets are not Euclidean spaces. A standard approach is to consider the Fr\'echet mean. In this work, we equip a set of graph with the pseudometric defined by the $\ell_2$ norm between the eigenvalues of their respective adjacency matrix . Unlike the edit distance, this pseudometric reveals structural changes at multiple scales, and is well adapted to studying various statistical problems on sets of graphs. We describe an algorithm to compute an approximation to the Fr\'echet mean of a set of undirected unweighted graphs with a fixed size., Comment: 28 pages

Comparison of graph structure is a ubiquitous task in data analysis and machine learning, with diverse applications in fields such as neuroscience, cyber security, social network analysis, and bioinformatics, among others. Discovery and comparison of structures such as modular communities, rich clubs, hubs, and trees in data in these fields yields insight into the generative mechanisms and functional properties of the graph. Often, two graphs are compared via a pairwise distance measure, with a small distance indicating structural similarity and vice versa. Common choices include spectral distances (also known as $\lambda$ distances) and distances based on node affinities. However, there has of yet been no comparative study of the efficacy of these distance measures in discerning between common graph topologies and different structural scales. In this work, we compare commonly used graph metrics and distance measures, and demonstrate their ability to discern between common topological features found in both random graph models and empirical datasets. We put forward a multi-scale picture of graph structure, in which the effect of global and local structure upon the distance measures is considered. We make recommendations on the applicability of different distance measures to empirical graph data problem based on this multi-scale view. Finally, we introduce the Python library NetComp which implements the graph distances used in this work.

The study of time-varying (dynamic) networks (graphs) is of fundamental importance for computer network analytics. Several methods have been proposed to detect the effect of significant structural changes in a time series of graphs. The main contribution of this work is a detailed analysis of a dynamic community graph model. This model is formed by adding new vertices, and randomly attaching them to the existing nodes. It is a dynamic extension of the well-known stochastic blockmodel. The goal of the work is to detect the time at which the graph dynamics switches from a normal evolution -- where balanced communities grow at the same rate -- to an abnormal behavior -- where communities start merging. In order to circumvent the problem of decomposing each graph into communities, we use a metric to quantify changes in the graph topology as a function of time. The detection of anomalies becomes one of testing the hypothesis that the graph is undergoing a significant structural change. In addition the the theoretical analysis of the test statistic, we perform Monte Carlo simulations of our dynamic graph model to demonstrate that our test can detect changes in graph topology.

We describe here the recent results of a multidisciplinary effort to design a biomarker that can actively and continuously decode the progressive changes in neuronal organization leading to epilepsy, a process known as epileptogenesis. Using an animal model of acquired epilepsy, wechronically record hippocampal evoked potentials elicited by an auditory stimulus. Using a set of reduced coordinates, our algorithm can identify universal smooth low-dimensional configurations of the auditory evoked potentials that correspond to distinct stages of epileptogenesis. We use a hidden Markov model to learn the dynamics of the evoked potential, as it evolves along these smooth low-dimensional subsets. We provide experimental evidence that the biomarker is able to exploit subtle changes in the evoked potential to reliably decode the stage of epileptogenesis and predict whether an animal will eventually recover from the injury, or develop spontaneous seizures.

Covariance matrices are fundamental to the analysis and forecast of economic, physical and biological systems. Although the eigenvalues $\{\lambda_i\}$ and eigenvectors $\{{\bf u}_i\}$ of a covariance matrix are central to such endeavors, in practice one must inevitably approximate the covariance matrix based on data with finite sample size $n$ to obtain empirical eigenvalues $\{\tilde{\lambda}_i\}$ and eigenvectors $\{\tilde{{\bf u}}_i\}$, and therefore understanding the error so introduced is of central importance. We analyze eigenvector error $\|{\bf u}_i - \tilde{{\bf u}}_i \|^2$ while leveraging the assumption that the true covariance matrix having size $p$ is drawn from a matrix ensemble with known spectral properties---particularly, we assume the distribution of population eigenvalues weakly converges as $p\to\infty$ to a spectral density $\rho(\lambda)$ and that the spacing between population eigenvalues is similar to that for the Gaussian orthogonal ensemble. Our approach complements previous analyses of eigenvector error that require the full set of eigenvalues to be known, which can be computationally infeasible when $p$ is large. To provide a scalable approach for uncertainty quantification of eigenvector error, we consider a fixed eigenvalue $\lambda$ and approximate the distribution of the expected square error $r= \mathbb{E}\left[\| {\bf u}_i - \tilde{{\bf u}}_i \|^2\right]$ across the matrix ensemble for all ${\bf u}_i$ associated with $\lambda_i=\lambda$. We find, for example, that for sufficiently large matrix size $p$ and sample size $n>p$, the probability density of $r$ scales as $1/nr^2$. This power-law scaling implies that eigenvector error is extremely heterogeneous---even if $r$ is very small for most eigenvectors, it can be large for others with non-negligible probability. We support this and further results with numerical experiments., Comment: 24 pages, 8 figures

To quantify the fundamental evolution of time-varying networks, and detect abnormal behavior, one needs a notion of temporal difference that captures significant organizational changes between two successive instants. In this work, we propose a family of distances that can be tuned to quantify structural changes occurring on a graph at different scales: from the local scale formed by the neighbors of each vertex, to the largest scale that quantifies the connections between clusters, or communities. Our approach results in the definition of a true distance, and not merely a notion of similarity. We propose fast (linear in the number of edges) randomized algorithms that can quickly compute an approximation to the graph metric. The third contribution involves a fast algorithm to increase the robustness of a network by optimally decreasing the Kirchhoff index. Finally, we conduct several experiments on synthetic graphs and real networks, and we demonstrate that we can detect configurational changes that are directly related to the hidden variables governing the evolution of dynamic networks.

In this work we address the problem of detecting overlapping communities in social networks. Because the word "community" is an ambiguous term, it is necessary to quantify what it means to be a community within the context of a particular type of problem. Our interpretation is that this quantification must be done at a minimum of three scales. These scales are at the level of: individual nodes, individual communities, and the network as a whole. Each of these scales involves quantitative features of community structure that are not accurately represented at the other scales, but are important for defining a particular notion of community. Our work focuses on providing sensible ways to quantify what is desired at each of these scales for a notion of community applicable to social networks, and using these models to develop a community detection algorithm. Appealing features of our approach is that it naturally allows for nodes to belong to multiple communities, and is computationally efficient for large networks with low overall edge density. The scaling of the algorithm is $O(N~\overline{k^2} + \overline{N_{com}^2})$, where $N$ is the number of nodes in the network, $\overline{N_{com}^2}$ is the average squared community size, and $\overline{k^2}$ is the expected value of a node's degree squared. Although our work focuses on developing a computationally efficient algorithm for overlapping community detection in the context of social networks, our primary contribution is developing a methodology that is highly modular and can easily be adapted to target specific notions of community.

Nonlinear dimensionality reduction embeddings computed from datasets do not provide a mechanism to compute the inverse map. In this paper, we address the problem of computing a stable inverse map to such a general bi-Lipschitz map. Our approach relies on radial basis functions (RBFs) to interpolate the inverse map everywhere on the low-dimensional image of the forward map. We demonstrate that the scale-free cubic RBF kernel performs better than the Gaussian kernel: it does not suffer from ill-conditioning, and does not require the choice of a scale. The proposed construction is shown to be similar to the Nystr\"om extension of the eigenvectors of the symmetric normalized graph Laplacian matrix. Based on this observation, we provide a new interpretation of the Nystr\"om extension with suggestions for improvement., Comment: Accepted for publication in Applied and Computational Harmonic Analysis

The original contributions of this paper are twofold: a new understanding of the influence of noise on the eigenvectors of the graph Laplacian of a set of image patches, and an algorithm to estimate a denoised set of patches from a noisy image. The algorithm relies on the following two observations: (1) the low-index eigenvectors of the diffusion, or graph Laplacian, operators are very robust to random perturbations of the weights and random changes in the connections of the patch-graph; and (2) patches extracted from smooth regions of the image are organized along smooth low-dimensional structures in the patch-set, and therefore can be reconstructed with few eigenvectors. Experiments demonstrate that our denoising algorithm outperforms the denoising gold-standards.

Constructing an efficient parameterization of a large, noisy data set of points lying close to a smooth manifold in high dimension remains a fundamental problem. One approach consists in recovering a local parameterization using the local tangent plane. Principal component analysis (PCA) is often the tool of choice, as it returns an optimal basis in the case of noise-free samples from a linear subspace. To process noisy data samples from a nonlinear manifold, PCA must be applied locally, at a scale small enough such that the manifold is approximately linear, but at a scale large enough such that structure may be discerned from noise. Using eigenspace perturbation theory and non-asymptotic random matrix theory, we study the stability of the subspace estimated by PCA as a function of scale, and bound (with high probability) the angle it forms with the true tangent space. By adaptively selecting the scale that minimizes this bound, our analysis reveals an appropriate scale for local tangent plane recovery. We also introduce a geometric uncertainty principle quantifying the limits of noise-curvature perturbation for stable recovery. With the purpose of providing perturbation bounds that can be used in practice, we propose plug-in estimates that make it possible to directly apply the theoretical results to real data sets., Comment: 53 pages. Revised manuscript with new content addressing application of results to real data sets

In this paper we address the problem of understanding the success of algorithms that organize patches according to graph-based metrics. Algorithms that analyze patches extracted from images or time series have led to state-of-the art techniques for classification, denoising, and the study of nonlinear dynamics. The main contribution of this work is to provide a theoretical explanation for the above experimental observations. Our approach relies on a detailed analysis of the commute time metric on prototypical graph models that epitomize the geometry observed in general patch graphs. We prove that a parametrization of the graph based on commute times shrinks the mutual distances between patches that correspond to rapid local changes in the signal, while the distances between patches that correspond to slow local changes expand. In effect, our results explain why the parametrization of the set of patches based on the eigenfunctions of the Laplacian can concentrate patches that correspond to rapid local changes, which would otherwise be shattered in the space of patches. While our results are based on a large sample analysis, numerical experimentations on synthetic and real data indicate that the results hold for datasets that are very small in practice.

Huang's Empirical Mode Decomposition (EMD) is an algorithm for analyzing nonstationary data that provides a localized time-frequency representation by decomposing the data into adaptively defined modes. EMD can be used to estimate a signal's instantaneous frequency (IF) but suffers from poor performance in the presence of noise. To produce a meaningful IF, each mode of the decomposition must be nearly monochromatic, a condition that is not guaranteed by the algorithm and fails to be met when the signal is corrupted by noise. In this work, the extraction of modes containing both signal and noise is identified as the cause of poor IF estimation. The specific mechanism by which such "transition" modes are extracted is detailed and builds on the observation of Flandrin and Goncalves that EMD acts in a filter bank manner when analyzing pure noise. The mechanism is shown to be dependent on spectral leak between modes and the phase of the underlying signal. These ideas are developed through the use of simple signals and are tested on a synthetic seismic waveform., Comment: 28 pages, 19 figures. High quality color figures available on Daniel Kaslovsky's website: http://amath.colorado.edu/student/kaslovsk/

We propose a new method to analyze seismic time series and estimate the arrival-times of seismic waves. Our approach combines two ingredients: the times series are first lifted into a high-dimensional space using time-delay embedding; the resulting phase space is then parametrized using a nonlinear method based on the eigenvectors of the graph Laplacian. We validate our approach using a dataset of seismic events that occurred in Idaho, Montana, Wyoming, and Utah, between 2005 and 2006. Our approach outperforms methods based on singular-spectrum analysis, waveleta nalysis, and STA/LTA., Comment: 21 pages, 13 figures

Functional Magnetic Resonance Imaging (fMRI) provides dynamical access into the complex functioning of the human brain, detailing the hemodynamic activity of thousands of voxels during hundreds of sequential time points. One approach towards illuminating the connection between fMRI and cognitive function is through decoding; how do the time series of voxel activities combine to provide information about internal and external experience? Here we seek models of fMRI decoding which are balanced between the simplicity of their interpretation and the effectiveness of their prediction. We use signals from a subject immersed in virtual reality to compare global and local methods of prediction applying both linear and nonlinear techniques of dimensionality reduction. We find that the prediction of complex stimuli is remarkably low-dimensional, saturating with less than 100 features. In particular, we build effective models based on the decorrelated components of cognitive activity in the classically-defined Brodmann areas. For some of the stimuli, the top predictive areas were surprisingly transparent, including Wernicke's area for verbal instructions, visual cortex for facial and body features, and visual-temporal regions for velocity. Direct sensory experience resulted in the most robust predictions, with the highest correlation ($c \sim 0.8$) between the predicted and experienced time series of verbal instructions. Techniques based on non-linear dimensionality reduction (Laplacian eigenmaps) performed similarly. The interpretability and relative simplicity of our approach provides a conceptual basis upon which to build more sophisticated techniques for fMRI decoding and offers a window into cognitive function during dynamic, natural experience., Comment: To appear in: Advances in Neural Information Processing Systems 20, Scholkopf B., Platt J. and Hofmann T. (Editors), MIT Press, 2008

We propose a novel method to embed a functional magnetic resonance imaging (fMRI) dataset in a low-dimensional space. The embedding optimally preserves the local functional coupling between fMRI time series and provides a low-dimensional coordinate system for detecting activated voxels. To compute the embedding, we build a graph of functionally connected voxels. We use the commute time, instead of the geodesic distance, to measure functional distances on the graph. Because the commute time can be computed directly from the eigenvectors of (a symmetric version) the graph probability transition matrix, we use these eigenvectors to embed the dataset in low dimensions. After clustering the datasets in low dimensions, coherent structures emerge that can be easily interpreted. We performed an extensive evaluation of our method comparing it to linear and nonlinear techniques using synthetic datasets and in vivo datasets. We analyzed datasets from the EBC competition obtained with subjects interacting in an urban virtual reality environment. Our exploratory approach is able to detect independently visual areas (V1/V2, V5/MT), auditory areas, and language areas. Our method can be used to analyze fMRI collected during ``natural stimuli''., Comment: 16 pages, 21 figures, submitted to Neuroimage, revised (Jan. 2008)