Search

Finding meaningful representations and distances of hierarchical data is important in many fields. This paper presents a new method for hierarchical data embedding and distance. Our method relies on combining diffusion geometry, a central approach to manifold learning, and hyperbolic geometry. Specifically, using diffusion geometry, we build multi-scale densities on the data, aimed to reveal their hierarchical structure, and then embed them into a product of hyperbolic spaces. We show theoretically that our embedding and distance recover the underlying hierarchical structure. In addition, we demonstrate the efficacy of the proposed method and its advantages compared to existing methods on graph embedding benchmarks and hierarchical datasets.

Starting with sets of disorganized observations of spatially varying and temporally evolving systems, obtained at different (also disorganized) sets of parameters, we demonstrate the data-driven derivation of parameter dependent, evolutionary partial differential equation (PDE) models capable of generating the data. This tensor type of data is reminiscent of shuffled (multi-dimensional) puzzle tiles. The independent variables for the evolution equations (their "space" and "time") as well as their effective parameters are all "emergent", i.e., determined in a data-driven way from our disorganized observations of behavior in them. We use a diffusion map based "questionnaire" approach to build a parametrization of our emergent space/time/parameter space for the data. This approach iteratively processes the data by successively observing them on the "space", the "time", and the "parameter" axes of a tensor. Once the data are organized, we use machine learning (here, neural networks) to approximate the operators governing the evolution equations in this emergent space. Our illustrative example is based on a previously developed vertex-plus-signaling model of Drosophila embryonic development. This allows us to discuss features of the process like symmetry breaking, translational invariance, and autonomousness of the emergent PDE model, as well as its interpretability., 9 main pages (8 figures) with 7 SI pages (5 figures) Submitted to PNAS Nexus

We describe recent nonlinear analytic approximation tools in the classical setting of Hardy spaces in the upper half plane and show how to transfer them to the higher dimensional real setting of harmonic functions in upper half spaces. It is known [6] that all harmonic functions in higher dimensions are combinations of holomorphic functions on 2 dimensional planes, extended as, constant in normal directions. We derive representation theorems, with corresponding isometries, opening the door for applications in higher dimensions, to the processing of highly oscillatory multidimensional signals., Comment: arXiv admin note: substantial text overlap with arXiv:2101.05311

An inspiration at the origin of wavelet analysis (when Grossmann, Morlet, Meyer and collaborators were interacting and exploring versions of multiscale representations) was provided by the analysis of holomorphic signals, for which the images of the phase of Cauchy wavelets were remarkable in their ability to reveal intricate singularities or dynamic structures, such as instantaneous frequency jumps in musical recordings. Our goal is to follow their seminal work and introduce recent developments in nonlinear analysis. In particular we sketch methods extending conventional Fourier analysis, exploiting both phase and amplitudes of holomorphic functions. The Blaschke factors are a key ingredient, in building analytic tools, starting with the Malmquist Takenaka orthonormal bases of the Hardy space, continuing with "best" adapted bases obtained through phase unwinding, and concluding with relations to composition of Blaschke products and their dynamics. We also remark that the phase of a Blaschke product is a one layer neural net with arctan as an activation sigmoid and that the composition is a "Deep Neural Net" whose depth is the number of compositions. Our results provide a wealth of related library of orthonormal bases.

A fundamental step in many data-analysis techniques is the construction of an affinity matrix describing similarities between data points. When the data points reside in Euclidean space, a widespread approach is to from an affinity matrix by the Gaussian kernel with pairwise distances, and to follow with a certain normalization (e.g. the row-stochastic normalization or its symmetric variant). We demonstrate that the doubly-stochastic normalization of the Gaussian kernel with zero main diagonal (i.e., no self loops) is robust to heteroskedastic noise. That is, the doubly-stochastic normalization is advantageous in that it automatically accounts for observations with different noise variances. Specifically, we prove that in a suitable high-dimensional setting where heteroskedastic noise does not concentrate too much in any particular direction in space, the resulting (doubly-stochastic) noisy affinity matrix converges to its clean counterpart with rate $m^{-1/2}$, where $m$ is the ambient dimension. We demonstrate this result numerically, and show that in contrast, the popular row-stochastic and symmetric normalizations behave unfavorably under heteroskedastic noise. Furthermore, we provide examples of simulated and experimental single-cell RNA sequence data with intrinsic heteroskedasticity, where the advantage of the doubly-stochastic normalization for exploratory analysis is evident.

Representation learning is typically applied to only one mode of a data matrix, either its rows or columns. Yet in many applications, there is an underlying geometry to both the rows and the columns. We propose utilizing this coupled structure to perform co-manifold learning: uncovering the underlying geometry of both the rows and the columns of a given matrix, where we focus on a missing data setting. Our unsupervised approach consists of three components. We first solve a family of optimization problems to estimate a complete matrix at multiple scales of smoothness. We then use this collection of smooth matrix estimates to compute pairwise distances on the rows and columns based on a new multi-scale metric that implicitly introduces a coupling between the rows and the columns. Finally, we construct row and column representations from these multi-scale metrics. We demonstrate that our approach outperforms competing methods in both data visualization and clustering., Comment: 16 pages, 9 figures

The discovery of physical laws consistent with empirical observations lies at the heart of (applied) science and engineering. These laws typically take the form of nonlinear differential equations depending on parameters, dynamical systems theory provides, through the appropriate normal forms, an "intrinsic", prototypical characterization of the types of dynamical regimes accessible to a given model. Using an implementation of data-informed geometry learning we directly reconstruct the relevant "normal forms": a quantitative mapping from empirical observations to prototypical realizations of the underlying dynamics. Interestingly, the state variables and the parameters of these realizations are inferred from the empirical observations, without prior knowledge or understanding, they parametrize the dynamics {\em intrinsically}, without explicit reference to fundamental physical quantities.

Multiple time scale stochastic dynamical systems are ubiquitous in science and engineering, and the reduction of such systems and their models to only their slow components is often essential for scientific computation and further analysis. Rather than being available in the form of an explicit analytical model, often such systems can only be observed as a data set which exhibits dynamics on several time scales. We will focus on applying and adapting data mining and manifold learning techniques to detect the slow components in such multiscale data. Traditional data mining methods are based on metrics (and thus, geometries) which are not informed of the multiscale nature of the underlying system dynamics; such methods cannot successfully recover the slow variables. Here, we present an approach which utilizes both the local geometry and the local dynamics within the data set through a metric which is both insensitive to the fast variables and more general than simple statistical averaging. Our analysis of the approach provides conditions for successfully recovering the underlying slow variables, as well as an empirical protocol guiding the selection of the method parameters.

In this paper, we provide a framework based upon diffusion processes for finding meaningful geometric descriptions of data sets. We show that eigenfunctions of Markov matrices can be used to construct coordinates called diffusion maps that generate efficient representations of complex geometric structures. The associated family of diffusion distances, obtained by iterating the Markov matrix, defines multiscale geometries that prove to be useful in the context of data parametrization and dimensionality reduction. The proposed framework relates the spectral properties of Markov processes to their geometric counterparts and it unifies ideas arising in a variety of contexts such as machine learning, spectral graph theory and eigenmap methods.

This paper presents a diffusion based probabilistic interpretation of spectral clustering and dimensionality reduction algorithms that use the eigenvectors of the normalized graph Laplacian. Given the pairwise adjacency matrix of all points, we define a diffusion distance between any two data points and show that the low dimensional representation of the data by the first few eigenvectors of the corresponding Markov matrix is optimal under a certain mean squared error criterion. Furthermore, assuming that data points are random samples from a density $p(\x) = e^{-U(\x)}$ we identify these eigenvectors as discrete approximations of eigenfunctions of a Fokker-Planck operator in a potential $2U(\x)$ with reflecting boundary conditions. Finally, applying known results regarding the eigenvalues and eigenfunctions of the continuous Fokker-Planck operator, we provide a mathematical justification for the success of spectral clustering and dimensional reduction algorithms based on these first few eigenvectors. This analysis elucidates, in terms of the characteristics of diffusion processes, many empirical findings regarding spectral clustering algorithms., submitted to NIPS 2005