Search

A variety of change point estimation and detection algorithms have been developed for random variables observed over time. The acquisition of data in current practice often results in multiple subjects studied. The traditional treatment of such observations involves the assumption of their independence. In practice, however, this assumption is often inadequate or unrealistic. We propose an effective and modern computerized approach to estimating and detecting change points in linear model time series processes in the situation when the assumption of independent observations is not feasible. The developed methodology relies on the multivariate transformation and matrix normal distribution. The latter is used for separating the sources of variability. The application of the back-transform of the exponential transformation leads to a flexible distribution that effectively accounts for deviations from normality. The developed procedure has been successfully tested in various settings and applied to a crime rate data set. [ABSTRACT FROM AUTHOR]

This article presents the "Generalized Modified Matrix Bessel Distribution (GMMaB)", which is a generalization of the "Generalized Multivariate Modified Bessel Distribution" (GMMB). The features of the suggested distribution, including characteristic functions (C.f.), moments, marginal and conditional distributions, and the linear combination of Matrix variate in several forms have been investigated. [ABSTRACT FROM AUTHOR]

Low-rank approximation models of data matrices have become important machine learning and data mining tools in many fields, including computer vision, text mining, bioinformatics, and many others. They allow for embedding high-dimensional data into low-dimensional spaces, which mitigates the effects of noise and uncovers latent relations. In order to make the learned representations inherit the structures in the original data, graph-regularization terms are often added to the loss function. However, the prior graph construction often fails to reflect the true network connectivity and the intrinsic relationships. In addition, many graph-regularized methods fail to take the dual spaces into account. Probabilistic models are often used to model the distribution of the representations, but most of previous methods often assume that the hidden variables are independent and identically distributed for simplicity. To this end, we propose a learnable graph-regularization model for matrix decomposition (LGMD), which builds a bridge between graph-regularized methods and probabilistic matrix decomposition models for the first time. LGMD incorporates two graphical structures (i.e., two precision matrices) learned in an iterative manner via sparse precision matrix estimation and is more robust to noise and missing entries. Extensive numerical results and comparison with competing methods demonstrate its effectiveness. [ABSTRACT FROM AUTHOR]

Principal component analysis (PCA) is one of the most widely used dimension reduction and multivariate statistical techniques. From a probabilistic perspective, PCA seeks a low-dimensional representation of data in the presence of independent identical Gaussian noise. Probabilistic PCA (PPCA) and its variants have been extensively studied for decades. Most of them assume the underlying noise follows a certain independent identical distribution. However, the noise in the real world is usually complicated and structured. To address this challenge, some variants of PCA for non-IID data have been proposed. However, most of the existing methods only assume that the noise is correlated in the feature space while there may exist two-way structured noise. To this end, we propose a powerful and intuitive PCA method (MN-PCA) through modeling the graphical noise by the matrix normal distribution, which enables us to explore the structure of noise in both the feature space and the sample space. MN-PCA obtains a low-rank representation of data and the structure of noise simultaneously. And it can be explained as approximating data over the generalized Mahalanobis distance. We first solve this model by a standard approach – maximizing the regularized likelihood – and then develop a novel algorithm that exploits the Wasserstein distance, which is more robust. Extensive experiments on various data demonstrate their effectiveness. • MN-PCA discovers the low-rank representation and the noise structure simultaneously. • Theoretically analyze the error bounds when the noise is matrix normal. • Develop two algorithms based on the Wasserstein distance and the regularized likelihood. [ABSTRACT FROM AUTHOR]

We introduce a model-based approach for clustering multivariate functional data observations. We utilize theoretical results regarding a surrogate density on the truncated Karhunen–Loeve expansions along with a direct sum specification of the functional space to define a matrix normal distribution on functional principal components. This formulation allows for individual parsimonious modelling of the function space and coefficient space of the univariate components of the multivariate functional observations in the form a subspace projection and latent factor analyzers, respectively. The approach facilitates interpretation at both the full multivariate level and the component level, which is of specific interest when the component functions have clear meaning. We derive an AECM algorithm for fitting the model, and discuss appropriate initialization strategies, convergence and model selection criteria. We demonstrate the model’s applicability through simulation and two data analyses on observations that have many functional components. [ABSTRACT FROM AUTHOR]

In this paper, a hidden Markov model for modelling matrix‐variate time series data is developed. It relies on matrix‐variate distribution and presents a promising alternative to the existing methods. Simulation study is carefully conducted and uses benchmark tests with pre‐specified overlapping values. Compared with the existing methods, the proposed model demonstrates rather high accuracy in state classification. Results suggest that such an approach is indeed competitive. Interesting applications are presented for real‐life data illustration. [ABSTRACT FROM AUTHOR]

We propose a novel regularized mixture model for clustering matrix‐valued data. The proposed method assumes a separable covariance structure for each cluster and imposes a sparsity structure (eg, low rankness, spatial sparsity) for the mean signal of each cluster. We formulate the problem as a finite mixture model of matrix‐normal distributions with regularization terms, and then develop an expectation maximization type of algorithm for efficient computation. In theory, we show that the proposed estimators are strongly consistent for various choices of penalty functions. Simulation and two applications on brain signal studies confirm the excellent performance of the proposed method including a better prediction accuracy than the competitors and the scientific interpretability of the solution. [ABSTRACT FROM AUTHOR]

We introduce a novel framework for model-free variable selection with matrix-valued predictors. To test the importance of rows, columns, and submatrices of the predictor matrix in terms of predicting the response, three types of hypotheses are formulated under a unified framework. The asymptotic properties of the test statistics under the null hypothesis are established and a permutation testing algorithm is also introduced to approximate the distribution of the test statistics. A maximum ratio criterion (MRC) is proposed to facilitate the model-free variable selection. Unlike the traditional stepwise regression procedures that require calculating p-values at each step, the MRC is a noniterative procedure that does not require p-value calculation and is guaranteed to achieve variable selection consistency under mild conditions. Performance of the proposed method is evaluated in extensive simulations and demonstrated through the analysis of an electroencephalography data. for this article are available online. [ABSTRACT FROM AUTHOR]

Most nonrigid motions use shape-based methods to solve the problem; however, the use of discrete cosine transform trajectory-based methods to solve the nonrigid motion problem is also very prominent. The signal undergoes discrete transformation due to the transform characteristics of the discrete cosine transform. The correlation of the data is well extracted such that a better compression of data is achieved. However, it is important to select the number and sequence of discrete cosine transform trajectory basis appropriately. The error of reconstruction and operational costs will increase for a high value of K (number of trajectory basis). On the other hand, a lower value of K would lead to the exclusion of information components. This will lead to poor accuracy as the structure of the object cannot be fully represented. When the number of trajectory basis is determined, the combination form has a considerable influence on the reconstruction algorithm. This article selects an appropriate number and combination of trajectory basis by analyzing the spectrum of re-projection errors and realizes the automatic selection of trajectory basis. Then, combining with the probability framework of normal distribution of a low-order model matrix, the energy information of the high-frequency part is retained, which not only helps maintain accuracy but also improves reconstruction efficiency. The proposed method can be used to reconstruct the three-dimensional structure of sparse data under more precise prior conditions and lower computational costs. [ABSTRACT FROM AUTHOR]

Along with advances in technology, matrix data, such as medical/industrial images, have emerged in many practical fields. These data usually have high dimensions and are not easy to cluster due to their intrinsic correlated structure among rows and columns. Most approaches convert matrix data to multi dimensional vectors and apply conventional clustering methods to them, and thus, suffer from an extreme high-dimensionality problem as well as a lack of interpretability of the correlated structure among row/column variables. Recently, a regularized model was proposed for clustering matrix-valued data by imposing a sparsity structure for the mean signal of each cluster. We extend their approach by regularizing further on the covariance to cope better with the curse of dimensionality for large size images. A penalized matrix normal mixture model with lasso-type penalty terms in both mean and covariance matrices is proposed, and then an expectation maximization algorithm is developed to estimate the parameters. The proposed method has the competence of both parsimonious modeling and reflecting the proper conditional correlation structure. The estimators are consistent, and their limiting distributions are derived. We applied the proposed method to simulated data as well as real datasets and measured its clustering performance with the clustering accuracy (ACC) and the adjusted rand index (ARI). The experiment results show that the proposed method performed better with higher ACC and ARI than those of conventional methods.

Due to its flexibility and good restoration performance, the Expected Patch Log Likelihood (EPLL) method has attracted extensive attention and has been further developed. However, the basic EPLL method is mainly applied for gray image restoration. For color image denoising with different channel noise levels, concatenating the RGB values into a vector and applying the basic EPLL directly can produce false colors and artifacts. In this paper, a Multi-Channel Expected Patch Log Likelihood (MC-EPLL) method is proposed for color image denoising with different channel noise levels. Considering the within and between channel correlation, the noise model of the concatenated vector of RGB channels can be constructed as a Matrix Normal Distribution. Under the KL divergence framework, the MC-EPLL model can be derived by combining the noise model and Gaussian Mixture Model (GMM) based patch prior. Based on the half quadratic splitting (HQS) strategy, the MC-EPLL model is decomposed into two sub-minimization problems and the closed-form solution of each sub-problem can be obtained. Experiments show the feasibility and superiority of the proposed MC-EPLL over the compared denoising methods. [ABSTRACT FROM AUTHOR]

Summary: Matrix‐valued data, where the sampling unit is a matrix consisting of rows and columns of measurements, are emerging in numerous scientific and business applications. Matrix Gaussian graphical models are a useful tool to characterize the conditional dependence structure of rows and columns. We employ non‐convex penalization to tackle the estimation of multiple graphs from matrix‐valued data under a matrix normal distribution. We propose a highly efficient non‐convex optimization algorithm that can scale up for graphs with hundreds of nodes. We establish the asymptotic properties of the estimator, which requires less stringent conditions and has a sharper probability error bound than existing results. We demonstrate the efficacy of our proposed method through both simulations and real functional magnetic resonance imaging analyses. [ABSTRACT FROM AUTHOR]

We study the classification performance of Kronecker-structured (K-S) subpsace models in two asymptotic regimes and develop an algorithm for fast and compact K-S subspace learning for better classification and representation of multidimensional signals by exploiting the structure in the signal. First, we study the classification performance in terms of diversity order and pairwise geometry of the subspaces. We derive an exact expression for the diversity order as a function of the signal and subspace dimensions of a K-S model. Next, we study the classification capacity, the maximum rate at which the number of classes can grow as the signal dimension goes to infinity. Then, we describe a fast algorithm for Kronecker-structured learning of discriminative dictionaries (K-SLD $^2$). Finally, we evaluate the empirical classification performance of K-S models for the synthetic data, showing that they agree with the diversity order analysis. We also evaluate the performance of K-SLD $^2$ on synthetic and real-world datasets showing that the K-SLD $^2$ balances compact signal representation and good classification performance. [ABSTRACT FROM AUTHOR]

Recovering dynamic 3D structures from 2D image observations is highly under-constrained because of projection and missing data, motivating the use of strong priors to constrain shape deformation. In this paper, we empirically show that the spatiotemporal covariance of natural deformations is dominated by a Kronecker pattern. We demonstrate that this pattern arises as the limit of a spatiotemporal autoregressive process, and derive a Kronecker Markov Random Field as a prior distribution over dynamic structures. This distribution unifies shape and trajectory models of prior art and has the individual models as its marginals. The key assumption of the Kronecker MRF is that the spatiotemporal covariance is separable into the product of a temporal and a shape covariance, and can therefore be modeled using the matrix normal distribution. Analysis on motion capture data validates that this distribution is an accurate approximation with significantly fewer free parameters. Using the trace-norm, we present a convex method to estimate missing data from a single sequence when the marginal shape distribution is unknown. The Kronecker-Markov distribution, fit to a single sequence, outperforms state-of-the-art methods at inferring missing 3D data, and additionally provides covariance estimates of the uncertainty. [ABSTRACT FROM AUTHOR]

• An interpretable latent variable model for count matrix data is obtained through the Kronecker approach. • Estimators for the model parameters are developed and thoroughly studied and, using them, predictions for the latent variables are obtained. • An estimator for the latent dimensionality is proposed. • The method is illustrated and compared to existing methodology with several data examples. We develop a dimension reduction framework for data consisting of matrices of counts. Our model is based on the assumption of existence of a small amount of independent normal latent variables that drive the dependency structure of the observed data, and can be seen as the exact discrete analogue of a contaminated low-rank matrix normal model. We derive estimators for the model parameters and establish their limiting normality. An extension of a recent proposal from the literature is used to estimate the latent dimension of the model. The method is shown to outperform both its vectorization-based competitors and matrix methods assuming the continuity of the data distribution in analysing simulated data and real world abundance data. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, the multilinear normal distribution is introduced as an extension of the matrix-variate normal distribution. Basic properties such as marginal and conditional distributions, moments, and the characteristic function, are also presented. A trilinear example is used to explain the general contents at a simpler level. The estimation of parameters using a flip-flop algorithm is also briefly discussed. [Copyright &y& Elsevier]

Abstract: Motivated by analysis of gene expression data measured over different tissues or over time, we consider matrix-valued random variable and matrix-normal distribution, where the precision matrices have a graphical interpretation for genes and tissues, respectively. We present a penalized likelihood method and an efficient coordinate descent-based computational algorithm for model selection and estimation in such matrix normal graphical models (MNGMs). We provide theoretical results on the asymptotic distributions, the rates of convergence of the estimates and the sparsistency, allowing both the numbers of genes and tissues to diverge as the sample size goes to infinity. Simulation results demonstrate that the MNGMs can lead to a better estimate of the precision matrices and better identifications of the graph structures than the standard Gaussian graphical models. We illustrate the methods with an analysis of mouse gene expression data measured over ten different tissues. [Copyright &y& Elsevier]

In the first part of the paper, we introduce the matrix-variate generalized hyperbolic distribution by mixing the matrix normal distribution with the matrix generalized inverse Gaussian density. The p -dimensional generalized hyperbolic distribution of [Barndorff-Nielsen, O. (1978). Hyperbolic distributions and distributions on hyperbolae. Scand. J. Stat. , 5 , 151-157], the matrix- T distribution and many well-known distributions are shown to be special cases of the new distribution. Some properties of the distribution are also studied. The second part of the paper deals with the application of the distribution in the Bayesian analysis of the normal multivariate linear model. [ABSTRACT FROM AUTHOR]

Along with advances in technology, matrix data, such as medical/industrial images, have emerged in many practical fields. These data usually have high dimensions and are not easy to cluster due to their intrinsic correlated structure among rows and columns. Most approaches convert matrix data to multi dimensional vectors and apply conventional clustering methods to them, and thus, suffer from an extreme high-dimensionality problem as well as a lack of interpretability of the correlated structure among row/column variables. Recently, a regularized model was proposed for clustering matrix-valued data by imposing a sparsity structure for the mean signal of each cluster. We extend their approach by regularizing further on the covariance to cope better with the curse of dimensionality for large size images. A penalized matrix normal mixture model with lasso-type penalty terms in both mean and covariance matrices is proposed, and then an expectation maximization algorithm is developed to estimate the parameters. The proposed method has the competence of both parsimonious modeling and reflecting the proper conditional correlation structure. The estimators are consistent, and their limiting distributions are derived. We applied the proposed method to simulated data as well as real datasets and measured its clustering performance with the clustering accuracy (ACC) and the adjusted rand index (ARI). The experiment results show that the proposed method performed better with higher ACC and ARI than those of conventional methods. [ABSTRACT FROM AUTHOR]

Conventional simultaneous equation models assume that the error terms are serially independent. In some situations, data may present hierarchical or grouped structure and this assumption may be invalid. A new multivariate model referred as to Multilevel Simultaneous Equation Model (MSEM) is developed under this motivation. The maximum likelihood estimation of the parameters of an MSEM is considered. A matrix-valued distribution, namely, the matrix normal distribution, is introduced to incorporate an among-row and an among-column covariance matrix structure in the specification of the model. In the absence of an analytical solution of the system of likelihood equations, a general-purpose optimization solver is employed to obtain the maximum likelihood estimators. In a first approach to the solution of the problem, the adequacy of the matrix normal distribution is evaluated empirically in the case in which the double covariance structure is known. Using simulated data under the model assumptions, the performance of the maximum likelihood estimator (MLE) is assessed with regard to other conventional alternatives such as two-stage least squares estimator (2SLS). [ABSTRACT FROM AUTHOR]