Search

Multivariate regression is an important task in domains such as machine learning and statistics. We cast this regression problem as a linear system with a solution that is a vectorized symmetric tensor, which is assumed to be of low rank. We show that the structure of the data and the decomposition can be exploited to obtain efficient optimization methods. Furthermore, we show that an algebraic algorithm can be derived even if the number of given data points is low. We demonstrate good performance of our regression model using a real-life dataset from materials science.

A low multilinear rank approximation (LMLRA) of a tensor is often used to compress a large tensor into a much more compact form, while still maintaining most of the information in the tensor. This can be achieved by only storing the principal subspaces in every mode and dropping the other singular vectors. These subspaces are then combined using a core tensor of (much) smaller dimensions than the original tensor. When the tensor is non-static, for example when new tensor slices are regularly added in one mode, one can expect that the LMLRA of the tensor changes only slightly and an approximation of the new tensor can be derived from the previous one. In this paper, a method is derived to track the different subspaces of a tensor by generalizing the rank-adaptive SURV method of Zhou et al. for tracking matrix subspaces to higher orders. Using a sequential truncation approach, this leads to efficient and accurate updates of the LMLRA.

The advantages of tensor- over matrix-based methods have been recently demonstrated in the context of functional magnetic resonance imaging (fMRI) blind source unmixing. However, these methods rely on the assumption of a Gaussian distribution for the noise, which suggests a least squares criterion for the tensor decomposition. One can instead argue that a Rician model for the fMRI noise is much more accurate and hence alternative cost functions should also be investigated. In this paper, β-divergences are used to parametrize the Canonical Polyadic Decomposition (CPD) fitting to fMRI data and the effect of β on the source separation performance is evaluated, for different values of signal to noise ratio (SNR). Our results confirm that the commonly used squared error is not the best choice, particularly at low SNRs.

Many interesting matrix decompositions/factorizations, and especially many tensor decompositions, have to be solved by non-convex optimization-based algorithms, that may converge to local optima. Hence, when interpretability of the components is a requirement, practitioners have to compute the decomposition (e.g. CPD) many times, with different initializations, to verify whether the components are reproducible over repetitions of the optimization. However, it is non-trivial to assess such reliability or stability when multiple local optima are encountered. We propose an efficient algorithm that clusters the different repetitions of the decomposition according to the local optimum that they belong to, offering a diagnostic tool to practitioners. Our algorithm employs a graph-based representation of the decomposition, in which every repetition corresponds to a node, and similarities between components are encoded as edges. Clustering is then performed by exploiting a property known as cycle consistency, leading to a low-rank approximation of the graph. We demonstrate the applicability of our method on realistic electroencephalographic (EEG) data and synthetic data.

$\beta $-divergence cost functions generalize three popular cost functions for low-rank tensor approximation by interpolating between them: the least-squares (LS) distance, the Kullback-Leibler (KL) divergence and the Itakura-Saito (IS) divergence. For certain types of data and specific noise distributions, beta-divergence cost functions can lead to more meaningful low-rank approximations than those obtained with the LS cost function. Unfortunately, much of the low-rank structure that is heavily exploited in existing second-order LS methods, is no longer exploitable when moving to general $\beta $-divergences. In this paper, we show that, unlike in the general rank-$R$ case, rank-1 structure can still be exploited. We therefore propose an efficient method that uses second-order information to compute nonnegative rank-l approximations of tensors for general $\beta -$ divergence cost functions.

The fourth release of Tensorlab — a Matlab toolbox which bundles state-of-the-art tensor algorithms and tools — introduces a number of algorithms which allow a variety of new types of problems to be solved. For example, Gauss–Newton type algorithms for dealing with non-identical noise distributions or implicitly given tensors are discussed. To deal with large-scale datasets, incomplete tensors are combined with constraints, and updating techniques enable real-time tracking of time-varying tensors. A more robust algorithm for computing the decomposition in block terms is presented as well. To make tensor algorithms more accessible, graphical user interfaces for computing a decomposition in rank-1 terms or to compress a tensor are given.

The Canonical Polyadic Decomposition (CPD) is one of the most basic tensor models used in signal processing and machine learning. Despite its wide applicability, identifiability conditions and algorithms for CPD in cases where the tensor is incomplete are lagging behind its practical use. We first present a tensor-based framework for bilinear factorizations subject to monomial constraints, called monomial factorizations. Next, we explain that the CPD of a tensor that has missing fibers can be interpreted as a monomial factorization problem. Finally, using the monomial factorization interpretation, we show that CPD recovery conditions can be obtained that only rely on the observed fibers of the tensor.

© 2018 IEEE. By exploiting the intrinsic structure and/or sparsity of the system coefficients in large-scale system identification, one can enable efficient processing. In this paper, we employ this strategy for large-scale single-input multiple-output autoregressive system identification by assuming the coefficients can be well approximated by Kronecker products of smaller vectors. We show that the identification problem can be refor-mulated as the computation of a Kronecker product equation, allowing one to use optimization-based and algebraic solvers. ispartof: pages:1348-1352 ispartof: Proc. of the 2018 6th IEEE Global Conference on Signal and Information Processing pages:1348-1352 ispartof: 6th IEEE Global conference on Signal and information processing (GlobalSIP 2018) location:Anaheim, CA, USA date:26 Nov - 29 Nov 2018 status: published

© 2017 IEEE. Coupled matrix-tensor factorizations have proved to be a powerful tool for data fusion problems in a variety of applications. Uniqueness conditions for such coupled decompositions have only recently been reported, demonstrating that coupling through a common factor can ensure uniqueness beyond what is possible when considering separate decompositions. In view of the increasing interest in application scenarios involving more general notions of coupling, we revisit in this paper the uniqueness question for the important case where the factors common to the tensor and the matrix only share some of their columns. Related computational aspects and numerical examples are also discussed. ispartof: pages:711-715 ispartof: Proc. of the Asilomar Conference on Signals, Systems and Computers vol:2017-October pages:711-715 ispartof: Asilomar Conference on Signals, Systems and Computers location:Pacific Grove, California date:28 Oct - 30 Oct 2018 status: published

© 2018 IEEE. Atrial fibrillation (AF) is the most common cardiac arrhythmia, increasing the risk of a stroke substantially. Hence, early and accurate detection of AF is paramount. We present a matrix-and tensor-based method for AF detection in single-and multi-lead electrocardiogram (ECG) signals. First, the recordings are compressed into one heartbeat via the singular value decomposition (SVD). These representative heartbeats, single-lead, are collected in a matrix with modes time and recordings. In the multi-lead case, we obtain a tensor with modes lead, time and recording. By modeling the matrix (tensor) with a (multilinear) SVD, each recording, as well as new recordings, can be expressed by a coefficient vector. The comparison of a new coefficient vector with those of the model set results in morphological features, which are combined with heart rate variability information in a Support Vector Machine classifier to detect AF. The SVD-based method is tested on the 2017 PhysioNet/CinC Challenge dataset, resulting in an F1-score of 0.77. The multilinear SVD-based method is applied on the MIT-BIH AF IB and AF TDB dataset, resulting in a perfect separation. An advantage of our methods is the interpretability of the features, which is a key element in the application of automatic methods in clinical practice. ispartof: pages:799-805 ispartof: Proc. 52nd Asilomar Conference on Signals, systems and computers vol:2018-October pages:799-805 ispartof: 52nd Asilomar Conference on Signals, systems and computers location:Pacific Grove, CA, USA date:28 Oct - 1 Nov 2018 status: published

© 2018 IEEE. Tensor updating methods enable tensor decompositions to adapt quickly when new data is added to the tensor. At present, updating methods for the canonical polyadic decomposition (CPD) give every tensor entry the same weight. In practice, however, data quality or relative importance might differ between tensor entries, which warrants the use of more general weighting schemes. In this paper, an NLS updating method is developed for the CPD that uses a weighted least squares (WLS) approach with a low-rank weight tensor. This weight tensor itself can also be updated to allow dynamic weighting schemes. By exploiting the CPD structure of both the data and weight tensors, the algorithm obtains better accuracy than the unweighted updating methods, while being more time- and memory efficient than batch WLS methods. ispartof: pages:6368-6372 ispartof: Proc. of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing vol:2018-April pages:6368-6372 ispartof: ICASSP 2018 location:Calgary, Canada date:15 Apr - 20 Apr 2018 status: published

© 2018 IEEE. The classification of brain states using neural recordings such as electroencephalography (EEG) finds applications in both medical and non-medical contexts, such as detecting epileptic seizures or discriminating mental states in brain-computer interfaces, respectively. Although this endeavor is well-established, existing solutions are typically restricted to lab or hospital conditions because they operate on recordings from a set of EEG electrodes that covers the whole head. By contrast, a true breakthrough for these applications would be the deployment 'in the real world', by means of wearable devices that encompass just one (or a few) channels. Such a reduction of the available information inevitably makes the classification task more challenging. We tackle this issue by means of a multilinear subspace learning step (using data from multiple channels during training) and subsequently solving a regression problem with a low-rank structure to classify new trials (using data from only a single channel during testing). We demonstrate the feasibility of this approach on EEG data recorded during a mental arithmetic task. ispartof: IEEE 28th International Workshop on Machine Learning for Signal Processing (MLSP) vol:2018-September ispartof: 2018 IEEE 28TH INTERNATIONAL WORKSHOP ON MACHINE LEARNING FOR SIGNAL PROCESSING (MLSP) location:Aalborg, Denmark date:17 Sep - 20 Sep 2018 status: published

The canonical polyadic decomposition (CPD) is an important tensor tool in signal processing with various applications in blind source separation and sensor array processing. Many algorithms have been developed for the computation of a CPD using a least squares cost function. Standard least-squares methods assumes that the residuals are uncorrelated and have equal variances which is often not true in practice, rendering the approach suboptimal. Weighted least squares allows one to explicitly accommodate for general (co)variances in the cost function. In this paper, we develop a new nonlinear least-squares algorithm for the computation of a CPD using low-rank weights which enables efficient weighting of the residuals. We briefly illustrate our algorithm for direction-of-arrival estimation using an array of sensors with varying quality.

Various parameters influence face recognition such as expression, pose, and illumination. In contrast to matrices, tensors can be used to naturally accommodate for the different modes of variation. The multilinear singular value decomposition (MLSVD) then allows one to describe each mode with a factor matrix and the interaction between the modes with a coefficient tensor. In this paper, we show that each image in the tensor satisfying an MLSVD model can be expressed as a structured linear system called a Kronecker Product Equation (KPE). By solving a similar KPE for a new image, we can extract a feature vector that allows us to recognize the person with high performance. Additionally, more robust results can be obtained by using multiple images of the same person under different conditions, leading to a coupled KPE. Finally, our method can be used to update the database with an unknown person using only a few images instead of an image for each combination of conditions. We illustrate our method for the extended Yale Face Database B, achieving better performance than conventional methods such as Eigenfaces and other tensor-based techniques.

© 2017 EURASIP. Current batch tensor methods often struggle to keep up with fast-arriving data. Even storing the full tensors that have to be decomposed can be problematic. To alleviate these limitations, tensor updating methods modify a tensor decomposition using efficient updates instead of recomputing the entire decomposition when new data becomes available. In this paper, the structure of the decomposition is exploited to achieve fast updates for the canonical polyadic decomposition whenever new slices are added to the tensor in a certain mode. A batch NLS-algorithm is adapted so that it can be used in an updating context. By only storing the old decomposition and the new slice of the tensor, the algorithm is both time- and memory efficient. Experimental results show that the proposed method is faster than batch ALS and NLS methods, while maintaining a good accuracy for the decomposition. ispartof: pages:693-697 ispartof: Proc. of the 25th European Signal Processing Conference vol:2017-January pages:693-697 ispartof: EUSIPCO 2017 location:Kos, Greece date:Aug - Aug 2017 status: published

Cardiac arrhythmia or irregular heartbeats are an important feature to assess the risk on sudden cardiac death and other cardiac disorders. Automatic classification of irregular heartbeats is therefore an important part of ECG analysis. We propose a tensor-based method for single- and multi-channel irregular heartbeat classification. The method tensorizes the ECG data matrix by segmenting each signal beat-by-beat and then stacking the result into a third-order tensor with dimensions channel × time × heartbeat. We use the multilinear singular value decomposition to model the obtained tensor. Next, we formulate the classification task as the computation of a Kronecker Product Equation. We apply our method on the INCART dataset, illustrating promising results. ispartof: pages:438-441 ispartof: Proc. 39th Annual International Conference of the IEEE Engineering in Medicine & Biology Society vol:2017 pages:438-441 ispartof: EMBC 2017 location:JeJu Island, South Korea date:Jul - Jul 2017 status: published

We give an overview of recent developments in numerical optimization-based computation of tensor decompositions that have led to the release of Tensorlab 3.0 in March 2016 (www.tensorlab.net). By careful exploitation of tensor product structure in methods such as quasi-Newton and nonlinear least squares, good convergence is combined with fast computation. A modular approach extends the computation to coupled factorizations and structured factors. Given large datasets, different compact representations (polyadic, Tucker,…) may be obtained by stochastic optimization, randomization, compressed sensing, etc. Exploiting the representation structure allows us to scale the algorithms for constrained/coupled factorizations to large problem sizes.

© 2016 IEEE. A new method for the blind identification of largescale finite impulse response (FIR) systems is presented. It exploits the fact that the system coefficients in large-scale problems often depend on much fewer parameters than the total number of entries in the coefficient vectors. We use low-rank models to compactly represent matricized versions of these compressible system coefficients.We show that blind system identification (BSI) then reduces to the computation of a structured tensor decomposition by using a deterministic tensorization technique called segmentation on the observed outputs. This careful exploitation of the low-rank structure enables the unique identification of both the system coefficients and the inputs. The approach does not require the input signals to be statistically independent. ispartof: pages:2015-2019 ispartof: Proc. 24th European Signal Processing Conference vol:2016-November pages:2015-2019 ispartof: EUSIPCO 2016 location:Budapest, Hungary date:Sep - Sep 2016 status: published

© 2016 IEEE. Tensors have proven to be useful tools for array processing. Most attention has been paid to separable arrays, which lead to a Canonical Polyadic Decomposition (CPD). For more general geometries, and in particular for sparse arrays and arrays with missing sensors, more general tensor methods are required. The recently proposed coupled CPD framework allows a data fission/fusion approach in which one zooms in on partial structures and combines the partial CPDs through which the latter are imposed. This approach yields explicit algebraic conditions under which the solution is unique. The exact solution can be found with a matrix eigenvalue decomposition in the noiseless case, similar to ESPRIT in the case of uniform linear arrays. We study in detail the case of sparse spatial sampling where sensors are located on points of a two-dimensional grid. Despite the fact that the array is incomplete, coupled CPD allows us to exploit the rectangularity of the grid as well as the uniformity of the spatial sampling in both dimensions. ispartof: pages:1-5 ispartof: Proc. of the Ninth IEEE Sensor Array and Multichannel Signal Processing Workshop vol:2016-September pages:1-5 ispartof: Ninth IEEE Sensor Array and Multichannel Signal Processing Workshop location:Rio de Janeiro, Brazil date:Jul - Jul 2016 status: published

© 2016 IEEE. Coupled decompositions of multiple tensors are fundamental tools for multi-set data fusion. In this paper, we introduce a coupled version of the rank-(Lm, Ln, •) block term decomposition (BTD), applicable to joint independent subspace analysis. We propose two algorithms for its computation based on a coupled block simultaneous generalized Schur decomposition scheme. Numerical results are given to show the performance of the proposed algorithms. ispartof: pages:2554-2558 ispartof: Proc. 2016 IEEE International Conference on Acoustics, Speech and Signal Processing vol:2016-May pages:2554-2558 ispartof: ICASSP 2016 location:Shanghai, China date:Mar - Mar 2016 status: published

A novel deterministic blind signal separation technique for separating signals into rational functions is proposed, applicable in various situations. This new technique is based on a tensorization of the observed data matrix into a set of Lowner matrices. The obtained tensor can then be decomposed with a block tensor decomposition, resulting in a unique separation into rational functions under mild conditions. This approach provides a viable alternative to independent component analysis (ICA) in cases where the independence assumption is not valid or where the sources can be modeled well by rational functions, such as frequency spectra. In contrast to ICA, this technique is deterministic and not based on statistics, and therefore works well even with a small number of samples.

The ESPRIT method is a classical method for one-dimensional harmonic retrieval. During the past two decades it has become apparent that several applications in signal processing correspond to the less studied Multi-dimensional Harmonic Retrieval (MHR) problem. In order to accommodate this demand, we propose an extension of ESPRIT to MHR based on the coupled canonical polyadic decomposition. This leads to a dedicated uniqueness condition and an algebraic framework for MHR. © 2014 IEEE. ispartof: pages:441-444 ispartof: 2014 IEEE 8TH SENSOR ARRAY AND MULTICHANNEL SIGNAL PROCESSING WORKSHOP (SAM) pages:441-444 ispartof: 8th IEEE sensor array and multichannel signal processing worshop location:A Courna, Spain date:22 Jun - 25 Jun 2014 status: published

For the case of a single colocated receive antenna array and additional linear diversity (e.g. oversampling or polarization), tensor decomposition based signal separation is now well-established. For increasing the spatial diversity of communication systems, the use of several widely separated colocated antenna arrays has been suggested. However, for such problems no algebraic framework has been proposed. We explain that recently developed coupled tensor decompositions provide such a framework. In particular, we explain that the use of several widely separated colocated antenna arrays may lead to improved identifiability results.

Tensor decompositions involving a Vandermonde factor are common in signal processing. For instance, they show up in sensor array processing and in wireless communication. We illustrate that by simultaneously taking the tensor nature and the Vandermonde structure of the problem into account new uniqueness results and numerical methods for computing a tensor decomposition with Vandermonde structure can be obtained.

Tensor decompositions with Toeplitz or block-Toeplitz structure are common in signal processing. For instance, they show up in blind system identification and deconvolution. We illustrate that by simultaneously taking the tensor nature and the block-Toeplitz structure of the problem into account new uniqueness results and numerical methods for computing a tensor decomposition with block-Toeplitz structure can be obtained.

In signal processing several problems have been formulated as Simultaneous Generalized Schur Decomposition (SGSD) problems. Applications are found in blind source separation and multidimensional harmonic retrieval. Furthermore, SGSD methods for computing a third-order Canonical Polyadic (CP) decomposition have been proposed. The original SGSD method requires that all three matrix factors of the CP decomposition have full column rank. We first propose a new version of the SGSD method for computing a third-order CP decomposition. The proposed method mainly differs from the existing method in the way the triangular matrices are computed. Second, we propose an alternative SGSD method which only requires that two of the matrix factors of the CP decomposition have full column rank.

This contribution studies the problem of learning sparse, nonparametric models from observations drawn from an arbitrary, unknown distribution. This specific problem leads us to an algorithm extending techniques for Multiple Kernel Learning (MKL), functional ANOVA models and the Component Selection and Smoothing Operator (COSSO). The key element is to use a data-dependent regularization scheme adapting to the specific distribution underlying the data. We then present empirical evidence supporting the proposed learning algorithm.

Blind deterministic receivers for DS-CDMA systems based on the PARAFAC model have been proposed in several papers since their conception in [1]. In many cases, the transmitted signals can be considered uncorrelated. Hence, we develop PARAFAC receivers for uncorrelated signals. We introduce several numerical algorithms for orthogonality constrained PARAFAC on which receivers for uncorrelated signals can be based. Simulation results show an increase in performance when the PARAFAC receiver takes the uncorrelatedness of the transmitted signals into account.

In this paper, we show that the Block Component Decomposition in rank-(L,L,1) terms of a third-order tensor, referred to as BCD-(L,L,1), can be reformulated as a Joint Block Diagonalization (JBD) problem, provided that certain assumptions on the dimensions are satisfied. This JBD-based reformulation leads to a new uniqueness bound for the BCD-(L,L,1). We also propose a closed-form solution to solve exact JBD problems. For approximate JBD problems, this closed-form solution yields a good starting value for iterative optimization algorithms. The performance of our technique is illustrated by its application to blind CDMA signal separation.

Matrix decompositions have always been at the heart of signal, circuit and system theory. In particular, the Singular Value Decomposition (SVD) has been an important tool. There is currently a shift of paradigm in the algebraic foundations of these fields. Quite recently, Nonnegative Matrix Factorization (NMF) has been shown to outperform SVD at a number of tasks. Increasing research efforts are spent on the study and application of decompositions of higher-order tensors or multi-way arrays. This paper is a partial survey on tensor generalizations of the SVD and their applications. We also touch on Nonnegative Tensor Factorizations.

We address the problem of blind identification of a convolutive Multiple-Input Multiple-Output (MIMO) system with more inputs than outputs, and in particular, the 3-input 2-output case. We assume that the inputs are temporally white, non-Gaussian distributed, spatially independent and that the system impulse response can be complex. In this paper, we look at the problem in the frequency domain, where, for each frequency we construct two tensors based on cross-polyspectra of the output. These tensors lead to the system frequency response within frequency dependent scaling and permutation ambiguities. We propose ways to resolve these ambiguities, and show that it is possible to obtain the system response within a scalar and a linear phase.