Search

In this paper, we propose a new adaptive cross algorithm for computing a low tubal rank approximation of third-order tensors, with less memory usage and lower computational complexity than the truncated tensor SVD (t-SVD). This makes it applicable for decomposing large-scale tensors. We conduct numerical experiments on synthetic and real-world datasets to confirm the efficiency and feasibility of the proposed algorithm. The simulation results show more than one order of magnitude acceleration in the computation of low tubal rank approximations for large-scale tensors. An application to pedestrian attribute recognition is also presented. [ABSTRACT FROM AUTHOR]

Introduction: The adrenal gland is a frequent site of metastases in different types of cancer. The aim of this study was to assess the results of metastatic adrenalectomy in a single institution and to identify factors for survival. Material and methods: A retrospective, single-centre analysis of outcomes of 39 patients (22 male, 17 female) with adrenal metastases who underwent surgery within 14 years (2004- 2017) was performed. The median age at the time of adrenal surgery was 64.8 years (range 49-79 years). Results: In our study group non-small cell lung cancer (NSCLC) was the most frequent primary tumour type (15 pts), followed by renal cell carcinoma (RCC) (14 pts) and colon cancer (6 pts). Most of the metastases - 36 (92%) - were metachronous (> 6 months). All synchronous metastases were NSCLC. The mean time from primary cancer to adrenalectomy was 42.3 months (range 1-176) and was statistically longer for RCC. In 3 patients (8%) metastases were bilateral and both adrenal glands were removed. In all patients, surgery was limited to the adrenal gland, and no major complications of surgery were observed. The median overall survival after metastasectomy was 18 months (3-81) and was statistically longer for colon cancer - 29.5 months (p = 0.012). In patients who died, tumours were significantly bigger than in survivors, 76.5 mm vs. 52.5 mm (p = 0.026). Conclusions: Surgery for adrenal metastasis is safe and indications for this procedure should be individualized. In selected patients, surgical removal of adrenal metastasis was associated with longer survival. [ABSTRACT FROM AUTHOR]

The adrenal glands are one of the most prevalent sites for metastases from various malignancies and metastases are indeed the most frequent malignant tumors of the adrenals. The identification of incidental adrenal lesions, of which a high proportion are metastases, have increased since surveillance protocols in cancer treated patients were widely introduced. More sensitive and reliable methods of diagnostic imaging leads to earlier detection of adrenal metastases. If there are no signs and symptoms of disseminated malignant disease, it is essential to differentiate adrenal lesions. CT imaging diagnostics and MRI play the main role in differential diagnosis. Adrenal biopsy is not recommended and is of limited diagnostic value. Appropriate selection of patients with adrenal metastases for curative treatment is a great challenge. Hormonal evaluation should be performed prior to treatment. Adrenalectomy seems to be a reasonable treatment for isolated adrenal metastasis in suitable patients. Minimally invasive laparoscopic adrenalectomy is as effective as the open approach with proven reduction in postoperative pain, morbidity and length of stay. Nonsurgical treatment such as stereotactic body radiation therapy (SBRT), radiotherapy with CyberKnife, percutaneous radiofrequency ablation, percutaneous microwave ablation, radiofrequency plus chemoembolization combined, have been reported with curative and palliative intent and have shown varied results. [ABSTRACT FROM AUTHOR]

This paper presents the neural dynamical network to compute a best rank-one approximation of a real-valued tensor. We implement the neural network model by the ordinary differential equations (ODE), which is a class of continuous-time recurrent neural network. Several new properties of solutions for the neural network are established. We prove that the locally asymptotic stability of solutions for ODE by constructive an appropriate Lyapunov function under mild conditions. Furthermore, we also discuss how to use the proposed neural networks for solving the tensor eigenvalue problem including the tensor H-eigenvalue problem, the tensor Z-eigenvalue problem, and the generalized eigenvalue problem with symmetric-definite tensor pairs. Finally, we generalize the proposed neural networks to the computation of the restricted singular values and the associated restricted singular vectors of real-valued tensors. We illustrate and validate theoretical results via numerical simulations. [ABSTRACT FROM AUTHOR]

We propose a new method for low-rank approximation of Moore-Penrose pseudo-inverses of large-scale matrices using tensor networks. The computed pseudoinverses can be useful for solving or preconditioning of large-scale overdetermined or underdetermined systems of linear equations. The computation is performed efficiently and stably based on the modified alternating least squares scheme using low-rank tensor train (TT) decompositions and tensor network contractions. The formulated large-scale optimization problem is reduced to sequential smaller-scale problems for which any standard and stable algorithms can be applied. A regularization technique is incorporated in order to alleviate ill-posedness and obtain robust low-rank approximations. Numerical simulation results illustrate that the regularized pseudoinverses of a wide class of nonsquare or nonsymmetric matrices admit good approximate low-rank TT representations. Moreover, we demonstrated that the computational cost of the proposed method is only logarithmic in the matrix size given that the TT ranks of a data matrix and its approximate pseudoinverse are bounded. It is illustrated that a strongly nonsymmetric convection-diffusion problem can be efficiently solved by using the preconditioners computed by the proposed method. [ABSTRACT FROM AUTHOR]

We propose new algorithms for singular value decomposition (SVD) of large-scale matrices based on a low-rank tensor approximation technique called the tensor train (TT) format. The proposed algorithms can compute a few extreme (i.e., largest or smallest) singular values and corresponding singular vectors for large-scale structured matrices given in a TT format. The computational complexity of the proposed methods scales logarithmically with the matrix size under the assumption that both the matrix and the singular vectors admit approximate low-rank TT decompositions. The proposed methods, which are called the alternating least squares SVD (ALS-SVD) and modified alternating least squares SVD (MALS-SVD), compute the left and right singular vectors approximately through block TT decompositions. A large-scale optimization problem is reduced to sequential small-scale optimization problems, and each core tensor of the block TT decompositions can be updated by applying any standard SVD method. The optimal ranks of the block TT decompositions are determined adaptively during the iteration process, so that we can achieve high approximation accuracy. Extensive numerical simulations are conducted for several types of TT-structured matrices such as the Hilbert matrix, a random matrix with prescribed singular values, Hankel and Toeplitz matrices, and tridiagonal matrices. The proposed methods are also applied for computing stationary solutions of a partial differential equation. The experimental results demonstrate the effectiveness of the proposed methods in estimating a few extreme singular values and vectors, compared with the performance of standard SVD algorithms and TT-based algorithms developed for symmetric eigenvalue decomposition. [ABSTRACT FROM AUTHOR]

Compressed sensing ( CS) comprises a set of relatively new techniques that exploit the underlying structure of data sets allowing their reconstruction from compressed versions or incomplete information. CS reconstruction algorithms are essentially nonlinear, demanding heavy computation overhead and large storage memory, especially in the case of multidimensional signals. Excellent review papers discussing CS state-of-the-art theory and algorithms already exist in the literature, which mostly consider data sets in vector forms. In this paper, we give an overview of existing techniques with special focus on the treatment of multidimensional signals (tensors). We discuss recent trends that exploit the natural multidimensional structure of signals (tensors) achieving simple and efficient CS algorithms. The Kronecker structure of dictionaries is emphasized and its equivalence to the Tucker tensor decomposition is exploited allowing us to use tensor tools and models for CS. Several examples based on real world multidimensional signals are presented, illustrating common problems in signal processing such as the recovery of signals from compressed measurements for magnetic resonance imaging ( MRI) signals or for hyper-spectral imaging, and the tensor completion problem (multidimensional inpainting). WIREs Data Mining Knowl Discov 2013, 3:355-380. doi: 10.1002/widm.1108 Conflict of interest: The authors have declared no conflicts of interest for this article. For further resources related to this article, please visit the WIREs website. [ABSTRACT FROM AUTHOR]

We propose a generalized method of the canonical correlation analysis using Alpha-Beta divergence, called AB-canonical analysis (ABCA). From observations of two random variables, x ε RP and y ε RQ, ABCA finds directions, wx ε RP and wy ε RQ, such that the AB-divergence between the joint distribution of (wT xx;wTyy) and the product of their marginal distributions is maximized. The number of significant non-zero canonical coefficients are determined by using a sequential permutation test. The advantage of our method over the standard canonical correlation analysis (CCA) is that it can reconstruct the hidden non-linear relationship between wTxx and wT yy, and it is robust against outliers. We extend ABCA when data are observed in terms of tensors. We further generalize this method by imposing sparseness constraints. Extensive simulation study is performed to justify our approach. [ABSTRACT FROM AUTHOR]

Recently there has been great interest in sparse representations of signals under the assumption that signals (data sets) can be well approximated by a linear combination of few elements of a known basis (dictionary). Many algorithms have been developed to find such representations for one-dimensional signals (vectors), which requires finding the sparsest solution of an underdetermined linear system of algebraic equations. In this letter, we generalize the theory of sparse representations of vectors to multiway arrays (tensors)--signals with a multidimensional structure--by using the Tucker model. Thus, the problem is reduced to solving a large-scale underdetermined linear system of equations possessing a Kronecker structure, for which we have developed a greedy algorithm, Kronecker-OMP, as a generalization of the classical orthogonal matching pursuit (OMP) algorithm for vectors. We also introduce the concept of multiway block-sparse representation of N-way arrays and develop a new greedy algorithm that exploits not only the Kronecker structure but also block sparsity. This allows us to derive a very fast and memory efficient algorithm called N-BOMP (N-way block OMP).We theoretically demonstrate that under the block-sparsity assumption, our N-BOMP algorithm not only has a considerably lower complexity but is also more precise than the classic OMP algorithm. Moreover, our algorithms can be used for very large-scale problems, which are intractable using standard approaches. We provide several simulations illustrating our results and comparing our algorithms to classical algorithms such as OMP and BP (basis pursuit) algorithms. We also apply the N-BOMP algorithm as a fast solution for the compressed sensing (CS) problem with large-scale data sets, in particular, for 2D compressive imaging (CI) and 3D hyperspectral CI, and we show examples with real-world multidimensional signals. [ABSTRACT FROM AUTHOR]

Nonnegative matrix factorization (NMF) algorithms often suffer from slow convergence speed due to the nonnegativity constraints, especially for large-scale problems. Low-rank approximation methods such as principle component analysis (PCA) are widely used in matrix factorizations to suppress noise, reduce computational complexity and memory requirements. However, they cannot be applied to NMF directly so far as they result in factors with mixed signs. In this paper, low-rank approximation is introduced to NMF (named lraNMF), which is not only able to reduce the computational complexity of NMF algorithms significantly, but also suppress bipolar noise. In fact, the new update rules are typically about M/R times faster than traditional ones of NMF, here M is the number of observations and R is the low rank of latent factors. Therefore lraNMF is particularly efficient in the case where R\ll M, which is the general case in NMF. The proposed update rules can also be incorporated into most existing NMF algorithms straightforwardly as long as they are based on Euclidean distance. Then the concept of lraNMF is generalized to the tensor field to perform a fast sequential nonnegative Tucker decomposition (NTD). By applying the proposed methods, the practicability of NMF/NTD is significantly improved. Simulations on synthetic and real data show the validity and efficiency of the proposed approaches. [ABSTRACT FROM AUTHOR]

Alternative least squares (ALS) algorithm is considered as a 'work-horse' algorithm for general tensor factorizations. A common form of this algorithm for nonnegative tensor factorizations (NTF) is always combined with a nonlinear projection (rectifier) to enforce nonnegative entries during the estimation. Such simple modification often provides acceptable results for general data. However, this does not establish an appropriate ALS algorithm for NTF. This kind of ALS algorithm often converges slowly, or cannot converge to the desired solution, especially for collinear data. To this end, in this paper, we reinvestigate the nonnegative quadratic programming, propose a recursive method for solving this problem. Then, we formulate a novel ALS algorithm for NTF. The validity and high performance of the proposed algorithm has been confirmed for difficult benchmarks, and also in an application of object classification, and analysis of EEG signals. [ABSTRACT FROM AUTHOR]

The paper reports our research aiming at utilization of human interactive communication modeling principles in application to a novel interaction paradigm designed for brain-computer/machine-interfacing (BCI/BMI) technologies as well as for socially aware intelligent environments or communication support systems. Automatic procedures for human affective responses or emotional states estimation are still a hot topic of contemporary research. We propose to utilize human brain and bodily physiological responses for affective/emotional as well as communicative interactivity estimation, which potentially could be used in the future for human-machine/environment interaction design. As a test platform for such an intelligent human-machine communication application, an emotional stimuli paradigm was chosen to evaluate brain responses to various affective stimuli in an emotional empathy mode. Videos with moving faces expressing various emotional displays as well as speech stimuli with similarly emotionally articulated sentences are presented to the subjects in order to further analyze different affective responses. From information processing point of view, several challenges with multimodal signal conditioning and stimuli dynamic response extraction in time frequency domain are addressed. Emotions play an important role in human daily life and human-to-human communication. This is why involvement of affective stimuli principles to human-machine communication or machine-mediated communication with utilization of multichannel neurophysiological and periphery physiological signals monitoring techniques, allowing real-time subjective brain responses evaluation, is discussed. We present our preliminary results and discuss potential applications of brain/body affective responses estimation for future interactive/smart environments. [ABSTRACT FROM AUTHOR]

Abstract: Analysis of high dimensional data in modern applications, such as neuroscience, text mining, spectral analysis, chemometrices naturally requires tensor decomposition methods. The Tucker decompositions allow us to extract hidden factors (component matrices) with different dimension in each mode, and investigate interactions among various modalities. The alternating least squares (ALS) algorithms have been confirmed effective and efficient in most of tensor decompositions, especially Tucker with orthogonality constraints. However, for nonnegative Tucker decomposition (NTD), standard ALS algorithms suffer from unstable convergence properties, demand high computational cost for large scale problems due to matrix inverse, and often return suboptimal solutions. Moreover they are quite sensitive with respect to noise, and can be relatively slow in the special case when data are nearly collinear. In this paper, we propose a new algorithm for nonnegative Tucker decomposition based on constrained minimization of a set of local cost functions and hierarchical alternating least squares (HALS). The developed NTD-HALS algorithm sequentially updates components, hence avoids matrix inverse, and is suitable for large-scale problems. The proposed algorithm is also regularized with additional constraint terms such as sparseness, orthogonality, smoothness, and especially discriminant. Extensive experiments confirm the validity and higher performance of the developed algorithm in comparison with other existing algorithms. [Copyright &y& Elsevier]

Abstract: Parallel factor analysis (PARAFAC) is a tensor (multiway array) factorization method which allows to find hidden factors (component matrices) from a multidimensional data. Most of the existing algorithms for the PARAFAC, especially the alternating least squares (ALS) algorithm need to compute Khatri–Rao products of tall factors and multiplication of large matrices, and due to this require high computational cost and large memory and are not suitable for very large-scale-problems. Hence, PARAFAC for large-scale data tensors is still a challenging problem. In this paper, we propose a new approach based on a modified ALS algorithm which computes Hadamard products, instead Khatri–Rao products, and employs relatively small matrices. The new algorithms are able to process extremely large-scale tensors with billions of entries. Extensive experiments confirm the validity and high performance of the developed algorithm in comparison with other well-known algorithms. [Copyright &y& Elsevier]

We propose a class of multiplicative algorithms for Nonnegative Matrix Factorization (NMF) which are robust with respect to noise and outliers. To achieve this, we formulate a new family generalized divergences referred to as the Alpha-Beta-divergences (AB-divergences), which are parameterized by the two tuning parameters, alpha and beta, and smoothly connect the fundamental Alpha-, Beta- and Gamma-divergences. By adjusting these tuning parameters, we show that a wide range of standard and new divergences can be obtained. The corresponding learning algorithms for NMF are shown to integrate and generalize many existing ones, including the Lee-Seung, ISRA (Image Space Reconstruction Algorithm), EMML (Expectation Maximization Maximum Likelihood), Alpha-NMF, and Beta-NMF. Owing to more degrees of freedom in tuning the parameters, the proposed family of AB-multiplicative NMF algorithms is shown to improve robustness with respect to noise and outliers. The analysis illuminates the links of between AB-divergence and other divergences, especially Gamma- and Itakura-Saito divergences. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, we provide two generalizations of the CUR matrix decomposition (also known as pseudo-skeleton approximation method ) to the case of -way arrays (tensors). These generalizations, which we called Fiber Sampling Tensor Decomposition types 1 and 2 (FSTD1 and FSTD2), provide explicit formulas for the parameters of a rank- Tucker representation (the core tensor of size and the matrix factors of sizes , ) based only on some selected entries of the original tensor. FSTD1 uses -mode fibers of the original tensor while FSTD2 uses exactly fibers in each mode as matrix factors, as suggested by the existence theorem provided in Oseledets et al. (2008) , with a core tensor defined in terms of the entries of a subtensor of size . For our results are reduced to the already known CUR matrix decomposition where the core matrix is defined as the inverse of the intersection submatrix, i.e. . Additionally, we provide an adaptive type algorithm for the selection of proper fibers in the FSTD1 model which is useful for large scale applications. Several numerical results are presented showing the performance of our FSTD1 Adaptive Algorithm compared to two recently proposed approximation methods for 3-way tensors. [Copyright &y& Elsevier]

In this paper, we extend and overview wide families of Alpha-, Beta- and Gamma-divergences and discuss their fundamental properties. In literature usually only one single asymmetric (Alpha, Beta or Gamma) divergence is considered. We show in this paper that there exist families of such divergences with the same consistent properties. Moreover, we establish links and correspondences among these divergences by applying suitable nonlinear transformations. For example, we can generate the Beta-divergences directly from Alpha-divergences and vice versa. Furthermore, we show that a new wide class of Gamma-divergences can be generated not only from the family of Beta-divergences but also from a family of Alpha-divergences. The paper bridges these divergences and shows also their links to Tsallis and Rényi entropies. Most of these divergences have a natural information theoretic interpretation. [ABSTRACT FROM AUTHOR]

In this letter, we propose a new algorithm for estimating sparse nonnegative sources from a set of noisy linear mixtures. In particular, we consider difficult situations with high noise levels and more sources than sensors (underdetermined case). We show that when sources are very sparse in time and overlapped at some locations, they can be recovered even with very low signal-to-noise ratio, and by using many fewer sensors than sources. A theoretical analysis based on Bayesian estimation tools is included showing strong connections with algorithms in related areas of research such as ICA, NMF, FOCUSS, and sparse representation of data with overcomplete dictionaries. Our algorithm uses a Bayesian approach by modeling sparse signals through mixed-state random variables. This new model for priors imposes ℓ0 norm-based sparsity. We start our analysis for the case of nonoverlapped sources (1-sparse), which allows us to simplify the search of the posterior maximum avoiding a combinatorial search. General algorithms for overlapped cases, such as 2-sparse and k-sparse sources, are derived by using the algorithm for 1-sparse signals recursively. Additionally, a combination of our MAP algorithm with the NN-KSVD algorithm is proposed for estimating the mixing matrix and the sources simultaneously in a real blind fashion. A complete set of simulation results is included showing the performance of our algorithm. [ABSTRACT FROM AUTHOR]

Abstract: Nonnegative matrix factorization (NMF) seeks a decomposition of a nonnegative matrix into a product of two nonnegative factor matrices and , such that a discrepancy between and is minimized. Assuming in the decomposition (for ), kernel NMF (KNMF) is easily derived in the framework of least squares optimization. In this paper we make use of KNMF to extract discriminative spectral features from the time–frequency representation of electroencephalogram (EEG) data, which is an important task in EEG classification. Especially when KNMF with linear kernel is used, spectral features are easily computed by a matrix multiplication, while in the standard NMF multiplicative update should be performed repeatedly with the other factor matrix fixed, or the pseudo-inverse of a matrix is required. Moreover in KNMF with linear kernel, one can easily perform feature selection or data selection, because of its sparsity nature. Experiments on two EEG datasets in brain computer interface (BCI) competition indicate the useful behavior of our proposed methods. [Copyright &y& Elsevier]

Using the convolutive nonnegative matrix factorization (NMF) model due to Smaragdis, we develop a novel algorithm for matrix decomposition based on the squared Euclidean distance criterion. The algorithm features new formally derived learning rules and an efficient update for the reconstructed nonnegative matrix. Performance comparisons in terms of computational load and audio onset detection accuracy indicate the advantage of the Euclidean distance criterion over the Kullback-Leibler divergence criterion. [ABSTRACT FROM AUTHOR]

Abstract: A two-stage clustering-then--optimization approach has been often used for sparse component analysis (SCA). The first challenging task of this approach is to estimate the basis matrix by cluster analysis. In this paper, a robust K-hyperline clustering (K-HLC) algorithm is developed for this task. The novelty of our method is that it is not only able to implement hyperline clustering, but also is capable of detecting the number of hidden hyperlines (or sparse components). K-HLC seamlessly integrates “the hyperline clustering” and “hyperline number detection” in the same algorithm. In addition, three strategies are proposed to tackle this problem: (1) reject the outliers by overestimating the number of hyperlines; (2) escape from local minima by using a multilayer initialization and (3) suppress the noise by a multilayer K-HLC. By taking these strategies into account, the robust K-HLC procedure can be briefly described as follows: first, we overestimate the number of hyperlines; then, a confidence index is given to evaluate the significance of each hyperline. Subsequently, we determine the number of hyperlines by checking the gap in the sorted confidence indices. Moreover, we select those hyperlines corresponding to large confidence indices with high rank priority and remove spurious ones with small confidence indices. The high performance of our clustering scheme is illustrated by extensive numerical experiments including some challenging benchmarks, e.g., very ill-conditioned basis matrix (Hilbert matrix), or the observations with strong outliers. [Copyright &y& Elsevier]

FOCal Underdetermined System Solver (FOCUSS) is a powerful tool for sparse representation and underdetermined inverse problems. In this correspondence, we strengthen the FOCUSS method with the following main contributions: 1) we give a more rigorous derivation of the FOCUSS for the sparsity parameter 0 < p < 1 by a nonlinear transform and 2) we develop the CG-FOCUSS by incorporating the conjugate gradient (CG) method to the FOCUSS, which significantly reduces a computational cost with respect to the standard FOCUSS and extends its availability for large scale problems. We justify the CG-FOCUSS based on a probability theory. Furthermore, the high performance of the CG-FOCUSS is demonstrated with experiments. [ABSTRACT FROM AUTHOR]

This paper discusses the estimation and numerical calculation of the probability that the 0-norm and 1-norm solutions of underdetermined linear equations are equivalent in the case of sparse representation. First, we define the sparsity degree of a signal. Two equivalence probability estimates are obtained when the entries of the 0-norm solution have different sparsity degrees. One is for the case in which the basis matrix is given or estimated, and the other is for the case in which the basis matrix is random. However, the computational burden to calculate these probabilities increases exponentially as the number of columns of the basis matrix increases. This computational complexity problem can be avoided through a sampling method. Next, we analyze the sparsity degree of mixtures and establish the relationship between the equivalence probability and the sparsity degree of the mixtures. This relationship can be used to analyze the performance of blind source separation (BSS). Furthermore, we extend the equivalence probability estimates to the small noise case. Finally, we illustrate how to use these theoretical results to guarantee a satisfactory performance in underdetermined BSS. [ABSTRACT FROM AUTHOR]

The FOCal Underdetermined System Solver (FOCUSS) algorithm has already found many applications in signal processing and data analysis, whereas the regularized M-FOCUSS algorithm has been recently proposed by Cotter et a!. for finding sparse solutions to an underdetermined system of linear equations with multiple measurement vectors. In this paper, we propose three modifications to the M-FOCUSS algorithm to make it more efficient for sparse and locally smooth solutions. First, motivated by the simultaneously autoregressive (SAR) model, we incorporate an additional weighting (smoothing) matrix into the Tikhonov regularization term. Next, the entire set of measurement vectors is divided into blocks, and the solution is updated sequentially, based on the overlapping of data blocks. The last modification is based on an alternating minimization technique to provide data-driven (simultaneous) estimation of the regularization parameter with the generalized cross-validation (GCV) approach. Finally, the simulation results demonstrating the benefits of the proposed modifications support the analysis. [ABSTRACT FROM AUTHOR]

The article discusses brain-computer interfaces (BCIs). BCIs enable persons to control external devices via brain waves. A BCI comprises three main functions--the recording and interpretation of waves, and translation of the interpretation into action. Topics covered by the authors include basic-signal-processing blocks, the four fundamental BCI paradigms, and the current state of BCIs and possible future directions. Special attention is paid to models for decomposing nth-order tensors that arise from brain-wave data.

Abstract: Non-negative matrix factorization (NMF) is a popular technique for pattern recognition, data analysis, and dimensionality reduction, the goal of which is to decompose non-negative data matrix into a product of basis matrix and encoding variable matrix with both and allowed to have only non-negative elements. In this paper, we consider Amari’s α-divergence as a discrepancy measure and rigorously derive a multiplicative updating algorithm (proposed in our recent work) which iteratively minimizes the α-divergence between and . We analyze and prove the monotonic convergence of the algorithm using auxiliary functions. In addition, we show that the same algorithm can be also derived using Karush–Kuhn–Tucker (KKT) conditions as well as the projected gradient. We provide two empirical study for image denoising and EEG classification, showing the interesting and useful behavior of the algorithm in cases where different values of α () are used. [Copyright &y& Elsevier]

Abstract: Nonnegative matrix factorization (NMF) solves the following problem: find such nonnegative matrices and that , given only and the assigned index J (). Basically, the factorization is achieved by alternating minimization of a given cost function subject to nonnegativity constraints. In the paper, we propose to use quadratic programming (QP) to solve the minimization problems. The Tikhonov regularized squared Euclidean cost function is extended with a logarithmic barrier function (which satisfies nonnegativity constraints), and then using second-order Taylor expansion, a QP problem is formulated. This problem is solved with some trust-region subproblem algorithm. The numerical tests are performed on the blind source separation problems. [Copyright &y& Elsevier]

Abstract: We propose a new algorithm for identifying a mixing (basis) matrix knowing only sensor (data) matrix for linear model , under some weak or relaxed conditions, expressed in terms of sparsity of latent (hidden) components represented by the unknown matrix . We present a simple and efficient adaptive algorithm for such identification and illustrate its performance by estimation of the unknown mixing matrix and source signals (sparse components) represented by rows of the matrix . The main feature of the proposed algorithm is its adaptivity to changing (non-stationary) environment and robustness with respect to outliers that do not necessarily satisfy sparseness conditions. [Copyright &y& Elsevier]

With statistical testing, corrections for multiple comparisons, such as Bonferroni adjustments, have given rise to controversies in the scientific community, because of their negative impact on statistical power. This impact is especially problematic for high-multidimensional data, such as multi-electrode brain recordings. With brain imaging data, a reliable method is needed to assess statistical significance of the data without losing statistical power. Conjunction analysis allows the combination of significance and consistency of an effect. Through a balanced combination of information from retest experiments (multiple trials split testing), we present an intuitively appealing, novel approach for brain imaging conjunction. The method is then tested and validated on synthetic data followed by a real-world test on QEEG data from patients with Alzheimer’s disease. This latter application requires both reliable type-I error and type-II error rates, because of the poor signal-to-noise ratio inherent in EEG signals. [ABSTRACT FROM AUTHOR]

Recently, a considerable growth of interest in projected gradient (PG) methods has been observed due to their high efficiency in solving large-scale convex minimization problems subject to linear constraints. Since the minimization problems underlying nonnegative matrix factorization (NMF) of large matrices well matches this class of minimization problems, we investigate and test some recent PG methods in the context of their applicability to NMF. In particular, the paper focuses on the following modified methods: projected Landweber, Barzilai-Borwein gradient projection, projected sequential subspace optimization (PSESOP), interior-point Newton (IPN), and sequential coordinate-wise. The proposed and implemented NMF PG algorithms are compared with respect to their performance in terms of signal-to-interference ratio (SIR) and elapsed time, using a simple benchmark of mixed partially dependent nonnegative signals. [ABSTRACT FROM AUTHOR]

The most popular algorithms for Nonnegative Matrix Factorization (NMF) belong to a class of multiplicative Lee-Seung algorithms which have usually relative low complexity but are characterized by slow-convergence and the risk of getting stuck to in local minima. In this paper, we present and compare the performance of additive algorithms based on three different variations of a projected gradient approach. Additionally, we discuss a novel multilayer approach to NMF algorithms combined with multi-start initializations procedure, which in general, considerably improves the performance of all the NMF algorithms. We demonstrate that this approach (the multilayer system with projected gradient algorithms) can usually give much better performance than standard multiplicative algorithms, especially, if data are ill-conditioned, badly-scaled, and/or a number of observations is only slightly greater than a number of nonnegative hidden components. Our new implementations of NMF are demonstrated with the simulations performed for Blind Source Separation (BSS) data. [ABSTRACT FROM AUTHOR]

Abstract: Nonnegative matrix factorization (NMF) solves the following problem: find nonnegative matrices and such that , given only and the assigned index R. This method has found a wide spectrum of applications in signal and image processing, such as blind source separation (BSS), spectra recovering, pattern recognition, segmentation or clustering. Such a factorization is usually performed with an alternating gradient descent technique that is applied to the squared Euclidean distance or Kullback–Leibler divergence. This approach has been used in the widely known Lee–Seung NMF algorithms that belong to a class of multiplicative iterative algorithms. It is well known that these algorithms, in spite of their low complexity, are slowly convergent, give only a strictly positive solution, and can easily fall into local minima of a nonconvex cost function. In this paper, we propose to take advantage of the second-order terms of a cost function to overcome the disadvantages of gradient (multiplicative) algorithms. First, a projected quasi-Newton method is presented, where a regularized Hessian with the Levenberg–Marquardt approach is inverted with the Q-less QR decomposition. Since the matrices and/or are usually sparse, a more sophisticated hybrid approach based on the gradient projection conjugate gradient (GPCG) algorithm, which was invented by More and Toraldo, is adapted for NMF. The gradient projection (GP) method is exploited to find zero-value components (active), and then the Newton steps are taken only to compute positive components (inactive) with the conjugate gradient (CG) method. As a cost function, we used the -divergence that unifies many well-known cost functions. We applied our new NMF method to a BSS problem with mixed signals and images. The results demonstrate the high robustness of our method. [Copyright &y& Elsevier]

Abstract: Recently, the technique of principal component analysis (PCA) has been expressed as the maximum likelihood solution for a generative latent variable model. A central issue in PCA is choosing the number of principal components to retain. This can be considered as a problem of model selection. In this paper, the probabilistic reformulation of PCA is used as a basis for a Bayesian approach of PCA to derive a model selection criterion for determining the true dimensionality of data. The proposed criterion is similar to the Bayesian Information Criterion, BIC, with a particular goodness of fit term and it is consistent. A simulation example that illustrate its performance for the determination of the number of principal components to be retained is presented. [Copyright &y& Elsevier]

In this paper, we use a two-stage sparse factorization approach for blindly estimating the channel parameters and then estimating source components for electroencephalogram (EEG) signals. EEG signals are assumed to be linear mixtures of source components, artifacts, etc. Therefore, a raw EEG data matrix can be factored into the product of two matrices, one of which represents the mixing matrix and the other the source component matrix. Furthermore, the components are sparse in the time-frequency domain, i.e., the factorization is a sparse factorization in the time frequency domain. It is a challenging task to estimate the mixing matrix. Our extensive analysis and computational results, which were based on many sets of EEG data, not only provide firm evidences supporting the above assumption, but also prompt us to propose a new algorithm for estimating the mixing matrix. After the mixing matrix is estimated, the source components are estimated in the time frequency domain using a linear programming method. In an example of the potential applications of our approach, we analyzed the EEG data that was obtained from a modified Sternberg memory experiment. Two almost uncorrelated components obtained by applying the sparse factorization method were selected for phase synchronization analysis. Several interesting findings were obtained, especially that memory-related synchronization and desynchronization appear in the alpha band, and that the strength of alpha band synchronization is related to memory performance. [ABSTRACT FROM AUTHOR]

Our study aimed at evaluation of the relations between the plasma levels of mitotane (o,p'-DDD) and its metabolites, o,p'-DDA and o,p'-DDE, and the efficacy of Mitotane therapy during a long-term follow-up. Eighteen patients, aged 11 to 70 years, were included to the study. Metastatic or regional stage was diagnosed in 15 patients, while localized disease in three patients. Mitotane has been administered in daily doses of 3.0 to 10.0 g in metastatic and regional stages, and 1.5 to 4.0 g in the localized disease, simultaneously with hydrocortisone, prednisolone and fludrocortisone. Mitotane and its metabolites were determined by a high-pressure liquid chromatographic method. The plasma o,p'-DDD level exceeding 44_M/L, considered as curative one, was reached in nine cases. The o,p'-DDA/o,p'-DDD ratio rose significantly mainly in the first 1-3 months of therapy. The o,p'-DDE levels rose slowly, reaching higher values in long-term therapy, over 12 months of mitotane administration. In the group of patients with regional or metastatic stage, both the o,p'-DDE levels and the o,p'-DDE/o,p'-DDD ratios were higher in the survivors than in nonsurvivors. The results of our study suggest that the plasma concentrations of o,p'-DDE were more closely related to clinical improvement or remission than the o,p'-DDD levels. [ABSTRACT FROM AUTHOR]

Abstract: Objective: Development of an EEG preprocessing technique for improvement of detection of Alzheimer''s disease (AD). The technique is based on filtering of EEG data using blind source separation (BSS) and projection of components which are possibly sensitive to cortical neuronal impairment found in early stages of AD. Methods: Artifact-free 20s intervals of raw resting EEG recordings from 22 patients with Mild Cognitive Impairment (MCI) who later proceeded to AD and 38 age-matched normal controls were decomposed into spatio-temporally decorrelated components using BSS algorithm ‘AMUSE’. Filtered EEG was obtained by back projection of components with the highest linear predictability. Relative power of filtered data in delta, theta, alpha 1, alpha 2, beta 1, and beta 2 bands were processed with Linear Discriminant Analysis (LDA). Results: Preprocessing improved the percentage of correctly classified patients and controls computed with jack-knifing cross-validation from 59 to 73% and from 76 to 84%, correspondingly. Conclusions: The proposed approach can significantly improve the sensitivity and specificity of EEG based diagnosis. Significance: Filtering based on BSS can improve the performance of the existing EEG approaches to early diagnosis of Alzheimer''s disease. It may also have potential for improvement of EEG classification in other clinical areas or fundamental research. The developed method is quite general and flexible, allowing for various extensions and improvements. [Copyright &y& Elsevier]

This paper proposes a novel grouping decision approach for blind source estimation of FIR (finite impulse response) channels with binary sources. First, solvability is discussed for single-input systems and multi-input systems. Necessary and sufficient conditions for recoverability are derived. For single-input systems, a new deterministic algorithm based on grouping and decision is proposed to recover the source up to a delay. The algorithm is easy to implement and has several advantages. For instance, when the solvability conditions are satisfied, it can be applied to cases in which: (i) the channel has zeros on the unit circle or outside of the unit circle; (ii) there are fewer sensors than sources; (iii) the source is temporarily dependent. To improve noise tolerance and reduce computational cost, the algorithm is further elaborated for highly noisy channels and high-order FIR channels, respectively. For the channels with high unimodal noise, fewer peaks appear in the probability density function (pdf) of the outputs compared to the pdf of the outputs of channels with a higher SNR. After the peaks representing cluster centers are estimated using a maximum likelihood (ML) approach, the deterministic algorithm can be used. Similar to highly noisy channels, the algorithm is also effective for high-order, exponentially decaying channels after fewer cluster centers are estimated. Furthermore, blind source estimation for multi-input systems also can be carried out as with the case of single input systems. Two deflation algorithms are presented for temporarily dependent sources and i.i.d. sources. Based on the source estimation and deflation algorithms, the sources can be obtained one by one. Finally, the validity and performance of the algorithms are illustrated by several simulation examples. [Copyright &y& Elsevier]

This paper reports a study on the problem of the blind simultaneous extraction of specific groups of independent components from a linear mixture. This paper first presents a general overview and unification of several information theoretic criteria for the extraction of a single independent component. Then, our contribution fills the theoretical gap that exists between extraction and separation by presenting tools that extend these criteria to allow the simultaneous blind extraction of subsets with an arbitrary number of independent components. In addition, we analyze a family of learning algorithms based on Stiefel manifolds and the natural gradient ascent, present the nonlinear optimal activations (score) functions, and provide new or extended local stability conditions. Finally, we illustrate the performance and features of the proposed approach by computer-simulation experiments. [ABSTRACT FROM AUTHOR]

In this letter, we analyze a two-stage cluster-then-I1-optimization approach for sparse representation of a data matrix, which is also a promising approach for blind source separation (BSS) in which fewer sensors than sources are present. First, sparse representation (factorization) of a data matrix is discussed. For a given over-complete basis matrix, the corresponding sparse solution (coefficient matrix) with minimum l1 norm is unique with probability one, which can be obtained using a standard linear programming algorithm. The equivalence of the l1 -norm solution and the/O-norm solution is also analyzed according to a probabilistic framework. If the obtained l1-norm solution is sufficiently sparse, then it is equal to the l-norm solution with a high probability. Furthermore, the l1-norm solution is robust to noise, but the/O-norm solution is not, showing that the l1-norm is a good sparsity measure. These results can be used as a recoverability analysis of BSS, as discussed. The basis matrix in this article is estimated using a clustering algorithm followed by normalization, in which the matrix columns are the cluster centers of normalized data column vectors. Zibulevsky, Pearlmutter, Boll, and Kisilev (2000) used this kind of two-stage approach in underdetermined BSS. Our recoverability analysis shows that this approach can deal with the situation in which the sources are overlapped to some degree in the analyzed domain and with the case in which the source number is unknown. It is also robust to additive noise and estimation error in the mixing matrix. Finally, four simulation examples and an EEG data analysis example are presented to illustrate the algorithm's utility and demonstrate its performance. [ABSTRACT FROM AUTHOR]

In this paper, we present a new filter decomposition method for multichannel blind deconvolution of nonminimum-phase systems. With this approach, we decompose a doubly finite impulse response filter into a cascade form of two filters: a causal finite impulse response (FIR) filter and an anticausal FIR filter. After introducing a Lie group to the manifold of FIR filters, we discuss geometric properties of the FIR filter manifold. Using the nonholonomic transform, we derive the natural gradient on the FIR manifold. By simplifying the mutual information rate, we present a very simple cost function for blind deconvolution of nonminimum-phase systems. Subsequently, the natural gradient algorithms are developed both for the causal FIR filter and for the anticausal FIR filter. Simulations are presented to illustrate the validity and favorable learning performance of the proposed algorithms. [ABSTRACT FROM AUTHOR]

Independent component analysis is to extract independent signals from their linear mixtures without assuming prior knowledge of their mixing coefficients. As we know, a number of factors are likely to affect separation results in practical applications, such as the number of active sources, the distribution of source signals, and noise. The purpose of this paper to develop a general framework of blind separation from a practical point of view with special emphasis on the activation function adaptation. First, we propose the exponential generative model for probability density functions. A method of constructing an exponential generative model from the activation functions is discussed. Then, a Learning algorithm is derived to update the parameters in the exponential generative model. The learning algorithm for the activation function adaptation is consistent with the one for training the denting model. Stability analysis of the learning algorithm for the activation function is also discussed. Both theoretical analysis and simulations show that the proposed approach is universally convergent regardless of the distributions of sources. Finally, computer simulations are given to demonstrate the effectiveness and validity of the approach. [ABSTRACT FROM AUTHOR]

Redundancy reduction as a form of neural coding has been since the early sixties a topic of large research interest. A number of strategies has been proposed, but the one which is attracting most attention recently assumes that this coding is carried out so that the output signals are mutually independent. In this work we go one step further and suggest an strategy to deal also with non-orthogonal signals (i.e., ''dependent'' signals). Moreover, instead of working with the usual squared error, we design a neuron where the non-linearity is operating on the error. It is computationally more economic and, importantly, the permutation/scaling problem[sup 10] is avoided. The framework is given with a biological background, as we avocate throughout the manuscript that the algorithm fits well the single neuron and redundancy reduction doctrine.[sup 5] Moreover, we show that wavelet-like receptive fields emerges from natural images processed by this algorithm. [ABSTRACT FROM AUTHOR]

In many applications of signal processing, especially in communications and biomedicine, preprocessing is necessary to remove noise from data recorded by multiple sensors. Typically, each sensor or electrode measures the noisy mixture of original source signals. In this paper a noise reduction technique using independent component analysis (ICA) and subspace filtering is presented. In this approach we apply subspace filtering not to the observed raw data but to a demixed version of these data obtained by ICA. Finite impulse response filters are employed whose vectors are parameters estimated based on signal subspace extraction. ICA allows us to filter independent components. After the noise is removed we reconstruct the enhanced independent components to obtain clean original signals; i.e., we project the data to sensor level. Simulations as well as real application results for EEG-signal noise elimination are included to show the validity and effectiveness of the proposed approach. [ABSTRACT FROM AUTHOR]

Most of source separation methods focus on stationary sources, so higher-order statistics is necessary for successful separation, unless sources are temporally correlated. For nonstationary sources, however, it was shown [Neural Networks 8 (1995) 411] that source separation could be achieved by second-order decorrelation. In this paper, we consider the cost function proposed by Matsuoka et al. [Neural Networks 8 (1995) 411] and derive natural gradient learning algorithms for both fully connected recurrent network and feedforward network. Since our algorithms employ the natural gradient method, they possess the equivariant property and find a steepest descent direction unlike the algorithm [Neural Networks 8 (1995) 411]. We also show that our algorithms are always locally stable, regardless of probability distributions of nonstationary sources. [Copyright &y& Elsevier]

In this work we develop a very simple batch learning algorithm for semiblind extraction of a desired source signal with temporal structure from linear mixtures. Although we use the concept of sequential blind extraction of sources and independent component analysis, we do not carry out the extraction in a completely blind manner; neither do we assume that sources are statistically independent. In fact, we show that the a priori information about the autocorrelation function of primary sources can be used to extract the desired signals (sources of interest) from their linear mixtures. Extensive computer simulations and real data application experiments confirm the validity and high performance of the proposed algorithm. [ABSTRACT FROM AUTHOR]