Search

This paper proposes a pattern recognition method for the fault diagnosis of machinery based on an enhanced trace ratio linear discriminant analysis (ETR-LDA) algorithm. It is an extension of TR-LDA that pursues to maximize the between-class distance and minimize the within-class distance. Since TR-LDA focuses on the total between-class distance and ignores the attention to local features, it may lead to deficiencies in the classification of specific patterns/classes, which is defined as short board in this paper. To cope with this limitation, the proposed ETR-LDA makes a reconsideration of the relationship between the total between-class distance and the global separability. More specifically, a new objective function is derived in ETR-LDA by taking the smallest between-class distance into account. The optimal solution of this objective function is proved to improve the smallest between-class distance with no change to the convergence and the global optimality of the algorithm, and the separability of different classes is increased eventually. Both synthetic data and hard disk drive wearing experimental data are employed to verify the efficiency of the proposed method. The results show that ETR-LDA is able to distinguish those fault categories and outperform TR-LDA and other dimensionality reduction methods. [ABSTRACT FROM AUTHOR]

In recent years, we have witnessed the explosion of large-scale data in various fields. Classical statistical methodologies, such as linear regression or generalized linear regression, often show inadequate performance on heterogeneous data because the key homogeneity assumption fails. In this paper, we present a flexible framework to handle heterogeneous populations that can be naturally grouped into several ordered subtypes. A local model technique utilizing ordinal class labels during the training stage is proposed. We define a new “progression score” that captures the progression of ordinal classes, and use a truncated Gaussian kernel to construct the weight function in a local regression framework. Furthermore, given the weights, we apply sparse shrinkage on the local fitting to handle high dimensionality. In this way, our local model is able to conduct variable selection on each query point. Numerical studies show the superiority of our proposed method over several existing ones. Our method is also applied to the Alzheimer’s Disease Neuroimaging Initiative data to make predictions on the longitudinal clinical scores based on different modalities of baseline brain image features. [ABSTRACT FROM AUTHOR]

We consider the problem of multiantenna source detection with dimension reduction in distributed wireless networks. Traditional strategies typically collect and perform raw data at the fusion center. In moderate-to-large-scale networks, however, the schemes create the bottleneck of computation and communication for high-dimensional data. The goal of this paper is to design a distributed algorithm in multiple-antenna wireless networks to project the raw data into the low-dimensional data to reduce the communication and computation burden while maintaining high detection performance. In this paper, a pseudosketching matrix is constructed to transform the raw data of each multiple-antenna node into the low-dimensional data. Furthermore, it is transformed into the vector by using the subspace method. By gathering the data vectors of all nodes at the fusion center, the eigenvalue-based detection methods, such as the generalized likelihood ratio test (GLRT), can be applied directly to determine if the source signal exists or not. Moreover, using the concentration inequalities of subgamma random variables, the theoretical analysis is derived to support the claim that the proposed algorithm has high detection performance. The simulations are presented to demonstrate the effectiveness of our proposed algorithm. [ABSTRACT FROM AUTHOR]

Objective: Multi-modal brain functional connectivity (FC) data have shown great potential for providing insights into individual variations in behavioral and cognitive traits. The joint learning of multi-modal data can utilize intrinsic association, and thus can boost learning performance. Although several multi-task based learning models have already been proposed by viewing feature learning on each modality as one task, most of them ignore the structural information inherent across the modalities, which may play an important role in extracting discriminative features. Methods: In this paper, we propose a new manifold regularized multi-task learning model by simultaneously considering between-subject and between-modality relationships. Specifically, the $l_{2,1}$ -norm (i.e., group-sparsity) regularizer is enforced to jointly select a few common features across different modalities. A novelly designed manifold regularizer is further imposed as a crucial underpinning to preserve the structural information both within and between modalities. Such designed regularizers will make our model more adaptive to realistic neuroimaging data, which are usually of small sample size but high dimensional features. Results: Our model is validated on the Philadelphia Neurodevelopmental Cohort dataset, where our modalities are regarded as two types of functional MRI (fMRI) data collected under two paradigms. We conduct experimental studies on fMRI-based FC network data in two task conditions for intelligence quotient (IQ) prediction. The results show that our proposed model can not only achieve improved prediction performance, but also yield a set of IQ-relevant biomarkers. Conclusion and Significance: This paper develops a new multi-task learning model, enabling the discovery of significant biomarkers that may account for a proportion of the variance in human intelligence. [ABSTRACT FROM AUTHOR]

The generalized polynomial chaos (gPC) method recently advocated in the literature, exhibits impressive efficiency and accuracy in probabilistic power flow (PPF) calculations of small-scale power systems. However, it suffers from the “curse of dimensionality” and can only be applied to systems with input random variables that follow a set of standard probability distributions. This paper overcome these weaknesses by developing a hierarchical polynomial chaos analysis of variance (ANOVA) method that shows excellent performances in terms of accuracy, efficiency, rationality, and adaptability, for small- to large-scale power systems. By proving the equivalence between the polynomial chaos expansion and the ANOVA decomposition, which is executed at no extra computational cost, the dimensionality of the polynomial chaos expansion can be adaptively reduced to improve the computational efficiency of the generalized polynomial chaos method in high-dimensional problems. Furthermore, by resorting to the Stieltjes procedure, it is extended to any assumed probability distributions of the input random variables. Simulation results carried out on the IEEE 118-bus system and the 1354-bus European high voltage system with correlated renewable energy generations reveal that the developed method outperforms the generalized polynomial chaos method and the Monte Carlo (MC) method while being compatible with real-time applications in power systems. [ABSTRACT FROM AUTHOR]

The infrared atmospheric sounder interferometer (IASI) onboard the Metop-A and Metop-B satellites is an essential instrument for numerical weather prediction centers. The IASI, with its 8461 channels, enables the observation of the surface and atmospheric geophysical variables twice a day around the world. This data volume will increase in the future, stressing the need for practical and efficient dimension reduction techniques for storage and transmission but also for inversion or assimilation. In a previous paper by Aires et al., a new dimension reduction method, the so-called bottleneck channel (BC), has been introduced. BCs are a good compromise between compression and channel selection techniques. In this paper, we introduce several new technical configurations of the BC approach, comparing their advantages/drawbacks. Comparisons are also made with two classical methodologies widely used in the satellite community: principal component analysis (PCA) compression and entropy reduction channel selection. Beyond the compression ability, the methodologies are tested on the temperature, water vapor, and ozone retrieval application using IASI measurements. Two approaches have been tested. First, theoretical retrieval improvement has been measured when using the various data reduction techniques. The BC achieves good theoretical improvements, comparable to noise-adjusted PCA (NAPCA), for the retrieval of the temperature, water vapor, and ozone. Second, a neural network inversion scheme is tested on real IASI observations using the several data reduction techniques. It is shown that BCs provide similar or better results than the other methods. The BCs present compression, denoising, and information content results comparable to the NAPCA, while preserving the physical meaning (i.e., actual channel brightness temperatures) of a channel selection approach. This has numerous advantages for inversion and assimilation: the mixing problem, contamination effects, minor constituents representation, or radiative transfer speed, and each of these will be discussed. [ABSTRACT FROM AUTHOR]

In this paper, we investigate the anti-jamming problem of a directional modulation (DM) system with the aid of intelligent reflecting surface (IRS). As an efficient tool to combat malicious jamming, receive beamforming (RBF) is usually designed by using the statistical properties of the jamming received by Bob. Thus, it is very necessary to estimate the receive jamming covariance matrix (JCM) at Bob. To achieve a precise JCM estimation, three JCM estimation methods, including eigenvalue decomposition (EVD), parametric estimation method by gradient descend (PEM-GD) and parametric estimation method by alternating optimization (PEM-AO), are proposed. Here, the proposed EVD is derived according to the rank-2 constraint of JCM. The PEM-GD method fully explores the structure features of JCM and the PEM-AO is proposed to decrease the computational complexity of the former via dimensionality reduction. The simulation results show that the proposed three methods perform better than directly using sample covariance matrix. Additionally, the proposed PEM-GD and PEM-AO outperform EVD method and the clutter and disturbance covariance estimator RCML. [ABSTRACT FROM AUTHOR]

Nonnegative matrix factorization (NMF) is a linear dimensionality reduction technique for analyzing nonnegative data. A key aspect of NMF is the choice of the objective function that depends on the noise model (or statistics of the noise) assumed on the data. In many applications, the noise model is unknown and difficult to estimate. In this paper, we define a multi-objective NMF (MO-NMF) problem, where several objectives are combined within the same NMF model. We propose to use Lagrange duality to judiciously optimize for a set of weights to be used within the framework of the weighted-sum approach, that is, we minimize a single objective function which is a weighted sum of the all objective functions. We design a simple algorithm based on multiplicative updates to minimize this weighted sum. We show how this can be used to find distributionally robust NMF (DR-NMF) solutions, that is, solutions that minimize the largest error among all objectives, using a dual approach solved via a heuristic inspired from the Frank-Wolfe algorithm. We illustrate the effectiveness of this approach on synthetic, document and audio data sets. The results show that DR-NMF is robust to our incognizance of the noise model of the NMF problem. [ABSTRACT FROM AUTHOR]

Supervised dimensionality reduction for sequence data learns a transformation that maps the observations in sequences onto a low-dimensional subspace by maximizing the separability of sequences in different classes. It is typically more challenging than conventional dimensionality reduction for static data, because measuring the separability of sequences involves non-linear procedures to manipulate the temporal structures. In this paper, we propose a linear method, called order-preserving Wasserstein discriminant analysis (OWDA), and its deep extension, namely DeepOWDA, to learn linear and non-linear discriminative subspace for sequence data, respectively. We construct novel separability measures between sequence classes based on the order-preserving Wasserstein (OPW) distance to capture the essential differences among their temporal structures. Specifically, for each class, we extract the OPW barycenter and construct the intra-class scatter as the dispersion of the training sequences around the barycenter. The inter-class distance is measured as the OPW distance between the corresponding barycenters. We learn the linear and non-linear transformations by maximizing the inter-class distance and minimizing the intra-class scatter. In this way, the proposed OWDA and DeepOWDA are able to concentrate on the distinctive differences among classes by lifting the geometric relations with temporal constraints. Experiments on four 3D action recognition datasets show the effectiveness of OWDA and DeepOWDA. [ABSTRACT FROM AUTHOR]

In the vision research field, it is useful to make a comparison to see which one of two images exhibits a particular visual attribute more than the other. Feature representation and metric learning are two critical factors for a fine-grained visual comparison. In this paper, first an informative feature descriptor combining histograms of oriented gradient feature with gist feature is proposed, and then a method called relative attribute quadratic discriminant analysis is proposed for dimensionality reduction and metric learning simultaneously. In the method, those pairs most analogous to a test pair are identified by using the learned metric and then used to build a local ranking function. Finally, the test pair label is predicted by the ranking function for each attribute. Experimental results on the three databases of UT-Zap50K, Outdoor Scene Recognition dataset, and Public Figures faces dataset demonstrate the advantages of the proposed method. [ABSTRACT FROM AUTHOR]

In this paper, a novel dimension-reduction approach is presented to overcome challenges such as nonlinear relationships, heterogeneity, and noisy dimensions. Initially, the $p$ p attributes in the data are first organized into random groups. Next, to systematically remove redundant and noisy dimensions from the data, each group is independently mapped into a low dimensional space via a parametric mapping. The group-wise transformation parameters are estimated using a low-rank approximation of distance covariance. The transformed attributes are reorganized into groups based on the magnitude of their respective eigenvalues. The group-wise organization and reduction process is performed until a user-defined criterion on eigenvalues is satisfied. In addition, novel procedures are introduced to aggregate the transformation parameters when the data is available in batches. Overall performance is demonstrated with extensive simulation analysis on classification by employing 10 data-sets. [ABSTRACT FROM AUTHOR]

High-dimensional and large-scale time series processing has aroused considerable research interests during decades. It is difficult for traditional methods to reveal the evolution state in dynamical systems and discover the relationship among variables automatically. In this paper, we propose a unified framework for nonuniform embedding, dynamical system revealing, and time series prediction, termed as Structured Manifold Broad Learning System (SM-BLS). The structured manifold learning is introduced for nonuniform embedding and unsupervised manifold learning simultaneously. Graph embedding and feature selection are both considered to depict the intrinsic structure connections between chaotic time series and its low-dimensional manifold. Compared with traditional methods, the proposed framework could discover potential deterministic evolution information of dynamical systems and make the modeling more interpretable. It provides us a homogeneous way to recover the chaotic attractor from multivariate and heterogeneous time series. Simulation analysis and results show that SM-BLS has advantages in dynamic discovery and feature extraction of large-scale chaotic time series prediction. [ABSTRACT FROM AUTHOR]

This paper proposes a new semisupervised dimension reduction (DR) algorithm based on a discriminative locally enhanced alignment technique. The proposed DR method has two aims: to maximize the distance between different classes according to the separability of pairwise samples and, at the same time, to preserve the intrinsic geometric structure of the data by the use of both labeled and unlabeled samples. Furthermore, two key problems determining the performance of semisupervised methods are discussed in this paper. The first problem is the proper selection of the unlabeled sample set; the second problem is the accurate measurement of the similarity between samples. In this paper, multilevel segmentation results are employed to solve these problems. Experiments with extensive hyperspectral image data sets showed that the proposed algorithm is notably superior to other state-of-the-art dimensionality reduction methods for hyperspectral image classification. [ABSTRACT FROM AUTHOR]

The unknown statistical distributions of two effective elastic properties of Sn-3.0Ag-0.5Cu solder joint of leadless chip resistors (LCRs), induced by an assembly condition, are determined by the advanced statistical model calibration. Two key elements are involved in the model calibration: an uncertainty propagation (UP) analysis and an optimization process using a calibration metric. In this paper, the UP analysis utilizes an advanced approximate integration method, which allows us to take into account the statistical variations of six additional known input variables, including die thickness, solder joint height, termination length, and thickness and elastic moduli of a printed circuit board. The cyclic bending test results of LCR assemblies are used in conjunction with the maximum-likelihood metric to obtain the statistical distributions of the effective properties. The cycles-to-failure distribution of the identical LCR assemblies subjected to a different loading level is predicted accurately by the calibrated model, which corroborates the validity of the proposed approach. [ABSTRACT FROM AUTHOR]

Canonical correlation analysis (CCA) is a powerful technique for discovering whether or not hidden sources are commonly present in two (or more) datasets. Its well-appreciated merits include dimensionality reduction, clustering, classification, feature selection, and data fusion. The standard CCA, however, does not exploit the geometry of the common sources, which may be available from the given data or can be deduced from (cross-) correlations. In this paper, this extra information provided by the common sources generating the data is encoded in a graph, and is invoked as a graph regularizer. This leads to a novel graph-regularized CCA approach, that is termed graph (g) CCA. The novel gCCA accounts for the graph-induced knowledge of common sources, while minimizing the distance between the wanted canonical variables. Tailored for diverse practical settings where the number of data is smaller than the data vector dimensions, the dual formulation of gCCA is developed too. One such setting includes kernels that are incorporated to account for nonlinear data dependencies. The resultant graph-kernel CCA is also obtained in closed form. Finally, corroborating image classification tests over several real datasets are presented to showcase the merits of the novel linear, dual, and kernel approaches relative to competing alternatives. [ABSTRACT FROM AUTHOR]

Dimensionality reduction is an essential task in hyperspectral image processing. How to preserve the original intrinsic structure information and enhance the discriminant ability is still a challenge in this area. Recently, with the advantage of preserving global intrinsic structure information, low-rank representation has been applied to dimensionality reduction and achieved promising performance. By exploiting the submanifold information of the original data set, multimanifold learning is effective in enhancing the discriminant ability of the processed data set. In addition, due to the ability of preserving the spatial neighborhood structure information, the tensor analysis has become a popular technique for hyperspectral image processing. Motivated by the above-mentioned analysis, a novel tensor-based low-rank graph with multimanifold regularization (T-LGMR) for dimensionality reduction of hyperspectral images is proposed in this paper. In the T-LGMR, a low-rank constraint is employed to preserve the global data structure while multimanifold information is utilized to enhance the discriminant ability, and tensor representation is used to preserve the spatial neighborhood information. Finally, dimensionality reduction is achieved in the graph embedding framework. Experimental results on three real hyperspectral data sets demonstrate the superiority of the proposed method over several state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Principal component analysis (PCA) is a widely used technique for dimension reduction, data processing, and feature extraction. The three tasks are particularly useful and important in high-dimensional data analysis and statistical learning. However, the regular PCA encounters great fundamental challenges under high dimensionality and may produce “wrong” results. As a remedy, sparse PCA (SPCA) has been proposed and studied. SPCA is shown to offer a “right” solution under high dimensions. In this paper, we review methodological and theoretical developments of SPCA, as well as its applications in scientific studies. [ABSTRACT FROM AUTHOR]

Robust and effective shape prior modeling from a set of training data remains a challenging task, since the shape variation is complicated, and shape models should preserve local details as well as handle shape noises. To address these challenges, a novel robust projective dictionary learning (RPDL) scheme is proposed in this paper. Specifically, the RPDL method integrates the dimension reduction and dictionary learning into a unified framework for shape prior modeling, which can not only learn a robust and representative dictionary with the energy preservation of the training data, but also reduce the dimensionality and computational cost via the subspace learning. In addition, the proposed RPDL algorithm is regularized by using the \ell 1 norm to handle the outliers and noises, and is embedded in an online framework so that of memory and time efficiency. The proposed method is employed to model prostate shape prior for the application of magnetic resonance transrectal ultrasound registration. The experimental results demonstrate that our method provides more accurate and robust shape modeling than the state-of-the-art methods do. The proposed RPDL method is applicable for modeling other organs, and hence, a general solution for the problem of shape prior modeling. [ABSTRACT FROM PUBLISHER]

This paper introduces a new concept of band capacity (BC) of a hyperspectral image and further develops a theory for BC. Its idea is derived from information theory where a band channel can be constructed from a hyperspectral image with both its channel input space and channel output space specified by its full band set and the channel transition probabilities between the input and output spaces characterized by betweenband discrimination. In particular, a transition probability from a spectral band in the band channel input space to a spectral band in the band channel output space is calculated by their spectral discriminatory power/probability. By virtue of such a formulated band channel, its maximal mutual information can be defined as BC of a hyperspectral image to represent spectral discriminatory power per band measured by bits. Interestingly, BC provides a key to bridging the concept of virtual dimensionality defined as the number of spectrally distinct signatures and effective band dimensionality to be used to discriminate these spectrally distinct signatures one from another. Accordingly, an immediate application of BC is to determine the number of bands to be selected, nBS. Another application is band selection with the output space specified by a selected nBS-band subset. In this case, when BC is close to one, the selected band set tends to be optimal. [ABSTRACT FROM AUTHOR]

This paper presents a polynomial chaos (PC) formulation based on the concept of dimension reduction for the efficient uncertainty analysis of microwave and RF networks. This formulation exploits a high-dimensional model representation for quantifying the relative effect of each random dimension on the network responses surface. This information acts as problem-dependent sensitivity indices guiding the intelligent identification and subsequent pruning of the statistically unimportant random dimensions from the original parametric space. Performing a PC expansion in the resultant low-dimensional random subspace leads to the recovery of a sparser set of coefficients than that obtained from the full-dimensional random space with negligible loss in accuracy. Novel methodologies to reuse the preliminary PC bases and SPICE simulations required to estimate the sensitivity indices are presented, thereby making the proposed approach more efficient and accurate than standard sparse PC approaches. The validity of the proposed approach is demonstrated using three distributed network examples. [ABSTRACT FROM PUBLISHER]

Dimensionality reduction (DR) has attracted extensive attention since it provides discriminative information of hyperspectral images (HSI) and reduces the computational burden. Though DR has gained rapid development in recent years, it is difficult to achieve higher classification accuracy while preserving the relevant original information of the spectral bands. To relieve this limitation, in this paper, a different DR framework is proposed to perform feature extraction on the selected bands. The proposed method uses determinantal point process to select the representative bands and to preserve the relevant original information of the spectral bands. The performance of classification is further improved by performing multiple Laplacian eigenmaps (LEs) on the selected bands. Different from the traditional LEs, multiple Laplacian matrices in this paper are defined by encoding spatial–spectral proximity on each band. A common low-dimensional representation is generated to capture the joint manifold structure from multiple Laplacian matrices. Experimental results on three real-world HSIs demonstrate that the proposed framework can lead to a significant advancement in HSI classification compared with the state-of-the-art methods. [ABSTRACT FROM PUBLISHER]

Virtual dimensionality (VD) has received considerable interest in its use of specifying the number of spectrally distinct signatures present in hyperspectral data. Unfortunately, it never defines what such a signature is. For example, various targets of interest, such as anomalies and endmembers, should be considered as different types of spectrally distinct signatures and have their own different values of VD. Specifically, these targets are insignificant in terms of signal energies due to their relatively small populations. Accordingly, their contributions to second-order statistics (2OS) are rather limited. In this case, 2OS-based methods such as eigen-approaches to determine VD may not be effective in determining how many such type of signal sources as spectrally distinct signatures are. This paper develops a new theory that expands 2OS-VD theory to a high-order statistics (HOS)-based VD, called HOS-VD theory. Since there is no counterpart of the characteristic polynomial equation used to find eigenvalues in 2OS available for HOS, a direct extension is inapplicable. This paper re-invents a wheel by finding actual targets directly from the data rather than eigenvectors/singular vectors used in 2OS-VD theory which do not represent any real targets in the data. Consequently, comparing to 2OS-VD theory which can only be used to estimate the value of VD without finding real targets, the developed HOS-VD theory can accomplish both of tasks at the same time, i.e., determining the value of VD as well as finding actual targets directly from the data. [ABSTRACT FROM AUTHOR]

In machine learning, the idea of maximizing the margin between two classes is widely used in classifier design. Enlighted by the idea, this paper proposes a novel supervised dimensionality reduction method for tensor data based on local decision margin maximization. The proposed method seeks to preserve and protect the local discriminant information of the original data in the low-dimensional data space. Firstly, we depart the original tensor dataset into overlapped localities with discriminant information. Then, we extract the similarity and anti-similarity coefficients of each high-dimensional locality and preserve these coefficients in the embedding data space via the multilinear projection scheme. Under the combined effect of these coefficients, each dimension-reduced locality tends to be a convex set where strongly correlated intraclass points gather. Simultaneously, the local decision margin, which is defined as the shortest distance from the boundary of each locality to the nearest point of each side, will be maximized. Therefore, the local discriminant structure of the original data could be well maintained in the low-dimensional data space. Moreover, a simple iterative scheme is proposed to solve the final optimization problem. Finally, the experiment results on 6 real-world datasets demonstrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

In the last few years, great efforts have been made to extend the linear projection technique (LPT) for multidimensional data (i.e., tensor), generally referred to as the multilinear projection technique (MPT). The vectorized nature of LPT requires high-dimensional data to be converted into vector, and hence may lose spatial neighborhood information of raw data. MPT well addresses this problem by encoding multidimensional data as general tensors of a second or even higher order. In this paper, we propose a novel multilinear projection technique, called multilinear spatial discriminant analysis (MSDA), to identify the underlying manifold of high-order tensor data. MSDA considers both the nonlocal structure and the local structure of data in the transform domain, seeking to learn the projection matrices from all directions of tensor data that simultaneously maximize the nonlocal structure and minimize the local structure. Different from multilinear principal component analysis (MPCA) that aims to preserve the global structure and tensor locality preserving projection (TLPP) that is in favor of preserving the local structure, MSDA seeks a tradeoff between the nonlocal (global) and local structures so as to drive its discriminant information from the range of the non-local structure and the range of the local structure. This spatial discriminant characteristic makes MSDA have more powerful manifold preserving ability than TLPP and MPCA. Theoretical analysis shows that traditional MPTs, such as multilinear linear discriminant analysis, TLPP, MPCA, and tensor maximum margin criterion, could be derived from the MSDA model by setting different graphs and constraints. Extensive experiments on face databases (ORL, CMU PIE, and the extended Yale-B) and the Weizmann action database demonstrate the effectiveness of the proposed MSDA method. [ABSTRACT FROM AUTHOR]

In this paper, we introduce the so-called hierarchical interaction models, where we assume that the computation of the value of a function m: \mathbb R^d\rightarrow \mathbb R is done in several layers, where in each layer a function of at most d^* inputs computed by the previous layer is evaluated. We investigate two different regression estimates based on polynomial splines and on neural networks, and show that if the regression function satisfies a hierarchical interaction model and all occurring functions in the model are smooth, the rate of convergence of these estimates depends on d^* (and not on $d$ ). Hence, in this case, the estimates can achieve good rate of convergence even for large $d$ , and are in this sense able to circumvent the so-called curse of dimensionality. [ABSTRACT FROM PUBLISHER]

Dimension reduction is frequently adopted as a data preprocessing technique to facilitate data visualization, interpretation, and classification. Traditional dimension reduction methods such as linear discriminant analysis focus on maximizing the overall discrimination between all classes, which may be easily affected by outliers. To overcome this disadvantage, this paper proposes a novel method for multiclass dimension reduction, named dimension reduction by minimum error minimax probability machine (DR-MEMPM). It elaborately ensures that each pair of classes is well separated in the projected subspace by utilizing the separation probability between different pairwise classes. Therefore, it can put more emphasis on those less distinguishable classes, and the learned projection will not be dominated by some “outlier” classes which lie far away from other classes. We evaluate the proposed DR-MEMPM on a number of synthetic and real-world data sets, and show that it outperforms other state-of-the-art dimension reduction methods in terms of visual intuition and classification accuracy, especially when the distances between classes are unevenly distributed. [ABSTRACT FROM AUTHOR]

The problem of extracting low-dimensional structure from high-dimensional data arises in many applications such as machine learning, statistical pattern recognition, wireless sensor networks, and data compression. If the data is restricted to a lower dimensional subspace, then simple algorithms using linear projections can find the subspace and consequently estimate its dimensionality. However, if the data lies on a low-dimensional but nonlinear space (e.g., manifolds), then its structure may be highly nonlinear and, hence, linear methods are doomed to fail. In this paper, we introduce a new technique for dimensionality reduction based on point-wise operators. More precisely, let \bf An\times n be a matrix of rank k\ll n and assume that the matrix \bf Bn\times n is generated by taking the elements of \bf A to some real power p. In this paper, we show that based on the values of the data matrix \bf B, one can estimate the value p and, therefore, the underlying low-rank matrix \bf A; i.e., we are reducing the dimensionality of \bf B by using point-wise operators. Moreover, the estimation algorithm does not need to know the rank of \bf A. We also provide bounds on the quality of the approximation and validate the stability of the proposed algorithm with simulations in noisy environments. [ABSTRACT FROM PUBLISHER]

Many-objective optimization refers to the optimization problems containing large number of objectives, typically more than four. Non-dominance is an inadequate strategy for convergence to the Pareto front for such problems, as almost all solutions in the population become non-dominated, resulting in loss of convergence pressure. However, for some problems, it may be possible to generate the Pareto front using only a few of the objectives, rendering the rest of the objectives redundant. Such problems may be reducible to a manageable number of relevant objectives, which can be optimized using conventional multiobjective evolutionary algorithms (MOEAs). For dimensionality reduction, most proposals in the paper rely on analysis of a representative set of solutions obtained by running a conventional MOEA for a large number of generations, which is computationally overbearing. A novel algorithm, Pareto corner search evolutionary algorithm (PCSEA), is introduced in this paper, which searches for the corners of the Pareto front instead of searching for the complete Pareto front. The solutions obtained using PCSEA are then used for dimensionality reduction to identify the relevant objectives. The potential of the proposed approach is demonstrated by studying its performance on a set of benchmark test problems and two engineering examples. While the preliminary results obtained using PCSEA are promising, there are a number of areas that need further investigation. This paper provides a number of useful insights into dimensionality reduction and, in particular, highlights some of the roadblocks that need to be cleared for future development of algorithms attempting to use few selected solutions for identifying relevant objectives. [ABSTRACT FROM AUTHOR]

Manifold learning is a hot topic in the field of computer science, particularly since nonlinear dimensionality reduction based on manifold learning was proposed in Science in 2000. The work has achieved great success. The main purpose of current manifold-learning approaches is to search for independent intrinsic variables underlying high dimensional inputs which lie on a low dimensional manifold. In this paper, a new manifold is built up in the training step of the process, on which the input training samples are set to be close to each other if the values of their intrinsic variables are close to each other. Then, the process of dimensionality reduction is transformed into a procedure of preserving the continuity of the intrinsic variables. By utilizing the new manifold, the dynamic tracking of a human who can move and rotate freely is achieved. From the theoretical point of view, it is the first approach to transfer the manifold-learning framework to dynamic tracking. From the application point of view, a new and low dimensional feature for visual tracking is obtained and successfully applied to the real-time tracking of a free-moving object from a dynamic vision system. Experimental results from a dynamic tracking system which is mounted on a dynamic robot validate the effectiveness of the new algorithm. [ABSTRACT FROM AUTHOR]

Structural equation models (SEMs) and vector autoregressive models (VARMs) are two broad families of approaches that have been shown useful in effective brain connectivity studies. While VARMs postulate that a given region of interest in the brain is directionally connected to another one by virtue of time-lagged influences, SEMs assert that directed dependencies arise due to instantaneous effects, and may even be adopted when nodal measurements are not necessarily multivariate time series. To unify these complementary perspectives, linear structural vector autoregressive models (SVARMs) that leverage both instantaneous and time-lagged nodal data have recently been put forth. Albeit simple and tractable, linear SVARMs are quite limited since they are incapable of modeling nonlinear dependencies between neuronal time series. To this end, the overarching goal of the present paper is to considerably broaden the span of linear SVARMs by capturing nonlinearities through kernels, which have recently emerged as a powerful nonlinear modeling framework in canonical machine learning tasks, e.g., regression, classification, and dimensionality reduction. The merits of kernel-based methods are extended here to the task of learning the effective brain connectivity, and an efficient regularized estimator is put forth to leverage the edge sparsity inherent to real-world complex networks. Judicious kernel choice from a preselected dictionary of kernels is also addressed using a data-driven approach. Numerical tests on ECoG data captured through a study on epileptic seizures demonstrate that it is possible to unveil previously unknown directed links between brain regions of interest. [ABSTRACT FROM AUTHOR]

Multi-dimensional scaling (MDS) plays a central role in data-exploration, dimensionality reduction and visualization. State-of-the-art MDS algorithms are not robust to outliers, yielding significant errors in the embedding even when only a handful of outliers are present. In this paper, we introduce a technique to detect and filter outliers based on geometric reasoning. We test the validity of triangles formed by three points, and mark a triangle as broken if its triangle inequality does not hold. The premise of our work is that unlike inliers, outlier distances tend to break many triangles. Our method is tested and its performance is evaluated on various datasets and distributions of outliers. We demonstrate that for a reasonable amount of outliers, e.g., under 20 percent, our method is effective, and leads to a high embedding quality. [ABSTRACT FROM AUTHOR]

To solve a parametric model in computational electromagnetics, the finite-element (FE) method is often used. To reduce the computational time and the memory requirement, the FE method can be combined with the model-order reduction technique like the proper orthogonal decomposition and (discrete) empirical interpolation methods. These three numerical methods introduce the errors of discretization, reduction, and interpolation, respectively. The solution of the parametric model will be efficient if the three errors are of the same order and so they need to be evaluated and compared. In this paper, we propose an a posteriori error estimator based on the verification of the constitutive law, which estimates the three different errors. An example of application in magnetostatics with 11 parameters is treated where it is shown how the error estimator can be used to control and to improve the accuracy of the solution of the reduced model. [ABSTRACT FROM PUBLISHER]

In recent years, the support vector machines (SVMs) have been very successful in remote sensing image classification, particularly when dealing with high-dimensional data and limited training samples. Nevertheless, the vector-based feature alignment of the SVM can lead to an information loss in representation of hyperspectral images, which intrinsically have a tensor-based data structure. In this paper, a new multiclass support tensor machine (STM) is specifically developed for hyperspectral image classification. Our newly proposed STM processes the hyperspectral image as a data cube and then identifies the information classes in tensor space. The multiclass STM is developed from a set of binary STM classifiers using the one-against-one parallel strategy. As a part of our tensor-based processing chain, a multilinear principal component analysis (MPCA) is used for preprocessing, in order to reduce the tensorial data redundancy and, at the same time, preserve the tensorial structure information in sparse and high-order subspaces. As a result, the contributions of this work are twofold: a new multiclass STM model for hyperspectral image classification is developed, and a tensorial image interpretation framework is constructed, which provides a system consisting of tensor-based feature representation, feature extraction, and classification. Experiments with four hyperspectral data sets, covering agricultural and urban areas, are conducted to validate the effectiveness of the proposed framework. Our experimental results show that the proposed STM and MPCA-STM can achieve better results than traditional SVM-based classifiers. [ABSTRACT FROM AUTHOR]

Additive models are regression methods which model the response variable as the sum of univariate transfer functions of the input variables. Key benefits of additive models are their accuracy and interpretability on many real-world tasks. Additive models are however not adapted to problems involving a large number (e.g., hundreds) of input variables, as they are prone to overfitting in addition to losing interpretability. In this paper, we introduce a novel framework for applying additive models to a large number of input variables. The key idea is to reduce the task dimensionality by deriving a small number of new covariates obtained by linear combinations of the inputs, where the linear weights are estimated with regard to the regression problem at hand. The weights are moreover constrained to prevent overfitting and facilitate the interpretation of the derived covariates. We establish identifiability of the proposed model under mild assumptions and present an efficient approximate learning algorithm. Experiments on synthetic and real-world data demonstrate that our approach compares favorably to baseline methods in terms of accuracy, while resulting in models of lower complexity and yielding practical insights into high-dimensional real-world regression tasks. Our framework broadens the applicability of additive models to high-dimensional problems while maintaining their interpretability and potential to provide practical insights. [ABSTRACT FROM PUBLISHER]

Classification of remotely sensed hyperspectral images via supervised approaches is typically affected by high dimensionality of the spectral data and a limited number of labeled samples. Dimensionality reduction via feature extraction and active learning (AL) are two approaches that researchers have investigated independently to deal with these two problems. In this paper, we propose a new method in which the feature extraction and AL steps are combined into a unique framework. The idea is to learn and update a reduced feature space in a supervised way at each iteration of the AL process, thus taking advantage of the increasing labeled information provided by the user. In particular, the computation of the reduced feature space is based on the large-margin nearest neighbor (LMNN) metric learning principle. This strategy is applied in conjunction with $k$-nearest neighbor ( $k$-NN) classification, for which a new sample selection strategy is proposed. The methodology is validated experimentally on four benchmark hyperspectral data sets. Good improvements in terms of classification accuracy and computational time are achieved with respect to the state-of-the-art strategies that do not combine feature extraction and AL. [ABSTRACT FROM PUBLISHER]

The availability of hyperspectral images with improved spectral and spatial resolutions provides the opportunity to obtain accurate land-cover classification. In this paper, a novel methodology that combines spectral and spatial information for supervised hyperspectral image classification is proposed. A feature reduction strategy based on independent component analysis is the main core of the spectral analysis, where the exploitation of prior information coupled to the evaluation of the reconstruction error assures the identification of the best class-informative subset of independent components. Reduced attribute profiles (APs), which are designed to address well-known issues related to information redundancy that affect the common morphological APs, are then employed for the modeling and fusion of the contextual information. Four real hyperspectral data sets, which are characterized by different spectral and spatial resolutions with a variety of scene typologies (urban, agriculture areas), have been used for assessing the accuracy and generalization capabilities of the proposed methodology. The obtained results demonstrate the classification effectiveness of the proposed approach in all different scene typologies, with respect to other state-of-the-art techniques. [ABSTRACT FROM PUBLISHER]

Despite the availability of an increasing amount of remote sensing images, problems still arise in that the knowledge from existing images is underutilized and the collection of reference knowledge for each newly obtained image is expensive. Recently, an attractive solution called “transfer learning” has received increasing attention in the remote sensing field, by transferring knowledge from source domains to help improve the learning procedure in the target domain. In this paper, we propose a sparse subspace correlation analysis-based supervised classification (SSCA-SC) method for transfer learning in hyperspectral remote sensing image classification, which is not restricted by the data dimensionality or the data acquisition sensors. Specifically, we first propose a sparse subspace correlation analysis (SSCA) method to simultaneously learn the optimal projection matrices for heterogeneous domains into a common subspace and obtain sparse reconstruction coefficients over a shared self-expressive dictionary in the derived subspace. In order to fully utilize the label information to improve the class separability, the SSCA-SC framework learns more discriminative representations for the input data by training a corresponding SSCA model for each class. As a result, the projected data belonging to the same class are maximally correlated and represented well, while those from different classes will have a low correlation. Another advantage of the SSCA-SC framework lies in the fact that it not only learns new representations for the data from different domains but it also designs a discriminative and robust classifier that properly adapts to the new representation. The proposed method was tested with three hyperspectral remote sensing data sets, and the experimental results confirm the effectiveness and reliability of the proposed SSCA-SC method. [ABSTRACT FROM AUTHOR]

This paper considers the problem of estimating a high-dimensional covariance matrix in a low sample support situation where the sample size is smaller, or not much larger, than the dimensionality of the data, which could potentially be very large. We develop a regularized sample covariance matrix (RSCM) estimator which can be applied in commonly occurring high-dimensional data problems. The proposed RSCM estimator is based on estimators of the unknown optimal (oracle) shrinkage parameters that yield the minimum mean squared error between the RSCM and the true covariance matrix when the data is sampled from an unspecified elliptically symmetric distribution. We propose two variants of the RSCM estimator which differ in the approach in which they estimate the underlying sphericity parameter involved in the theoretical optimal shrinkage parameter. The performance of the proposed RSCM estimators are evaluated with numerical simulation studies. In particular, when the sample sizes are low, the proposed RSCM estimators often show a significant improvement over the conventional RSCM estimator by Ledoit and Wolf (2004). We further evaluate the performance of the proposed estimators in a portfolio optimization problem with real data wherein the proposed methods are able to outperform the benchmark methods. [ABSTRACT FROM AUTHOR]

Dimensionality reduction is a critical technology in the domain of pattern recognition, and linear discriminant analysis (LDA) is one of the most popular supervised dimensionality reduction methods. However, whenever its distance criterion of objective function uses $L_2$ -norm, it is sensitive to outliers. In this paper, we propose a new formulation of linear discriminant analysis via joint $L_{2,1}$ -norm minimization on objective function to induce robustness, so as to efficiently alleviate the influence of outliers and improve the robustness of proposed method. An efficient iterative algorithm is proposed to solve the optimization problem and proved to be convergent. Extensive experiments are performed on an artificial data set, on UCI data sets, and on four face data sets, which sufficiently demonstrates the efficiency of comparing to other methods and robustness to outliers of our approach. [ABSTRACT FROM AUTHOR]

This paper provides a proof of global convergence of gradient search for low-rank matrix approximation. Such approximations have recently been of interest for large-scale problems, as well as for dictionary learning for sparse signal representations and matrix completion. The proof is based on the interpretation of the problem as an optimization on the Grassmann manifold and Fubiny–Study distance on this space. [ABSTRACT FROM PUBLISHER]

Image-based 3-D human pose recovery is usually conducted by retrieving relevant poses with image features. However, it suffers from the high dimensionality of image features and the low efficiency of the retrieving process. Particularly for multiview data, the integration of different types of features is difficult. In this paper, a novel approach is proposed to recover 3-D human poses from silhouettes. This approach improves traditional methods by adopting multiview locality-sensitive sparse coding in the retrieving process. First, it incorporates a local similarity preserving term into the objective of sparse coding, which groups similar silhouettes to alleviate the instability of sparse codes. Second, the objective function of sparse coding is improved by integrating multiview data. The experimental results show that the retrieval error has been reduced by 20% to 50%, which demonstrate the effectiveness of the proposed method. [ABSTRACT FROM PUBLISHER]

In high voltage (HV) technology, the electrical field distribution can be improved using different field grading methods. These can be categorized into two groups: 1) capacitive or geometrical field grading and 2) resistive field grading. To obtain the optimized field grading effects, some geometrical or material parameters of the HV devices should be optimized. However, these devices are often nonrotationally symmetric and simulation is computationally very time consuming. In this paper, a multilevel surrogate method using co-Kriging methodology is proposed to optimize such large-scale 3-D HV devices. To compute the electrical field distribution of these HV devices, a finite-element method simulator can be run at different levels of complexity, i.e., by reducing the 3-D model into a 2-D model under certain additional assumptions. The co-Kriging method combines expensive runs of highly complex 3-D simulations with relatively inexpensive dimension reduced 2-D simulations. This approach is shown to allow for a faster optimization of a large-scale nonrotationally symmetric problem, while preserving a sufficiently high level of accuracy. [ABSTRACT FROM PUBLISHER]

Recently, low-rank embedding (LRE) has yielded satisfactory results in dimensionality reduction (DR), for which low-rank representation and projection learning are integrated into one model to generate robust low-dimensional features. However, LRE requires to convert samples into vectors even if the data naturally appear in high-order form. Furthermore, LRE fails to take the label information into consideration. To address these problems, this paper proposes a novel supervised DR method based on multilinear algebra, i.e., the algebra of tensors. By the motivation of extending LRE into tensor space and simultaneously combining the tensor discriminant analysis, we establish tensor low-rank discriminant embedding (TLRDE) model for hyperspectral image (HSI) DR. The model of TLRDE is solved by an alternative iteration algorithm, whose convergence is also mathematically proven. The proposed TLRDE method employs the tensor representation to preserve the intrinsic geometrical structure, uses low-rank reconstruction to uncover the potential relationship among the data points, and combines label information to enhance the discriminability of features. Moreover, the proposed TLRDE does not suffer from the small sample size problem. The experimental results on three real HSI data sets validate the effectiveness of our proposed TLRDE method. [ABSTRACT FROM AUTHOR]

Smart grid (SG) is an integration of traditional power grid with advanced information and communication infrastructure for bidirectional energy flow between grid and end users. A huge amount of data is being generated by various smart devices deployed in SG systems. Such a massive data generation from various smart devices in SG systems may lead to various challenges for the networking infrastructure deployed between users and the grid. Hence, an efficient data transmission technique is required for providing desired QoS to the end users in this environment. Generally, the data generated by smart devices in SG has high dimensions in the form of multiple heterogeneous attributes, values of which are changed with time. The high dimensions of data may affect the performance of most of the designed solutions in this environment. Most of the existing schemes reported in the literature have complex operations for the data dimensionality reduction problem which may deteriorate the performance of any implemented solution for this problem. To address these challenges, in this paper, a tensor-based big data management scheme is proposed for dimensionality reduction problem of big data generated from various smart devices. In the proposed scheme, first the Frobenius norm is applied on high-order tensors (used for data representation) to minimize the reconstruction error of the reduced tensors. Then, an empirical probability-based control algorithm is designed to estimate an optimal path to forward the reduced data using software-defined networks for minimization of the network load and effective bandwidth utilization. The proposed scheme minimizes the transmission delay incurred during the movement of the dimensionally reduced data between different nodes. The efficacy of the proposed scheme has been evaluated using extensive simulations carried out on the data traces using ‘R’ programming and Matlab. The big data traces considered for evaluation consist of more than two million entries (2,075,259) collected at one minute sampling rate having hetrogenous features such as–voltage, energy, frequency, electric signals, etc. Moreover, a comparative study for different data traces and a real SG testbed is also presented to prove the efficacy of the proposed scheme. The results obtained depict the effectiveness of the proposed scheme with respect to the parameters such as- network delay, accuracy, and throughput. [ABSTRACT FROM AUTHOR]