Search

• A methodology to forecast time series with generalized neural networks is outlined. • A clever way of feature selection is proposed. • Strategies for dealing with time series with a trend are developed. This paper discusses how to forecast time series using generalized regression neural networks. The main goal is to take advantage of their inherent properties to generate fast, highly accurate forecasts. To this end, the key modeling decisions involved in forecasting with generalized regression neural networks are described. To deal with every modeling decision, several strategies are proposed. Each strategy is analyzed in terms of forecast accuracy and computational time. Apart from the modeling decisions, any successful time series forecasting methodology has to be able to capture the seasonal and trend patterns found in a time series. In this regard, some clever techniques to cope with these patterns are also suggested. The proposed methodology is able to forecast time series in an automatic way. Additionally, the paper introduces a publicly available R package that incorporates the best presented modeling approaches and transformations to forecast time series with generalized regression neural networks. [ABSTRACT FROM AUTHOR]

• This paper proposes a new feature selection method called SLMEA. • The sparse transformation and pseudo label matrix are combined to optimize. • The pseudo label matrix is used to guide the learning of low-dimensional space. • The similarity matrix is constructed adaptively. • The constraint of ℓ 2 , 1 / 2 -norm is applied to the matrix. • The SLMEA can have higher clustering accuracy and normalized mutual information. Traditional feature selection algorithms usually explore the relationship between data and cluster structure in a single space, so the internal relationship obtained is not very rich, and it is not enough to select more valuable features. To solve the above problems, in order to fully mine the intrinsic correlation information in different spaces, so that the relationship between data, features and clustering structure can be explored at the same time, this paper proposes a feature selection method based on sparse and low-dimensional representation with maximum entropy adaptive graph (SLMEA). Firstly, the SLMEA combines the sparse transform representation with pseudo-label matrix learning to optimize, and uses the pseudo-label matrix to guide the learning of sparse low-dimensional space. It can not only explore the relationship between data and pseudo-labels in data space, but also mine the association between features and pseudo-labels in feature space, so as to select more discriminative features. Secondly, based on the maximum entropy theory, the similarity matrix is constructed adaptively, so that the two different manifold structures corresponding to sparse transform representation and pseudo label matrix learning can be adaptively learned and retained in the iterative process. In addition, in order to ensure the sparsity of the transformation matrix, the constraint of ℓ 2 , 1 / 2 -norm is applied to the matrix, which can better deal with the redundant features and obtain more sparse solutions. Finally, the SLMEA uses an alternate iterative update method to optimize the objective function, and carries out extensive experiments on eight mainstream datasets. Compared with seven state-of-the-art algorithms, SLMEA can have higher clustering accuracy and normalized mutual information. [ABSTRACT FROM AUTHOR]

This paper introduces a novel feature selection method, called Feature Selection based on Importance Measures (FS-IM), to enhance the forecasting of crude oil returns. FS-IM innovatively combines active learning with the application of Gaussian noise to input features and selects the most relevant features using an optimal threshold value. The paper applies a ridge regression (RR) model based on FS-IM (FS-RR) to identify the factors that have important information for crude oil return forecasting. The paper compares FS-IM with other dimension reduction methods such as Principal Component Analysis (PCA), Kernel Principal Component Analysis (KPCA), and Independent Component Analysis (ICA). The results show that FS-IM can significantly improve model accuracy, demonstrating its effectiveness in finding key features. Moreover, FS-IM is more stable and consistent than other dimension reduction methods in enhancing the prediction accuracy in different scenarios, indicating its superior capability in capturing complex relationships between input and output variables. Furthermore, this study compares FS-RR model with other 13 prediction models by conducting experiments using a series of evaluation metrics, different statistical tests, and different step-ahead predictions and training sets. The results confirm that the RR model based on FS-IM can consistently outperform other model in terms of predictive performance and economic value, proving its effectiveness and robustness. This study contributes to the literature on crude oil price forecasting by addressing the challenges of high-dimensional and complex data, and by providing a robust, practical tool for professionals in energy economics and finance. • A new feature selection method for crude oil forecasting using importance scores and Gaussian noise is proposed. • The method selects the most relevant features based on the prediction errors of a base forecasting model. • The method enhances the accuracy and value of ridge regression model compared to other methods. • The method handles high-dimensional and complex data without losing information or interpretability. [ABSTRACT FROM AUTHOR]

• An analysis of lexicographic optimization for feature selection and data classification. • Cooperative co-evolution for feature selection while the classifier hyperparameters are also being optimized. • Optimization of multiple objectives using a simple evolutionary algorithm. • A new lexicographic ranking methodology to compare the results of different feature selection methods. • Quite good results for real high-dimensional datasets. This paper starts with two hypotheses. The first one is that the simultaneous optimization of the hyperparameters regulating the classifier within a wrapper method, while the best subset of features is being determined, should improve the results with respect to those obtained with a pre-parameterized classifier. The second one is that solving these two problems can be formulated as a lexicographic optimization problem, allowing the use of a simple single-objective evolutionary algorithm to solve this multi-objective problem. The fitness function is of key importance for such wrapper methods. It is responsible for guiding the search towards potentially good solutions and it also consumes most of the runtime. Having these issues in mind, this paper also proposes a new lexicographic fitness function, designed to minimize the runtime of the algorithm and also to avoid over-fitting. Furthermore, the execution time and the quality of the results obtained by the wrapper procedure also depend on some algorithmic hyperparameters: the similarity thresholds used when comparing two different solutions lexicographically and the percentage of data samples used for validation during the training process. Thus, an experimental analysis has been carried out to find adequate values for these hyperparameters. Finally, the lexicographic cooperative co-evolutionary wrapper approach, using the new fitness function proposed in this paper, has been tested with several datasets belonging to the University of California, Irvine (UCI) repository and also with some real high-dimensional datasets, obtaining quite good results, compared to other state-of-the-art wrapper methods. The comparison has also been made lexicographically, with a new methodology proposed in this paper. [ABSTRACT FROM AUTHOR]

In numerous domains, there is ample evidence indicating that a significant portion of real-world data exhibits temporal variations, such as medical research and meteorological studies. Particularly in the era of big data, not only does the size of data change dynamically but also its dimensions at an unprecedented pace. Consequently, employing traditional methods to handle such dynamic data becomes highly impractical. To address this limitation, this paper proposes an incremental feature selection algorithm tailored for dynamic feature variations. For scenarios involving an increase in features, the novel algorithm efficiently identifies a target subset with effective features. In order to showcase the efficacy of the proposed algorithm, this paper conduct experiments using four commonly used classifiers and five UCI data sets. The experimental results further validate both the feasibility and efficiency of the new approach. [ABSTRACT FROM AUTHOR]

Many traditional unsupervised feature selection algorithms utilize manifold information to mine the local structure of the data. However, the noise existing in the raw data reduces the accuracy of the manifold information of data, which affects the learning effect of the entire algorithm. In order to solve the above problems and more fully find the internal structure inside the data, this paper proposes an adaptive graph regularization and self-expression for noise-aware feature selection (ASNFS). Firstly, the algorithm adopts non-negative matrix factorization to decompose the raw data matrix, and utilizes the low-dimensional matrix generated after decomposition to replace the raw high-dimensional data matrix. This allows the algorithm to reveal some internal structural information of the raw data while reducing the dimensionality of the data. ASNFS also introduces the orthogonal basis clustering with excellent clustering effect, and the interpretability of the algorithm is enhanced. Secondly, in addition to preserving the manifold information in the low-dimensional projection subspace, the algorithm also preserves the manifold information in the non-negative matrix factorization subspace. Meanwhile, the adaptive graph regularization term added to the objective function enables the algorithm to continuously update the similarity matrix. It can effectively remove the noise inside the raw data, and prevent the occurrence of over-fitting phenomenon of the experimental results caused by the fixed similarity matrix. Finally, the similarity matrix retains the local structure information of the data with each iteration, and the results of the feature selection are reused for the construction of the similarity matrix. ASNFS adopts an alternate iterative method to optimize the objective function, which is simple and effective. Then, the algorithm complexity and convergence are analyzed. ASNFS is compared with seven feature selection algorithms on nine datasets, and the experimental results reflect the effectiveness of ASNFS in feature selection. [ABSTRACT FROM AUTHOR]

Battery degradation, caused by multiple coupled degradation mechanisms, severely affects the safety and sustainability of a battery management system (BMS). The battery state of health (SOH) is a commonly-adopted metric to evaluate a battery's degradation condition, which should be carefully modeled to facilitate the safety and reliability of a BMS. Recently, owing to the rapid progress of data science-related techniques, data-driven models for battery SOH estimation have attracted great attentions from both academia and industry communities. This paper aims to provide the scientists and engineers with a general overview of data-driven battery SOH estimation technology for BMSs. State-of-the-art models published during 2018–2022 are reviewed with care, including a) feature extraction and selection methods; b) benchmarks, variants and extensions of data-driven SOH estimation models; and c) publicly-available battery SOH datasets. Afterwards, experiments are conducted and analyzed on the Toyota & Stanford-MIT battery SOH datasets for benchmark study. Finally, existing challenges and feature trends are summarized. [ABSTRACT FROM AUTHOR]

ℓ 1 regularization has been used for logistic regression to circumvent the overfitting and use the estimated sparse coefficient for feature selection. However, the challenge of such regularization is that the ℓ 1 regularization is not differentiable, making the standard convex optimization algorithm not applicable to this problem. This paper presents a simple projection neural network for ℓ 1 -regularized logistics regression. In contrast to many available solvers in the literature, the proposed neural network does not require any extra auxiliary variable nor smooth approximation, and its complexity is almost identical to that of the gradient descent for logistic regression without ℓ 1 regularization, thanks to the projection operator. We also investigate the convergence of the proposed neural network by using the Lyapunov theory and show that it converges to a solution of the problem with any arbitrary initial value. The proposed neural solution significantly outperforms state-of-the-art methods concerning the execution time and is competitive in terms of accuracy and AUROC. [ABSTRACT FROM AUTHOR]

• We extend the NRS model that can be skillfully applied to figure out label distribution data and avoid the instability of results caused by different neighborhood granularity selection. • Based on the nearest neighbor information distribution, a label enhancement method is designed to transform logical labels into label distribution, thereby obtaining more potential label information to guide feature selection. • A novel forward algorithm of multi-label feature selection based on the extended neighborhood rough set is proposed, which leverages feature significance, label significance, and label-specific features, simultaneously. Multi-label feature selection is an indispensable technology in multi-semantic high-dimensional data preprocessing, which has been brought into focus in recent years. However, most existing methods explicitly assume that the significance of all relevant labels is the same for every instance, while ignoring the real scenarios that the significance of available labels to each instance is usually different. In this paper, we propose a novel multi-label feature selection based on label distribution and neighborhood rough set, known as LDRS. To be specific, we first construct a label enhancement method based on instance information distribution to convert the logical labels of multi-label data into label distribution, thereby capturing label significance to provide additional information for learning tasks. Then, we extend the neighborhood rough set model for label distribution learning, and discuss the related properties in detail. This extended model can effectively avoid the selection of neighborhood granularity and seamlessly apply to handle label distribution data. After that, two feature significance measures are established to realize the quality evaluation of features and the fusion of label-specific features. Finally, a novel feature selection framework is designed, which takes into account feature significance, label significance, and label-specific features, simultaneously. Experiments on both public and real-world datasets exhibit the advantages of the proposed method. [ABSTRACT FROM AUTHOR]

As a powerful paradigm, deep learning (DL) models have been used in many applications for classification tasks in images, text, and audio. Through DL models, we can learn task-driven features from big data. However, DL models are fully deterministic and cannot handle uncertain and imprecise data. DL models are often sensitive to noise in data and do not operate well in areas where data are vague. Moreover, when there is a large feature set or high-dimensional data with irrelevant and redundant features, the DL models' performance decreases in classification due to training with irrelevant features. To gain reliable results in such high-dimensional problems, DL models require a large amount of data which usually grows exponentially concerning the number of features. Data uncertainty problems and irrelevant features in DL models cause low performance in classification tasks. This paper proposes an optimized fuzzy deep learning (OFDL) model for data classification based on Non-Dominated Sorting Genetic Algorithm II (NSGA-II). OFDL utilizes optimization in the composition of DL and fuzzy learning via the NSGA-II in multi-modal learning. To achieve effective classification, OFDL first considers intelligent feature selection by finding the best trade-offs between two conflicting objective functions, minimizing the number of features, and maximizing the accuracy (maximizing weights of selected features). Next, to reach optimized backpropagation and fuzzy membership functions, OFDL utilizes Pareto optimal solutions for multi-objective optimization using NSGA-II based on their objective functions. Furthermore, the fusion layer in OFDL fused optimized views of DL and fuzzy learning that provides a high-level representation of inputs and optimum features for classification tasks where the data contain high uncertainties and noises. This functionality gives valuable attributes during classification since identifying and selecting appropriate features ensures prompt and correct class. Also, it provides deep insight into tackling the effect of ambiguous data and each feature's uncertainty in the classification tasks. The examination of OFDL reveals good performance in terms of F-measure, accuracy, recall, precision, and True Positive Rate (TPR) compared to fuzzy classifiers. Furthermore, OFDL has higher accuracy in classification tasks than earlier fuzzy DNN models. [ABSTRACT FROM AUTHOR]

Crawler detection is always an important research topic in network security. With the development of web technology, crawlers are constantly updating and changing, and their types are becoming diverse. The diversity and dynamics of crawlers pose significant challenges for feature applicability and model robustness. Existing crawler detection methods can only detect a limited number of crawlers by predefined rules and can not cover all types of crawlers; worse, they can be completely invalidated by the emergence of new types of crawlers. In this paper, we propose a reinforcement learning based web crawler detection method for diversity and dynamics (WC3D), which is composed of a feature selector and a session classifier. The feature selector selects the appropriate feature set for different types of crawlers with deep deterministic policy gradient. The session classifier makes crawler detection and provides rewards to the feature selector. The two modules are trained jointly to optimize the feature selection and session classification processes. Extensive experiments demonstrate the existence of crawler diversity and that the proposed method is still highly robust against the new type of crawlers and achieves state-of-the-art performance even without considering the dynamics of the crawlers. [ABSTRACT FROM AUTHOR]

Feature selection plays an important role in reducing irrelevant and redundant features, while retaining the underlying semantics of selected ones. An effective feature selection method is expected to result in a significantly reduced subset of the original features without sacrificing the quality of problem-solving (e.g., classification). In this paper, a non-unique decision measure is proposed that captures the degree of a given feature subset being relevant to different categories. This helps to represent the uncertainty information in the boundary region of a granular model, such as rough sets or fuzzy-rough sets in an efficient manner. Based on this measure, the paper further introduce a differentiation entropy as an evaluator of feature subsets to implement a novel feature selection algorithm. The resulting feature selection method is capable of dealing with either nominal or real-valued data. Experimental results on both benchmark data sets and a real application problem demonstrate that the features selected by the proposed approach outperform those attained by state-of-the-art feature selection techniques, in terms of both the size of feature reduction and the classification accuracy. [ABSTRACT FROM AUTHOR]

Intrusion detection technology can actively detect abnormal behaviors in the network and is important to the security of Industrial Internet of Things (IIOT). However, there are some issues with the current intrusion detection technology for IIOT, such as extreme imbalance in the number of samples of different classes in the dataset, redundant and meaningless features in the samples, and the inability of traditional intrusion detection methods to meet the requirements of detection accuracy in the increasingly complex IIOT. In view of the extreme imbalance of classes, this paper applies the hierarchical clustering algorithm to the under-sampling technology, which reduces the number of majority samples while reducing the loss of information of majority samples, and solves the problem of missing detection and false detection of minority samples caused by sample imbalance. In order to avoid feature redundancy and interference, this paper proposes an optimal feature selection algorithm based on greedy thought. This algorithm can obtain the optimal feature subset of each type of data in the data set, thus eliminating redundant and interfering features. Aiming at the problem of insufficient detection ability of traditional detection methods, this paper proposes a deep neural network intrusion detection model based on the parallel connection of global and local subnetworks. This model obtains the overall benchmark detection of the dataset through the deep neural network, and then strengthens the detection effect of each subclass through the parallel connection of subnetworks, greatly improving the performance of the intrusion detection algorithm. The experimental results show that the method described in this paper can improve the intrusion detection for IIOT. [ABSTRACT FROM AUTHOR]

Intelligent fault diagnosis is a rapidly evolving field within power engineering. Using gas-in-oil data is a reliable method for transformer fault diagnosis that has been widely adopted in the power industry. However, traditional machine learning methods often suffer from low diagnostic accuracy due to the lack of a clear and effective feature set for gas-in-oil data as well as an imbalance between classes of sample size. To overcome this challenge, this paper proposes a novel transformer fault diagnosis model that utilizes a Filter-Wrapper Combined Feature Selection method and an AdaBoost integrated weighted broad learning system (AdaBoost-WBLS). More specifically, the original data is expanded to extract meaningful features, and the Filter-Wrapper combined feature selection method is used to eliminate preliminary redundancy, relevance, and significance of current features. The Wrapper algorithm is then used for precise screening to obtain the optimal feature subset, which effectively improves the quality of transformer features. Furthermore, to address the issue of imbalanced transformer samples, an improved BLS and AdaBoost integration method is introduced, and a fault diagnosis model based on AdaBoost-WBLS is proposed. Compared with existing power transformer fault diagnosis methods, the proposed method has a more accurate and balanced effect on fault classification. Overall, this paper provides a comprehensive and effective approach to transformer fault diagnosis, which has important implications for the reliability and safety of power systems. [ABSTRACT FROM AUTHOR]

Mutual information (MI) is a widely used similarity measure in pattern recognition. MI uses entropy as a measure of uncertainty to quantify the structural similarity of two vectors. Replacing entropy with variance as a measure of uncertainty, an analogous class of similarity measures can be derived and estimated by regression techniques. Recently, the non-linear piecewise constant regression (PWCR) has been proposed to drive similarity measures of this scheme, leading to competitive alternatives of MI. Although PWCR is based on binning, the optimal binning technique for certain problems remained an open question. In this paper, we show mathematically that the optimal binning needs to be aligned with the expected relationship between the vectors being compared. In general, approximately optimal binnings can be found by combinatorial optimization, and in certain cases the optimal binning can be determined by k-means clustering. The theoretical findings are supported by numerical experiments that show a 2–5% increase in the AUC scores in simulated pattern recognition scenarios and improved feature rankings in feature selection problems. The results suggest that the proposed binning techniques could improve the performance of PWCR-driven similarity measures in real-world applications. [ABSTRACT FROM AUTHOR]

• A novel Depth Guided Feature Selection Network (DGFSnet) for RGBD salient object detection is proposed. • The DGFSnet takes depth information as prior and dynamically selects the complementary RGB information for the RGBD salient object detection. • Two novel modules including Depth Weight Generation module (DWG) and Weight-guided Feature Aggregation module (WFA) are proposed for implementing DGFSnet. • Together with DWG and WFA, the DGFSnet is able to adaptively enhance and integrate complementary RGB features for effectively highlighting the salient regions. • Comprehensive experiments over five popular RGBD benchmarks illustrating that DGFSnet consistently outperforms the state-of-the-art methods on all datasets. Depth information can greatly benefit the saliency detection in RGBD images if they are utilized well. Prevalent methods generally directly fuse depth and RGB features in networks. However, due to the inherent inconsistent between RGB and depth information, the RGB features are easy to be interfered by the intrinsic noise existed in depth features, making the precise RGBD saliency detection still a challenge. In this paper, we propose a novel Depth Guided Feature Selection network (DGFSnet) that takes depth information as prior and dynamically selects the complementary RGB information for RGBD salient object detection. Specifically, DGFSnet first includes a Depth Weight Generation module (DWG) to learn a set of layer-specific weights from multi-scale depth features. Guiding by these learned weights, DGFSnet further devises a Weight-guided Feature Aggregation module (WFA) to assign them to their corresponding RGB layers for dynamically enhancing and selecting saliency-related RGB features. With two modules, DGFSnet is able to effectively integrate the multi-modality complementaries and further highlight salient regions. Experimental results over seven popular RGBD salient object detection benchmarks demonstrate that DGFSnet fairly locates salient regions and effectively segments the complete object. [ABSTRACT FROM AUTHOR]

• A new model agnostic feature selection method for supervised learning. • Sensitivity analysis with the predictive distribution of a scalable Gaussian process. • New L 2 divergence derivation to better measure classification results and changes. • Better performance on several learning models than existing methods. Feature selection is one of the most important issues in supervised learning and there are a lot of different feature selection approaches in the literature. Among them one recent approach is to use Gaussian process (GP) because it can capture well the hidden relevance between the features of the input and the output. However, the existing feature selection approaches with GP suffer from the scalability problem due to high computational cost of inference with GP. Moreover, they use the Kullback–Leibler (KL) divergence in the sensitivity analysis for feature selection, but we show in this paper that the KL divergence underestimates the relevance of important features in some cases of classification. To remedy such drawbacks of the existing GP based approaches, we propose a new feature selection method with scalable variational Gaussian process (SVGP) and L 2 divergence. With the help of SVGP the proposed method exploits given large data sets well for feature selection through so-called inducing points while avoiding the scalability problem. Moreover, we provide theoretical analysis to motivate the choice of L 2 divergence for feature selection in both classification and regression. To validate the performance of the proposed method, we compare it with other existing methods through experiments with synthetic and real data sets. [ABSTRACT FROM AUTHOR]

Feature selection (FS) may improve the performance, cost-efficiency, and understandability of supervised machine learning models. In this paper, FS for the recently introduced distance-based supervised machine learning model is considered for regression problems. The study is contextualized by first providing an umbrella review (review of reviews) of recent development in the research field. We then propose a saliency-based one-shot wrapper algorithm for FS, which is called MAS-FS. The algorithm is compared with a set of other popular FS algorithms, using a versatile set of simulated and benchmark datasets. Finally, experimental results underline the usefulness of FS for regression, confirming the utility of certain filter algorithms and particularly the proposed wrapper algorithm. [ABSTRACT FROM AUTHOR]

Most of the existing unsupervised feature selection methods learn the cluster structure through spectral clustering, and then use various regression models to introduce the data matrix into the indicator matrix to obtain feature selection matrix. In these methods, the clustering indicator matrix is usually continuous value, which is not the best choice for the matrix in terms of its supervising role in feature selection. Based on this, unsupervised feature selection via discrete spectral clustering and feature weights (FSDSC) is proposed in this paper. First, FSDSC integrates regression model and spectral clustering in a unified framework for feature selection, and introduces a feature weight matrix, which intuitively expresses the importance of each feature with its diagonal elements. Compared with the common feature selection matrix that requires constraints such as sparse regular items, the appearance of the feature weight matrix reduces the complexity of the model and simplifies the calculation process of feature evaluation. Secondly, for the value of the indicators matrix, the spectral clustering is improved to obtain a discrete clustering indicator matrix, which provides clearer guidance information for feature selection. Finally, in order to avoid trivial solutions, the transformation matrix is constrained by orthogonal constraint. The combination of the orthogonal regression model and spectral clustering enables the algorithm to perform feature selection and manifold information learning at the same time, thereby preserving the local geometric structure of data. Compared with other excellent unsupervised feature selection algorithms, the experimental results prove the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

• Unsupervised feature selection based on the distance of the selected features and an orthogonal set. • Feature selection is performed through the matrix factorization of the generated orthogonal set. • A dual-correlation model for considering both the correlation among the features and samples. Feature selection has been an outstanding strategy in eliminating redundant and inefficient features in high-dimensional data. This paper introduces a novel unsupervised feature selection based on the matrix factorization, namely Unsupervised Feature Selection Guided by Orthogonal Representation (UFGOR). The orthogonality between a pair of variables refers to a specific case of linear independence such that they are perfectly uncorrelated. Motivated by the benefits of the orthogonality concept, the proposed UFGOR method is established based on the distance between the selected feature set and an orthogonal set corresponding to the whole feature space. Moreover, this orthogonal set is generated via QR-matrix factorization over the whole features and is employed as the compact representation of data matrix. In the next step, an unsupervised feature selection method is performed through the matrix factorization of the generated orthogonal set. Additionally, a dual-correlation model is utilized in the objective function of UFGOR to simultaneously consider both the local correlation in a set of selected features and the global correlation among the samples of a data. A detailed convergence analysis in line with an effective iterative algorithm proposed for the UFGOR method is also given. Numerical experiments on several real-world datasets illustrate the superior efficiency of our approach in comparison with some state-of-the-art unsupervised feature selection methods. [ABSTRACT FROM AUTHOR]

Feature selection has been recognized for long as an important preprocessing technique to reduce dimensionality and improve the performance of regression and classification tasks. The class of sequential forward feature selection methods based on Mutual Information (MI) is widely used in practice, mainly due to its computational efficiency and independence from the specific classifier. A recent work introduced a theoretical framework for this class of methods which explains the existing proposals as approximations to an optimal target objective function. Such framework made clear the advantages and drawbacks of each proposal. Methods accounting for the redundancy of candidate features using a maximization function and considering the so-called complementary effect are among the best ones. However, they still penalize the complementarity, which is an important drawback. This paper proposes the Decomposed Mutual Information Maximization (DMIM) method, which keeps the good theoretical properties of the best methods proposed so far but overcomes the complementarity penalization by applying the maximization separately to the inter-feature and class-relevant redundancies. DMIM was extensively evaluated and compared with other methods, both theoretically and using two synthetic scenarios and 20 publicly available real datasets applied to specific classifiers. Our results show that DMIM achieves a better classification performance than the remaining forward feature selection methods based on MI. [ABSTRACT FROM AUTHOR]

Developing an efficient embedded feature selection method for both binary and multiclass classification problems is a fundamental topic to be further studied. In this paper, the conventional radial basis function (RBF) network is modified for joint feature selection and classification. By utilizing the anisotropic Gaussian basis function, an individual weight parameter is assigned to each feature for feature weighting. The L 1 -regularized loss function is designed by combining the softmax cross entropy loss with L 1 regularization terms imposing on feature and output weights. A specialized active-set limited-memory projected Quasi-Newton (SAL-PQN) algorithm is proposed to minimize the L 1 -regularized loss function, which can optimize all the adjustable parameters and zero out the redundant feature and output weights simultaneously. Furthermore, a safe dynamic pruning strategy is integrated into the SAL-PQN algorithm for dynamic sparse training of the modified RBF network, which continuously excludes the dynamically zeroed-out feature weights from the training process. It is theoretically assured of yielding the same gradient updates as SAL-PQN, thus the relevant computation of the zeroed-out feature weights in the forward and backward propagations can be safely eliminated. The experimental results demonstrate the effectiveness and efficiency of the proposed methods for joint feature selection and classification, and illustrate the practical utility of the inherent feature weighting capability. [ABSTRACT FROM AUTHOR]

The sparsity and redundancy of high-dimensional data poses challenges during processing. Dimensionality reduction is a commonly used approach, which helps to reduce data redundancy and noise, lower the complexity of learning algorithms, and improve classification accuracy. Typically, directly determining the intrinsic dimensionality of a dataset is challenging. Many of the present dimensionality reduction techniques suffer from pre-set target dimensions without considering the specific dataset features. To address this problem, a novel dimensionality reduction method based on flow model, which is called adaptive flow encoder (AFE), is proposed in this paper. Initially, the flow model is employed to achieve feature transformation and project the data onto an equip-dimensional feature space. Subsequently, a gating layer is utilized for feature selection. By configuring the gating layer parameters to be learnable, the model gains the ability to adaptively select features. To assess the efficacy of the proposed approach, the model's ability to preserve data characteristics is first qualitatively evaluated through visual inspection by projecting the data onto a two-dimensional space. Subsequently, five standard datasets are employed and compared against six traditional dimensionality reduction techniques to ascertain the impact on downstream classification tasks under the adaptive learning of intrinsic feature dimensions by the model. Moreover, the impact of key parameters, including the initialization scheme of the gate layer, amplification factor, and dimension factor, on the test results is examined. [ABSTRACT FROM AUTHOR]

Supervised feature selection (FS) as an interpretable dimensionality reduction technique has received increasing attention, where linear discriminative analysis (LDA)-based method can select informative features discriminatively and obtain promising performance. When original data has more features than samples, however, LDA-based method generally encounters degradation since the appearance of irreversible scatter matrix. This situation is called the small sample size (SSS) problem. To overcome it and enhance the discriminant power of selected feature subsets, in this paper, we design an elegant LDA-based FS model referred to as Top- k Discriminative FS (TDFS), which is constructed by seamlessly integrating the ℓ 2 , 0 -norm equation constraint into uncorrelated LDA model. More concretely, the ℓ 2 , 0 -norm equation constraint can explicitly characterize the number of selective features k to ensure the sparsity of projected matrix and select top features. The uncorrelated LDA model aims to improve discriminative ability based on uncorrelated data in projected subspace. Given the formidable nature of solving this non-convex model, a novel optimization algorithm is further developed and the SSS problem can be efficaciously addressed during the optimization process. We first decompose projection matrix into a discrete selection matrix and its corresponding nonzero projection matrix, then concurrently optimize above two matrices by employing a column-by-column update scheme, during which the reversibility of scatter matrix in selective feature subspace can be easily guaranteed to solve SSS problem. The extensive experiments on four synthetic data sets and eight real-world data sets show that the proposed method outperforms eight competitors validated by three classifiers. Moreover, although the theoretical analysis proves that our algorithm has quartic time complexity on the number of selected features k , the running time experiments verify that TDFS is still efficient and applicable in scenarios where only a small number of features need to be selected. From above perspectives, our algorithm shows desirable performance to achieve discriminative FS. [ABSTRACT FROM AUTHOR]

Feature selection is a popular preprocessing step to reduce the dimensionality of the data while preserving the important information. In this paper, we propose an efficient and green feature selection method based on information theory, with the novelty of using the logarithmic division and resorting to fixed-point precision. Moreover, we extend these advancements by adapting the Mutual Information calculation for federated scenarios. The results of experiments conducted on several datasets indicate the potential of our proposal, as it does not incur significant information loss compared to the double-precision method, both in the features selected and in the subsequent classification step. Our method has shown significant potential when applied in federated scenarios, where experimentation demonstrated a lossless feature selection process and maintains classification results compared with centralised versions. These findings open up possibilities towards a new family of green feature selection methods, which would help to minimise energy consumption, lower carbon emissions and increase adaptability to Internet of Things environments. • Implementation of green feature selection methods based on information theory. • Study of logarithmic division to reduce energy and memory consumption. • Federated Mutual Information calculation enables IoT environments data privacy. • Feature selection with logarithmic division obtains similar results as double-precision. • Lossless and robust methods have been developed for federated scenarios. [ABSTRACT FROM AUTHOR]

Unsupervised feature selection, one of the important dimensionality reduction methods for high-dimensional data, has been paid more and more attention by researchers due to its effectiveness of processing the data without labels. However, most unsupervised feature selection methods do not take into account the fact that real data tends to be mixed with noise or outliers, or only consider the sensitivity of the model to them. In addition, the local manifold structure of data is usually preserved by a simple graph based on fixed k nearest neighbors. On the one hand, the different sample distributions of different data are considered without distinction. On the other hand, simple graph can not well capture the inherent relations between samples when facing multi-modal data. In view of the above problems, a robust unsupervised feature selection algorithm, sparse and minimum-redundant subspace learning with dual regularization (SMRSDR), is proposed in this paper. Specifically, a self-paced learning regularization based on soft weight allocation is introduced into SMRSDR to strictly control the samples' entry into model learning. Normal samples are allowed to enter model learning in order of their importance, while noise or outliers are strictly blocked out of model learning. Besides, considering the difference in sample distribution and the multimodal nature of data, the traditional simple graph Laplacian regularization is replaced by the hypergraph Laplacian regularization based on adaptive nearest neighbors selection. Furthermore, the sparsity and minimum-redundancy constraints are imposed on the subspace learning framework so that the most representative and minimum-redundant features can be selected. Finally, the objective function of SMRSDR is solved by an alternating iterative optimization algorithm. And a series of experiments are performed to comprehensively prove the effectiveness and superiority of SMRSDR. [ABSTRACT FROM AUTHOR]

To improve the classification performance of existing adaptive boosting (AdaBoost) based algorithms effectively, an improved AdaBoost algorithm based on misclassified samples oriented feature selection and weighted non-negative matrix factorization (WNMF) is proposed in this paper. Firstly, in order to consider the effects of sample weights, a misclassified samples oriented feature selection (called MOFS) is proposed to select the most discriminative features which occur in the samples with high weights. Secondly, the explicit features and the part-based features of the training samples are both considered, and the WNMF algorithm is introduced and combined with MOFS to reduce the dimension of the training sample set. Finally, the concept of misclassification degree is introduced and a fine grained sample weight updating method is proposed to distinguish the samples with different misclassification degrees. Numerical experiments show that the proposed MOFS method achieves higher accuracy compared to traditional feature selection methods, and the proposed MOFS and WNMF based AdaBoost method obtains significant improvement on classification accuracy when comparing with typical existing AdaBoost based algorithms using different classifiers. [ABSTRACT FROM AUTHOR]

• An advanced shuffled frog leaping algorithm (DSSRLFLA) is developed for model evaluation and feature selection. • A dynamic step size adjustment strategy based on historical information is designed to balance exploration and exploitation. • Specular reflection learning mechanism increases the possibility of searching for valid solutions and enhances the search ability. • An improved simulated annealing strategy is executed for each memetic to improve the efficiency of local exploitation. • DSSRLFLA can obtain higher quality optimal solutions compared to some competitive algorithms. The shuffled frog leaping algorithm is a new optimization algorithm proposed to solve the combinatorial optimization problem, which effectively combines the memetic algorithm based on a memetic algorithm and the particle swarm algorithm based on population behavior. The algorithm is widely used because it is easy to implement and requires few parameters to be adjusted. However, there are still some characteristics of this method that need to be improved because it is easy to fall into local optimization or poor search ability. To alleviate this limitation, a new version of the improved SFLA is proposed in this paper, which incorporates a dynamic step size adjustment strategy based on historical information, a specular reflection learning mechanism, and a simulated annealing mechanism based on chaotic mapping and levy flight. Firstly, the dynamic step size adjustment strategy based on historical information effectively helps to balance local exploration and global exploitation and alleviates the problem of falling into local optimum. Second, the specular reflection learning mechanism increases the possibility of searching for valid solutions in feasible domains and enhances the search ability of individuals in the population. Finally, an improved simulated annealing strategy is executed for each memetic, which improves the efficiency of local exploitation. In order to test the performance of the proposed algorithm, 31 test functions were selected from IEEE CEC2014 and 23 essential benchmark functions, and comparative experiments were carried out from the two dimensions of 30 and 100. Furthermore, a series of competing algorithms are selected, which involve nine classical standard algorithms, including PSO, BA, SSA, FA, SCA, WOA, GWO, MFO, and SFLA, as well as, six well-known improved algorithms, including LSFLA, DDSFLA, GOTLBO, ALCPSO, BLPSO, CLPSO. Furthermore, Wilcoxon signed-rank test and Friedman test are used as testing tools to illustrate the scalability of the proposed algorithm. From the analysis of the results, it can be seen that this proposed method has been effectively improved concerning stability and the quality of the optimal solution obtained from the search, both in low and high dimensions, and the ability to jump out of the local optimum has also been enhanced. In addition, to prove that this method has a reliable performance in discrete problems and continuous problems, DSSRLFLA is mapped into a discrete space, and 24 UCI data sets are also selected to evaluate the performance of the new feature selection method. The experimental results illustrate that this improved method can obtain fewer features and higher classification accuracy than some well-known feature selection methods. [ABSTRACT FROM AUTHOR]

Salient object detection is one of the basic challenging problems in the area of computer vision. Traditional saliency models usually utilize handcrafted features and various prior cues to locate and segment salient objects from complicated surroundings. Recently, with the development of convolutional neural networks (CNNs), most of existing approaches explore the combination of multi-level feature maps for improving detection performance. However, how to learn powerful feature is still a challenge, and the multi-level and multi-scale feature maps are usually aggregated without distinction. In this paper, we propose a novel salient object detection approach named residual attentive feature learning (RAFL) to uniformly highlight salient objects and effectively suppress background noises. In our proposed RAFL, a global perception (GP) module is designed to obtain rich global features of input image. A pyramid feature extraction (PFE) module is utilized to capture the rich context of side-output information. Moreover, an attentive feature selection (AFS) module is applied to refine these side-output features and generate attentive features, so as to find out the important feature channels having high responses to the salient objects. Finally, a residual feature learning (RFL) module is proposed to make the shallow layers generate more accurate saliency maps with the help of the deep layers which contain more global semantic information. Experimental results show that our proposed approach performs favorably against 18 state-of-the-art representative methods on five benchmark datasets. [ABSTRACT FROM AUTHOR]

The complexity of data has been increasing not only in the dimension of the data, but also in the interrelated class relationships. Pre-defined hierarchical structure of classes helps to model such relationships in a way, which divides the whole classification problem into a set of subtasks. Feature selection, known as a data preprocessing technique, aims to selecting useful feature to improve the performance of learning algorithms. However, the issue of close class relationships, especially under the same parent node in the hierarchical structure, is adverse to the feature selection. In this paper, we introduce a novel distance metric based feature selection method with large margin nearest neighbor strategy for the hierarchical classification problems. Different from other algorithms, our proposed method is to project the data into a low-dimensional feature space for learning correct metrics, and the weights of features will be indicated in the transformation matrix. A sparsity regularization and iterative optimization ensure that superior feature subsets can be selected. Experimental evaluation on several datasets with hierarchical class structure demonstrates that the proposed approach is comparable or better than some other state-of-the-art hierarchical feature selection methods. [ABSTRACT FROM AUTHOR]

• Subset-based feature selection + multi-layer perceptron. • Subset-based feature selection + radial basis function neural networks. • Semi-wrapper feature subset selection approach based on Naïve Bayes. • Extensive experiments on 34 data sets (22 from University of California at Irvine –UCI- and 12 synthetic data sets). Data preparation via feature selection + neural networks. • Feature selection: 7 approaches, which are 5 filters and 2 semi-wrappers (based on Correlation, consistency and Naïve Bayes; other with SOAP or SU), have been tested on difficult classification problems. This paper explores widely the data preparation stage within the process of knowledge discovery and data mining via feature subset selection in the context of two very well-known neural models: radial basis function neural networks and multi-layer perceptron. It is known the best performance of wrapper attribute selection methods based on the evaluation measure provided by a classifier, although the temporal complexity of learning neural networks practically precludes the use of wrapper techniques, especially in complex databases with high dimensionality and a large number of labels. In this paper, we propose the use of the Naïve Bayes classifier as a fitness function within a semi-wrapper feature selection approach. The Naïve Bayes classifier is a good fast approach to a neural network and utilising it as a measure of goodness in a backward search on a ranking provides a specific attribute selection method for neural networks in complex data. The test-bed consists of 34 binary and multi-class classification problems and 7 feature selectors. Of these, there are 6 data sets with upwards of 5 classes. According to the reported accuracy results that have been supported by non-parametric statistical tests in different scenarios, our method has been shown to be very suitable for both kinds of neural networks. Moreover, the reduced feature-space is around 20% of the full attribute space. The speedup with the aforementioned semi-wrapper is very outstanding and its value fluctuates, on average, from about 1.5 with radial basis function neural networks to around 30 with multi-layer perceptron. [ABSTRACT FROM AUTHOR]

Feature selection is of great importance for two possible scenarios: (1) prediction, i.e., improving (or minimally degrading) the predictions of a target variable while discarding redundant or uninformative features and (2) discovery, i.e., identifying features that are truly dependent on the target and may be genuine causes to be determined in experimental verifications (for example for the task of drug target discovery in genomics). In both cases, if variables have a large number of missing values, imputing them may lead to false positives; features that are not associated with the target become dependent as a result of imputation. In the first scenario, this may not harm prediction, but in the second one, it will erroneously select irrelevant features. In this paper, we study the risk/benefit trade-off of missing value imputation in the context of feature selection, using causal graphs to characterize when structural bias arises. Our aim is also to investigate situations in which imputing missing values may be beneficial to reduce false negatives, a situation that might arise when there is a dependency between feature and target, but the dependency is below the significance level when only complete cases are considered. However, the benefits of reducing false negatives must be balanced against the increased number of false positives. In the case of binary target variable and continuous features, the t-test is often used for univariate feature selection. In this paper, we also introduce a de-biased version of the t-test allowing us to reap the benefits of imputation, while not incurring the penalty of increasing the number of false positives. [ABSTRACT FROM AUTHOR]

Abstract Feature selection is an important step in building classifiers for high-dimensional data problems, such as EEG classification for BCI applications. This paper proposes a new wrapper method for feature selection, based on a multi-objective evolutionary algorithm, where the representation of the individuals or potential solutions, along with the breeding operators and objective functions, have been carefully designed to select a small subset of features that has good generalization capability, trying to avoid the over-fitting problems that wrapper methods usually suffer. A novel feature ranking procedure is also proposed in order to analyze the stability of the proposed wrapper method. Four different classification schemes have been applied within the proposed wrapper method in order to evaluate its accuracy and stability for feature selection on a real motor imagery dataset. Experimental results show that the wrapper method presented in this paper is able to obtain very small subsets of features, which are quite stable and also achieve high classification accuracy, regardless of the classifiers used. [ABSTRACT FROM AUTHOR]

Abstract Feature (gene) selection of microarray data is a very important and challenging task. This paper proposes a new feature selection method, partial maximum correlation information (PMCI), for microarray data classification. The proposed method extracts several orthogonal components from the feature space to evaluate the importance of each feature. In PMCI method, the component extraction is based on the correlation between feature space and class coding space. Meanwhile, this paper also provides a new generic class-encoding scheme for multi-class problems. This type of encoding scheme can ensure the smooth implementation for PMCI and has a clear geometric meaning. To validate the performance of the proposed method, this paper selects six state-of-the-art algorithms for comparison based on ten widely used microarray benchmark datasets for experiment. The experimental results show that PMCI is an effective feature selection method with good performance and lower time complexity. [ABSTRACT FROM AUTHOR]

Abstract Dealing with high-dimensional dataset has always been an important problem and feature selection is one of useful tools. In this paper, we develop a new filter based supervised feature selection method by combining Trace Ratio Criterion of Linear Discriminant Analysis (TRC-LDA) and group sparsity regularization. The filter based supervised feature selection method is a classifier-independent method while the TRC-LDA criterion is a recently developed criterion for dimensionality reduction that can well preserve discriminative information of dataset. However, there are seldom methods by utilizing TRC-LDA criterion for feature selection. On the other hand, imposing the l2,0-norm to the projection matrix of TRC-LDA will force some rows in it to be zero while keep other rows nonzero making the index of nonzero rows to be the selected features, however, l 2,0 -nom minimizing problem is NP-hard and intractable. To solve the above problem, in this paper, we develop a new method, namely, Trace Ratio Criterion Discriminative Feature Selection (TRC-DFS), for feature selection. The proposed TRC-DFS has imposed l 2,1 -norm, i.e. an approximation of l 2,0 -norm, to the projection matrix W of TRC-LDA to achieve feature selection. As a result, the proposed TRC-DFS can both achieve feature selection as well as capture the discriminative structure of data. We also extend the proposed method with l 2, p -norm (0 < p < 1) regularization to grasp more sparse property and develop an iterative approach to calculate the optimal solution, which is rigorously proved to be converged. Extensive simulations based on several real-world datasets verify the effectiveness of the proposed methods. [ABSTRACT FROM AUTHOR]

Abstract Feature selection is an essential pre-processing part in multi-label learning. Multi-label learning is usually used to deal with many complicated tasks, in which each sample is associated with multiple labels simultaneously. Fuzzy rough set model is one of the most effective ways for multi-label learning. However, it treats feature space and label space separately, and only uses features to describe sample structure information. In this paper, we fully consider the internal correlation between feature space and label space while fusing kernelized information from respective spaces. Moreover, we integrate fuzzy rough set with multiple kernel learning to finally realize feature selection. To be specific, firstly, we leverage one kind of kernel function to reveal the similarity between samples in feature space, and another one to assess the degree of label overlap between samples in label space. Secondly, we combine the kernelized information from the two spaces through linear combination to achieve precisely the lower approximation and construct a robust multi-label kernelized fuzzy rough set model, called RMFRS in this paper. Meanwhile, we discuss its properties and give theoretical analysis. Finally, we define a measurement criterion for selecting optimal features to evaluate the performance of the proposed algorithm. As many as 10 publicly available data sets are used to validate the effectiveness of our methods, and the result shows a distinct advantage over the state-of-the-art. [ABSTRACT FROM AUTHOR]

To describe objects more comprehensively and accurately, multi-view learning has attracted considerable attention. Recently, graph embedding based multi-view feature selection methods have been proposed and shown efficient in many real applications. The existing methods generally construct one common graph matrix to exploit the local structure of multi-view data via the linear weight fusion or learning one common graph matrix across all views. However, since all views share the identical graph structure, this emphasizes the consistency too much, resulting in restricting the diversity among different views. In this paper, a tensor low-rank constrained graph embedding method is proposed for multi-view unsupervised feature selection. To embody the view-specific information of each view, our model constructs the graph structure for each corresponding view, respectively. To capture the consistency across views, a tensor low-rank regularization constraint is imposed on the tensor data formed by these graph matrices. An efficient optimization algorithm with theoretical convergence guarantee is designed to solve the proposed method. Extensive experimental results validate that the proposed method outperforms some state-of-the-art methods. The code of our model can be found at https://www.researchgate.net/publication/353902948_demoTLR. [ABSTRACT FROM AUTHOR]

This study introduces Polar Lights Optimization (PLO), an algorithm based on the aurora phenomenon or polar lights. The aurora is a unique natural spectacle that occurs when energetic particles from the solar wind converge at the Earth's poles, influenced by the geomagnetic field and the Earth's atmosphere. By analyzing the motion of high-energy particles and delving into the underlying principles of physics, we propose a unique model for mimicking particle motion. This model integrates gyration motion and aurora oval walk, with the former facilitating local exploitation, while the latter enabling global exploration. By synergistically combining these two strategies, the proposed PLO achieves a balanced approach to local exploitation and global exploration. Additionally, a particle collision strategy is introduced to enhance the efficiency of escaping local optima. To evaluate the performance of PLO, a qualitative analysis experiment is designed to assess its ability to explore the problem space and search for solutions. PLO is compared against 9 classic algorithms and 8 high-performance algorithms using 30 benchmark functions from IEEE CEC2014. Furthermore, we compare and analyze PLO with the current state-of-the-art methods in the field, utilizing 12 benchmark functions from IEEE CEC2022. Subsequently, PLO is successfully applied to multi-threshold image segmentation and feature selection. Specifically, a PLO-based multi-threshold segmentation model and a binary PLO-based feature selection method are developed. The performance of PLO is also evaluated using 10 images from the Invasive Ductal Carcinoma (IDC) medical dataset, while the overall adaptability and accuracy of the feature selection model are tested using 8 medical datasets. These results affirm the emergence of PLO as an effective optimization tool ready for solving real-world problems, including those in the medical field. The source codes of PLO are available at https://aliasgharheidari.com/PLO.html and other websites. • Inspired by the Northern Lights, this paper introduces the Polar Lights Optimization (PLO) for solving optimization problems. • PLO's performance is validated using IEEE CEC2014 and CEC2022 benchmarks, with comparisons to widely recognized algorithms. • Qualitative analysis details PLO's characteristics for more relevant application to various optimization problems. • PLO's optimization capabilities are validated in medicine, including multi-threshold image segmentation and feature selection. [ABSTRACT FROM AUTHOR]

Studies on feature selection have been extensively conducted in the literature, as it plays a significant role in both supervised and unsupervised machine learning tasks. Since the bulk of features in high-dimensional data sets might not be significant, feature selection plays a key role in removing unimportant variables and improving prediction and data analysis performance. Many of the current feature selection methods, meanwhile, become ineffective when used on contemporary datasets, which exhibit an escalating number of features in relation to sample size. This paper introduces a novel supervised feature selection method for regression problems. The proposed algorithm is called maximal Clique with Interaction Screening (ISClique). The ISClique algorithm's overall structure can be described in two steps. Initially, a filter approach is used to select relevant features from an initial feature space and examine the different interactions between them. This is done using an innovative coefficient based on Kendall's tau and partial Kendall's tau. Secondly, the maximal clique strategy is applied as a wrapper to the selected set from the previous step to construct subsets of features. The most optimal subset that minimizes prediction error is selected. The proposed method integrates the advantages of graph theory with feature screening. Additionally, because the criteria employed in developing the ISClique method accommodate variable heterogeneity, this method is equally suitable for classification tasks. The proposed hybrid approach has been evaluated through applications involving various simulation scenarios and real datasets. Experimental findings demonstrate the advantages of ISClique over comparable methods. [ABSTRACT FROM AUTHOR]

Existing supervised Multi-Target Feature Selection (MTFS) methods seldom consider the nearest-neighbor relationship and statistical correlation of samples underlying the output space, which leads the result of feature selection to be easily interfered by the output noise, thus making it difficult to achieve satisfactory performance. This paper proposes a novel MTFS method to preserve both the global and local target correlations. Specifically, the low-rank constraint is introduced to achieve multi-layer regression structure to better decouple the inter-input and inter-target relationships. Moreover, the local nearest-neighbor relationships and variable correlations of the sample points in the output space are also explored through adaptive graph and manifold learning, to better utilize the target correlations to improve the MTFS performance. Following the above principle, the resulting objective function and the corresponding optimization algorithm are proposed. Extensive experiments on several public datasets show that the proposed method is superior to other state-of-the-art methods. • A new Embedded MTFS method is proposed to learn the feature redundancy and target correlations in MTR problems. • The nearest-neighbor relationships and target correlations are introduced to achieve superior performance in MTFS tasks. • Extensive experiments on several public datasets show that the proposed method is superior to the compared methods. [ABSTRACT FROM AUTHOR]

Air pollution prediction is a burning issue, as pollutants can harm human health. Traditional machine learning models usually aim to improve the overall prediction accuracy but neglect the accuracy for peak values. Moreover, these models are not interpretable. They fail to explain the interactions between various determining factors and their impacts on air pollution. In this paper, we propose a new Hybrid Interpretable Predictive Machine Learning model for the Particulate Matter 2.5 prediction, which carries two novelties. First, a hybrid model structure is constructed with deep neural network and Nonlinear Auto Regressive Moving Average with Exogenous Input model. Second, automatic feature generation and feature selection procedures are integrated into this hybrid model. The experimental results demonstrate the superiority of our model over other models in prediction accuracy for peak values and model interpretability. The proposed model reveals how PM2.5 prediction is estimated by historical PM2.5, weather, and season. The accuracies (measured by correlation coefficients) of 1, 3 and 6-hour-ahead prediction are 0.9870, 0.9332 and 0.8587, respectively. More importantly, the proposed approach presents a new interpretable machine learning framework for time series data, enabling to explain complex dependence of multimode inputs, and to build reliable predictive models. [ABSTRACT FROM AUTHOR]

In multi-label learning, feature selection is a topical issue for addressing high-dimension data. However, most of existing methods adopt imperfect labels to perform feature selection. Although some graph-based multi-label feature selection methods are proposed to deal with the problem, they adopt the fixed graph Laplacian matrix so that the performances of these models are under-performing. To this end, this paper proposes a Dynamic Subspace dual-graph regularized Multi-label Feature Selection method named DSMFS. DSMFS decomposes the original label space into a low-dimensional subspace, and then both the dynamic label-level subspace graph and the feature-level graph are used to obtain a high-quality label subspace to conduct feature selection process. Seven state-of-the-art methods are compared to the proposed method on twelve multi-label benchmark data sets in the experiments. Experimental results demonstrate the superiority of DSMFS. [ABSTRACT FROM AUTHOR]

The technology for Visual Odometry (VO) that estimates the position and orientation of the moving object through analyzing the image sequences captured by on-board cameras, has been well investigated with the rising interest in autonomous driving. This paper studies monocular VO from the perspective of Deep Learning (DL). Unlike most current learning-based methods, our approach, called DeepAVO, is established on the intuition that features contribute discriminately to different motion patterns. Specifically, we present a novel four-branch network to learn the rotation and translation by leveraging Convolutional Neural Networks (CNNs) to focus on different quadrants of optical flow input. To enhance the ability of feature selection, we further introduce an effective channel-spatial attention mechanism to force each branch to explicitly distill related information for specific Frame to Frame (F2F) motion estimation. Experiments on various datasets involving outdoor driving and indoor walking scenarios show that the proposed DeepAVO outperforms the state-of-the-art monocular methods by a large margin, demonstrating competitive performance to the stereo VO algorithm and verifying promising potential for generalization. [ABSTRACT FROM AUTHOR]

Fuzzy rough set is a theoretical framework of fuzzy uncertainty management, and discernibility matrix offers a mathematical foundation for algorithm construction of feature learning. The approaches of fuzzy rough set and discernibility matrix have been successfully applied in single-label learning. However, few works have been done on investigating the foundation of fuzzy rough discernibility matrix on multi-label data. There will be two pivotal problems to be addressed when using fuzzy rough discernibility matrix for multi-label data analysis. One is how to extract sample-level and label-level correlations; and the other is how to utilize the discernibility matrix for algorithm construction. For this reason, in this paper the fuzzy rough discrimination matrix is introduced to deal with the problem of multi-label feature selection. First, the significance of labels in the label space is captured based on the label correlation. Labels with different significances contribute to different weights for measuring the similarity between samples. Hence, a sample similarity matrix in the label space is computed based on the label weighting strategy. Then, a framework of a fuzzy decision system is formalized, in which the discernibility matrix of fuzzy rough sets is introduced as a foundation to evaluate the sample discrimination ability of features. Under the discernibility matrix criterion, a multi-label learning algorithm is developed to select discriminative features from multi-label data. A series of experimental analysis verifies the effectiveness and efficiency of the proposed method. [ABSTRACT FROM AUTHOR]

Traditional approaches that represent observations in vector or matrix notation easily lead to structure information losses and dependency destruction between elements. Tensors are the generalizations of matrices, providing a natural representation of rich structure in real-world multiway arrays. Although tensor decomposition has been verified and used in classification tasks, the heterogeneity gap among different orders is often neglected. In this paper, we exploited tensor decomposition framework and presented a heterogeneous tensor decomposition for robust classification (HTDRC) approach, which integrates nuclear and ℓ 2 , 1 -norm for intrinsic representation learning. Specifically, to obtain the lowest-rank intrinsic representations of tensorial data from an underlying low-dimensional subspace, HTDRC simultaneously learns a set of orthogonality constrained factor matrices and a low-rank constrained representation matrix. To better guide the decomposition process, a robust discriminant feature selection scheme is utilized by imposing ℓ 2 , 1 -norm penalty on the constructed classification loss and the regularized terms simultaneously. Experiments on four datasets demonstrated the superior performance of our proposed approach. [ABSTRACT FROM AUTHOR]

An instance is often represented from different aspects (views or modalities), which leads to high-dimensional features and even multiple labels. In this paper, we focus on the feature selection problem in multi-label classification, for which a trivial solution is handling the labels dividedly. Obviously, such a scheme may not work well by leaving the label relationship out of consideration. Recently, several research works conduct feature selection directly under a multi-label framework by implicitly or explicitly modeling label relationship. However, these works assume that all labels share the same feature subset or subspace, which is not reasonable enough for some scenarios since different labels tend to convey different semantics. To address this problem, we develop a novel approach in this paper to select label-dependent features for multi-label classification. Specifically, we (1) formulate a convex model based on a more general and practical assumption that different labels convey different semantics with specific features; (2) design an alternating optimization algorithm based on Nesterov's method and L 1 -ball projection for efficiently finding the optimal solution, which can realize multi-label classification, feature selection, and label relationship estimation simultaneously. Finally, experiments on publicly available datasets show that the proposed algorithm achieves better performance than several related methods. [ABSTRACT FROM AUTHOR]

The paper investigates application of several methods of feature selection to identification of the most important genes in autism disorder. The study is based on the expression microarray of genes. The applied methods analyze the importance of genes on the basis of different principles of selection. The most important step is to fuse the results of these selections into common set of genes, which are the best associated with autism. These genes may be treated as the biomarkers of this disorder and used in early prediction of autism. The paper proposes and compares three different methods of such fusion: purity of the clusterization space, application of genetic algorithm and random forest in the role of integrator. The numerical experiments are concerned with the identification of the most important biomarkers and their application in autism recognition. They show the applied fusion strategy of many independent selection methods leads to the significant improvement of the autism recognition rate. [ABSTRACT FROM AUTHOR]

Both subspace learning methods and feature selection methods are often used for removing irrelative features from high-dimensional data. Studies have shown that feature selection methods have interpretation ability and subspace learning methods output stable performance. This paper proposes a new unsupervised feature selection by integrating a subspace learning method (i.e., Locality Preserving Projection (LPP)) into a new feature selection method (i.e., a sparse feature-level self-representation method), aim at simultaneously receiving stable performance and interpretation ability. Different from traditional sample-level self-representation where each sample is represented by all samples and has been popularly used in machine learning and computer vision. In this paper, we propose to represent each feature by its relevant features to conduct feature selection via devising a feature-level self-representation loss function plus an ℓ 2 , 1 -norm regularization term. Then we add a graph regularization term (i.e., LPP) into the resulting feature selection model to simultaneously conduct feature selection and subspace learning. The rationale of the LPP regularization term is that LPP preserves the original distribution of data after removing irrelative features. Finally, we conducted experiments on UCI data sets and other real data sets and the experimental results showed that the proposed approach outperformed all comparison algorithms. [ABSTRACT FROM AUTHOR]

Sparse Bayesian learning (SBL) provides state-of-the-art performance in accuracy, sparsity and probabilistic prediction for classification. In SBL, the regularization priors are automatically determined that avoids an exhaustive hyperparameter selection by cross-validation. However, scalability to large problems is a drawback of SBL because of the inversion of a potentially enormous covariance matrix for updating the regularization priors in every iteration. This paper develops an approximate SBL algorithm called ARP-SBL, where the regularization priors are approximated without inverting the covariance matrix. Therefore, our approach can easily scale up to problems with large data size or feature dimension. It alleviates the long training time and high memory complexity in SBL. Based on ARP-SBL, two scalable nonlinear SBL models: scalable relevance vector machine (ARP-RVM) and scalable sparse Bayesian extreme learning machine (ARP-SBELM) are developed for problems of large data size and large feature size respectively. Experiments on a variety of benchmarks have shown that the proposed models are with competitive accuracy compared to existing methods while i) converging faster; ii) requiring thousands of times less memory; iii) without exhaustive regularized hyperparameter selection; and iv) easily scaling up to large data size and high dimensional features. [ABSTRACT FROM AUTHOR]

In recent years, deep learning techniques have significantly improved the performance of single image super-resolution (SISR). However, this improvement is often achieved at the cost of introducing a large amount of parameters, which limits the real-world applications for SISR. In this paper, we propose a lightweight SISR network called Multi-scale Channel Attention Network for Image Super-Resolution (MCSN). Our contributions are threefold. First of all, the multi-scale feature fusion block (MSFFB) can extract multi-scale features by filters with different receptive fields. Secondly, the channel shuffle attention mechanism (CSAM) encourages the flow of the information across feature channels and enhances the ability of feature selection. Thirdly, the global feature fusion connection (GFFC) can effectively improve feature utilization. Extensive experiments demonstrate that the parameter amount of our method is reduced by 3/4 compared with the current state-of-the-art MSRN method, while both the subjective visual effect and objective quality of the reconstructed high-resolution images are significantly better. [ABSTRACT FROM AUTHOR]