Search

This paper introduces a novel feature selection method, called Feature Selection based on Importance Measures (FS-IM), to enhance the forecasting of crude oil returns. FS-IM innovatively combines active learning with the application of Gaussian noise to input features and selects the most relevant features using an optimal threshold value. The paper applies a ridge regression (RR) model based on FS-IM (FS-RR) to identify the factors that have important information for crude oil return forecasting. The paper compares FS-IM with other dimension reduction methods such as Principal Component Analysis (PCA), Kernel Principal Component Analysis (KPCA), and Independent Component Analysis (ICA). The results show that FS-IM can significantly improve model accuracy, demonstrating its effectiveness in finding key features. Moreover, FS-IM is more stable and consistent than other dimension reduction methods in enhancing the prediction accuracy in different scenarios, indicating its superior capability in capturing complex relationships between input and output variables. Furthermore, this study compares FS-RR model with other 13 prediction models by conducting experiments using a series of evaluation metrics, different statistical tests, and different step-ahead predictions and training sets. The results confirm that the RR model based on FS-IM can consistently outperform other model in terms of predictive performance and economic value, proving its effectiveness and robustness. This study contributes to the literature on crude oil price forecasting by addressing the challenges of high-dimensional and complex data, and by providing a robust, practical tool for professionals in energy economics and finance. • A new feature selection method for crude oil forecasting using importance scores and Gaussian noise is proposed. • The method selects the most relevant features based on the prediction errors of a base forecasting model. • The method enhances the accuracy and value of ridge regression model compared to other methods. • The method handles high-dimensional and complex data without losing information or interpretability. [ABSTRACT FROM AUTHOR]

Many traditional unsupervised feature selection algorithms utilize manifold information to mine the local structure of the data. However, the noise existing in the raw data reduces the accuracy of the manifold information of data, which affects the learning effect of the entire algorithm. In order to solve the above problems and more fully find the internal structure inside the data, this paper proposes an adaptive graph regularization and self-expression for noise-aware feature selection (ASNFS). Firstly, the algorithm adopts non-negative matrix factorization to decompose the raw data matrix, and utilizes the low-dimensional matrix generated after decomposition to replace the raw high-dimensional data matrix. This allows the algorithm to reveal some internal structural information of the raw data while reducing the dimensionality of the data. ASNFS also introduces the orthogonal basis clustering with excellent clustering effect, and the interpretability of the algorithm is enhanced. Secondly, in addition to preserving the manifold information in the low-dimensional projection subspace, the algorithm also preserves the manifold information in the non-negative matrix factorization subspace. Meanwhile, the adaptive graph regularization term added to the objective function enables the algorithm to continuously update the similarity matrix. It can effectively remove the noise inside the raw data, and prevent the occurrence of over-fitting phenomenon of the experimental results caused by the fixed similarity matrix. Finally, the similarity matrix retains the local structure information of the data with each iteration, and the results of the feature selection are reused for the construction of the similarity matrix. ASNFS adopts an alternate iterative method to optimize the objective function, which is simple and effective. Then, the algorithm complexity and convergence are analyzed. ASNFS is compared with seven feature selection algorithms on nine datasets, and the experimental results reflect the effectiveness of ASNFS in feature selection. [ABSTRACT FROM AUTHOR]

Battery degradation, caused by multiple coupled degradation mechanisms, severely affects the safety and sustainability of a battery management system (BMS). The battery state of health (SOH) is a commonly-adopted metric to evaluate a battery's degradation condition, which should be carefully modeled to facilitate the safety and reliability of a BMS. Recently, owing to the rapid progress of data science-related techniques, data-driven models for battery SOH estimation have attracted great attentions from both academia and industry communities. This paper aims to provide the scientists and engineers with a general overview of data-driven battery SOH estimation technology for BMSs. State-of-the-art models published during 2018–2022 are reviewed with care, including a) feature extraction and selection methods; b) benchmarks, variants and extensions of data-driven SOH estimation models; and c) publicly-available battery SOH datasets. Afterwards, experiments are conducted and analyzed on the Toyota & Stanford-MIT battery SOH datasets for benchmark study. Finally, existing challenges and feature trends are summarized. [ABSTRACT FROM AUTHOR]

• A methodology to forecast time series with generalized neural networks is outlined. • A clever way of feature selection is proposed. • Strategies for dealing with time series with a trend are developed. This paper discusses how to forecast time series using generalized regression neural networks. The main goal is to take advantage of their inherent properties to generate fast, highly accurate forecasts. To this end, the key modeling decisions involved in forecasting with generalized regression neural networks are described. To deal with every modeling decision, several strategies are proposed. Each strategy is analyzed in terms of forecast accuracy and computational time. Apart from the modeling decisions, any successful time series forecasting methodology has to be able to capture the seasonal and trend patterns found in a time series. In this regard, some clever techniques to cope with these patterns are also suggested. The proposed methodology is able to forecast time series in an automatic way. Additionally, the paper introduces a publicly available R package that incorporates the best presented modeling approaches and transformations to forecast time series with generalized regression neural networks. [ABSTRACT FROM AUTHOR]

• This paper proposes a new feature selection method called SLMEA. • The sparse transformation and pseudo label matrix are combined to optimize. • The pseudo label matrix is used to guide the learning of low-dimensional space. • The similarity matrix is constructed adaptively. • The constraint of ℓ 2 , 1 / 2 -norm is applied to the matrix. • The SLMEA can have higher clustering accuracy and normalized mutual information. Traditional feature selection algorithms usually explore the relationship between data and cluster structure in a single space, so the internal relationship obtained is not very rich, and it is not enough to select more valuable features. To solve the above problems, in order to fully mine the intrinsic correlation information in different spaces, so that the relationship between data, features and clustering structure can be explored at the same time, this paper proposes a feature selection method based on sparse and low-dimensional representation with maximum entropy adaptive graph (SLMEA). Firstly, the SLMEA combines the sparse transform representation with pseudo-label matrix learning to optimize, and uses the pseudo-label matrix to guide the learning of sparse low-dimensional space. It can not only explore the relationship between data and pseudo-labels in data space, but also mine the association between features and pseudo-labels in feature space, so as to select more discriminative features. Secondly, based on the maximum entropy theory, the similarity matrix is constructed adaptively, so that the two different manifold structures corresponding to sparse transform representation and pseudo label matrix learning can be adaptively learned and retained in the iterative process. In addition, in order to ensure the sparsity of the transformation matrix, the constraint of ℓ 2 , 1 / 2 -norm is applied to the matrix, which can better deal with the redundant features and obtain more sparse solutions. Finally, the SLMEA uses an alternate iterative update method to optimize the objective function, and carries out extensive experiments on eight mainstream datasets. Compared with seven state-of-the-art algorithms, SLMEA can have higher clustering accuracy and normalized mutual information. [ABSTRACT FROM AUTHOR]

ℓ 1 regularization has been used for logistic regression to circumvent the overfitting and use the estimated sparse coefficient for feature selection. However, the challenge of such regularization is that the ℓ 1 regularization is not differentiable, making the standard convex optimization algorithm not applicable to this problem. This paper presents a simple projection neural network for ℓ 1 -regularized logistics regression. In contrast to many available solvers in the literature, the proposed neural network does not require any extra auxiliary variable nor smooth approximation, and its complexity is almost identical to that of the gradient descent for logistic regression without ℓ 1 regularization, thanks to the projection operator. We also investigate the convergence of the proposed neural network by using the Lyapunov theory and show that it converges to a solution of the problem with any arbitrary initial value. The proposed neural solution significantly outperforms state-of-the-art methods concerning the execution time and is competitive in terms of accuracy and AUROC. [ABSTRACT FROM AUTHOR]

• We extend the NRS model that can be skillfully applied to figure out label distribution data and avoid the instability of results caused by different neighborhood granularity selection. • Based on the nearest neighbor information distribution, a label enhancement method is designed to transform logical labels into label distribution, thereby obtaining more potential label information to guide feature selection. • A novel forward algorithm of multi-label feature selection based on the extended neighborhood rough set is proposed, which leverages feature significance, label significance, and label-specific features, simultaneously. Multi-label feature selection is an indispensable technology in multi-semantic high-dimensional data preprocessing, which has been brought into focus in recent years. However, most existing methods explicitly assume that the significance of all relevant labels is the same for every instance, while ignoring the real scenarios that the significance of available labels to each instance is usually different. In this paper, we propose a novel multi-label feature selection based on label distribution and neighborhood rough set, known as LDRS. To be specific, we first construct a label enhancement method based on instance information distribution to convert the logical labels of multi-label data into label distribution, thereby capturing label significance to provide additional information for learning tasks. Then, we extend the neighborhood rough set model for label distribution learning, and discuss the related properties in detail. This extended model can effectively avoid the selection of neighborhood granularity and seamlessly apply to handle label distribution data. After that, two feature significance measures are established to realize the quality evaluation of features and the fusion of label-specific features. Finally, a novel feature selection framework is designed, which takes into account feature significance, label significance, and label-specific features, simultaneously. Experiments on both public and real-world datasets exhibit the advantages of the proposed method. [ABSTRACT FROM AUTHOR]

As a powerful paradigm, deep learning (DL) models have been used in many applications for classification tasks in images, text, and audio. Through DL models, we can learn task-driven features from big data. However, DL models are fully deterministic and cannot handle uncertain and imprecise data. DL models are often sensitive to noise in data and do not operate well in areas where data are vague. Moreover, when there is a large feature set or high-dimensional data with irrelevant and redundant features, the DL models' performance decreases in classification due to training with irrelevant features. To gain reliable results in such high-dimensional problems, DL models require a large amount of data which usually grows exponentially concerning the number of features. Data uncertainty problems and irrelevant features in DL models cause low performance in classification tasks. This paper proposes an optimized fuzzy deep learning (OFDL) model for data classification based on Non-Dominated Sorting Genetic Algorithm II (NSGA-II). OFDL utilizes optimization in the composition of DL and fuzzy learning via the NSGA-II in multi-modal learning. To achieve effective classification, OFDL first considers intelligent feature selection by finding the best trade-offs between two conflicting objective functions, minimizing the number of features, and maximizing the accuracy (maximizing weights of selected features). Next, to reach optimized backpropagation and fuzzy membership functions, OFDL utilizes Pareto optimal solutions for multi-objective optimization using NSGA-II based on their objective functions. Furthermore, the fusion layer in OFDL fused optimized views of DL and fuzzy learning that provides a high-level representation of inputs and optimum features for classification tasks where the data contain high uncertainties and noises. This functionality gives valuable attributes during classification since identifying and selecting appropriate features ensures prompt and correct class. Also, it provides deep insight into tackling the effect of ambiguous data and each feature's uncertainty in the classification tasks. The examination of OFDL reveals good performance in terms of F-measure, accuracy, recall, precision, and True Positive Rate (TPR) compared to fuzzy classifiers. Furthermore, OFDL has higher accuracy in classification tasks than earlier fuzzy DNN models. [ABSTRACT FROM AUTHOR]

Crawler detection is always an important research topic in network security. With the development of web technology, crawlers are constantly updating and changing, and their types are becoming diverse. The diversity and dynamics of crawlers pose significant challenges for feature applicability and model robustness. Existing crawler detection methods can only detect a limited number of crawlers by predefined rules and can not cover all types of crawlers; worse, they can be completely invalidated by the emergence of new types of crawlers. In this paper, we propose a reinforcement learning based web crawler detection method for diversity and dynamics (WC3D), which is composed of a feature selector and a session classifier. The feature selector selects the appropriate feature set for different types of crawlers with deep deterministic policy gradient. The session classifier makes crawler detection and provides rewards to the feature selector. The two modules are trained jointly to optimize the feature selection and session classification processes. Extensive experiments demonstrate the existence of crawler diversity and that the proposed method is still highly robust against the new type of crawlers and achieves state-of-the-art performance even without considering the dynamics of the crawlers. [ABSTRACT FROM AUTHOR]

In numerous domains, there is ample evidence indicating that a significant portion of real-world data exhibits temporal variations, such as medical research and meteorological studies. Particularly in the era of big data, not only does the size of data change dynamically but also its dimensions at an unprecedented pace. Consequently, employing traditional methods to handle such dynamic data becomes highly impractical. To address this limitation, this paper proposes an incremental feature selection algorithm tailored for dynamic feature variations. For scenarios involving an increase in features, the novel algorithm efficiently identifies a target subset with effective features. In order to showcase the efficacy of the proposed algorithm, this paper conduct experiments using four commonly used classifiers and five UCI data sets. The experimental results further validate both the feasibility and efficiency of the new approach. [ABSTRACT FROM AUTHOR]

• An analysis of lexicographic optimization for feature selection and data classification. • Cooperative co-evolution for feature selection while the classifier hyperparameters are also being optimized. • Optimization of multiple objectives using a simple evolutionary algorithm. • A new lexicographic ranking methodology to compare the results of different feature selection methods. • Quite good results for real high-dimensional datasets. This paper starts with two hypotheses. The first one is that the simultaneous optimization of the hyperparameters regulating the classifier within a wrapper method, while the best subset of features is being determined, should improve the results with respect to those obtained with a pre-parameterized classifier. The second one is that solving these two problems can be formulated as a lexicographic optimization problem, allowing the use of a simple single-objective evolutionary algorithm to solve this multi-objective problem. The fitness function is of key importance for such wrapper methods. It is responsible for guiding the search towards potentially good solutions and it also consumes most of the runtime. Having these issues in mind, this paper also proposes a new lexicographic fitness function, designed to minimize the runtime of the algorithm and also to avoid over-fitting. Furthermore, the execution time and the quality of the results obtained by the wrapper procedure also depend on some algorithmic hyperparameters: the similarity thresholds used when comparing two different solutions lexicographically and the percentage of data samples used for validation during the training process. Thus, an experimental analysis has been carried out to find adequate values for these hyperparameters. Finally, the lexicographic cooperative co-evolutionary wrapper approach, using the new fitness function proposed in this paper, has been tested with several datasets belonging to the University of California, Irvine (UCI) repository and also with some real high-dimensional datasets, obtaining quite good results, compared to other state-of-the-art wrapper methods. The comparison has also been made lexicographically, with a new methodology proposed in this paper. [ABSTRACT FROM AUTHOR]

Mutual information (MI) is a widely used similarity measure in pattern recognition. MI uses entropy as a measure of uncertainty to quantify the structural similarity of two vectors. Replacing entropy with variance as a measure of uncertainty, an analogous class of similarity measures can be derived and estimated by regression techniques. Recently, the non-linear piecewise constant regression (PWCR) has been proposed to drive similarity measures of this scheme, leading to competitive alternatives of MI. Although PWCR is based on binning, the optimal binning technique for certain problems remained an open question. In this paper, we show mathematically that the optimal binning needs to be aligned with the expected relationship between the vectors being compared. In general, approximately optimal binnings can be found by combinatorial optimization, and in certain cases the optimal binning can be determined by k-means clustering. The theoretical findings are supported by numerical experiments that show a 2–5% increase in the AUC scores in simulated pattern recognition scenarios and improved feature rankings in feature selection problems. The results suggest that the proposed binning techniques could improve the performance of PWCR-driven similarity measures in real-world applications. [ABSTRACT FROM AUTHOR]

• A novel Depth Guided Feature Selection Network (DGFSnet) for RGBD salient object detection is proposed. • The DGFSnet takes depth information as prior and dynamically selects the complementary RGB information for the RGBD salient object detection. • Two novel modules including Depth Weight Generation module (DWG) and Weight-guided Feature Aggregation module (WFA) are proposed for implementing DGFSnet. • Together with DWG and WFA, the DGFSnet is able to adaptively enhance and integrate complementary RGB features for effectively highlighting the salient regions. • Comprehensive experiments over five popular RGBD benchmarks illustrating that DGFSnet consistently outperforms the state-of-the-art methods on all datasets. Depth information can greatly benefit the saliency detection in RGBD images if they are utilized well. Prevalent methods generally directly fuse depth and RGB features in networks. However, due to the inherent inconsistent between RGB and depth information, the RGB features are easy to be interfered by the intrinsic noise existed in depth features, making the precise RGBD saliency detection still a challenge. In this paper, we propose a novel Depth Guided Feature Selection network (DGFSnet) that takes depth information as prior and dynamically selects the complementary RGB information for RGBD salient object detection. Specifically, DGFSnet first includes a Depth Weight Generation module (DWG) to learn a set of layer-specific weights from multi-scale depth features. Guiding by these learned weights, DGFSnet further devises a Weight-guided Feature Aggregation module (WFA) to assign them to their corresponding RGB layers for dynamically enhancing and selecting saliency-related RGB features. With two modules, DGFSnet is able to effectively integrate the multi-modality complementaries and further highlight salient regions. Experimental results over seven popular RGBD salient object detection benchmarks demonstrate that DGFSnet fairly locates salient regions and effectively segments the complete object. [ABSTRACT FROM AUTHOR]

Feature selection (FS) may improve the performance, cost-efficiency, and understandability of supervised machine learning models. In this paper, FS for the recently introduced distance-based supervised machine learning model is considered for regression problems. The study is contextualized by first providing an umbrella review (review of reviews) of recent development in the research field. We then propose a saliency-based one-shot wrapper algorithm for FS, which is called MAS-FS. The algorithm is compared with a set of other popular FS algorithms, using a versatile set of simulated and benchmark datasets. Finally, experimental results underline the usefulness of FS for regression, confirming the utility of certain filter algorithms and particularly the proposed wrapper algorithm. [ABSTRACT FROM AUTHOR]

• A new model agnostic feature selection method for supervised learning. • Sensitivity analysis with the predictive distribution of a scalable Gaussian process. • New L 2 divergence derivation to better measure classification results and changes. • Better performance on several learning models than existing methods. Feature selection is one of the most important issues in supervised learning and there are a lot of different feature selection approaches in the literature. Among them one recent approach is to use Gaussian process (GP) because it can capture well the hidden relevance between the features of the input and the output. However, the existing feature selection approaches with GP suffer from the scalability problem due to high computational cost of inference with GP. Moreover, they use the Kullback–Leibler (KL) divergence in the sensitivity analysis for feature selection, but we show in this paper that the KL divergence underestimates the relevance of important features in some cases of classification. To remedy such drawbacks of the existing GP based approaches, we propose a new feature selection method with scalable variational Gaussian process (SVGP) and L 2 divergence. With the help of SVGP the proposed method exploits given large data sets well for feature selection through so-called inducing points while avoiding the scalability problem. Moreover, we provide theoretical analysis to motivate the choice of L 2 divergence for feature selection in both classification and regression. To validate the performance of the proposed method, we compare it with other existing methods through experiments with synthetic and real data sets. [ABSTRACT FROM AUTHOR]

Most of the existing unsupervised feature selection methods learn the cluster structure through spectral clustering, and then use various regression models to introduce the data matrix into the indicator matrix to obtain feature selection matrix. In these methods, the clustering indicator matrix is usually continuous value, which is not the best choice for the matrix in terms of its supervising role in feature selection. Based on this, unsupervised feature selection via discrete spectral clustering and feature weights (FSDSC) is proposed in this paper. First, FSDSC integrates regression model and spectral clustering in a unified framework for feature selection, and introduces a feature weight matrix, which intuitively expresses the importance of each feature with its diagonal elements. Compared with the common feature selection matrix that requires constraints such as sparse regular items, the appearance of the feature weight matrix reduces the complexity of the model and simplifies the calculation process of feature evaluation. Secondly, for the value of the indicators matrix, the spectral clustering is improved to obtain a discrete clustering indicator matrix, which provides clearer guidance information for feature selection. Finally, in order to avoid trivial solutions, the transformation matrix is constrained by orthogonal constraint. The combination of the orthogonal regression model and spectral clustering enables the algorithm to perform feature selection and manifold information learning at the same time, thereby preserving the local geometric structure of data. Compared with other excellent unsupervised feature selection algorithms, the experimental results prove the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

• Unsupervised feature selection based on the distance of the selected features and an orthogonal set. • Feature selection is performed through the matrix factorization of the generated orthogonal set. • A dual-correlation model for considering both the correlation among the features and samples. Feature selection has been an outstanding strategy in eliminating redundant and inefficient features in high-dimensional data. This paper introduces a novel unsupervised feature selection based on the matrix factorization, namely Unsupervised Feature Selection Guided by Orthogonal Representation (UFGOR). The orthogonality between a pair of variables refers to a specific case of linear independence such that they are perfectly uncorrelated. Motivated by the benefits of the orthogonality concept, the proposed UFGOR method is established based on the distance between the selected feature set and an orthogonal set corresponding to the whole feature space. Moreover, this orthogonal set is generated via QR-matrix factorization over the whole features and is employed as the compact representation of data matrix. In the next step, an unsupervised feature selection method is performed through the matrix factorization of the generated orthogonal set. Additionally, a dual-correlation model is utilized in the objective function of UFGOR to simultaneously consider both the local correlation in a set of selected features and the global correlation among the samples of a data. A detailed convergence analysis in line with an effective iterative algorithm proposed for the UFGOR method is also given. Numerical experiments on several real-world datasets illustrate the superior efficiency of our approach in comparison with some state-of-the-art unsupervised feature selection methods. [ABSTRACT FROM AUTHOR]

Feature selection has been recognized for long as an important preprocessing technique to reduce dimensionality and improve the performance of regression and classification tasks. The class of sequential forward feature selection methods based on Mutual Information (MI) is widely used in practice, mainly due to its computational efficiency and independence from the specific classifier. A recent work introduced a theoretical framework for this class of methods which explains the existing proposals as approximations to an optimal target objective function. Such framework made clear the advantages and drawbacks of each proposal. Methods accounting for the redundancy of candidate features using a maximization function and considering the so-called complementary effect are among the best ones. However, they still penalize the complementarity, which is an important drawback. This paper proposes the Decomposed Mutual Information Maximization (DMIM) method, which keeps the good theoretical properties of the best methods proposed so far but overcomes the complementarity penalization by applying the maximization separately to the inter-feature and class-relevant redundancies. DMIM was extensively evaluated and compared with other methods, both theoretically and using two synthetic scenarios and 20 publicly available real datasets applied to specific classifiers. Our results show that DMIM achieves a better classification performance than the remaining forward feature selection methods based on MI. [ABSTRACT FROM AUTHOR]

Unsupervised feature selection, one of the important dimensionality reduction methods for high-dimensional data, has been paid more and more attention by researchers due to its effectiveness of processing the data without labels. However, most unsupervised feature selection methods do not take into account the fact that real data tends to be mixed with noise or outliers, or only consider the sensitivity of the model to them. In addition, the local manifold structure of data is usually preserved by a simple graph based on fixed k nearest neighbors. On the one hand, the different sample distributions of different data are considered without distinction. On the other hand, simple graph can not well capture the inherent relations between samples when facing multi-modal data. In view of the above problems, a robust unsupervised feature selection algorithm, sparse and minimum-redundant subspace learning with dual regularization (SMRSDR), is proposed in this paper. Specifically, a self-paced learning regularization based on soft weight allocation is introduced into SMRSDR to strictly control the samples' entry into model learning. Normal samples are allowed to enter model learning in order of their importance, while noise or outliers are strictly blocked out of model learning. Besides, considering the difference in sample distribution and the multimodal nature of data, the traditional simple graph Laplacian regularization is replaced by the hypergraph Laplacian regularization based on adaptive nearest neighbors selection. Furthermore, the sparsity and minimum-redundancy constraints are imposed on the subspace learning framework so that the most representative and minimum-redundant features can be selected. Finally, the objective function of SMRSDR is solved by an alternating iterative optimization algorithm. And a series of experiments are performed to comprehensively prove the effectiveness and superiority of SMRSDR. [ABSTRACT FROM AUTHOR]

To improve the classification performance of existing adaptive boosting (AdaBoost) based algorithms effectively, an improved AdaBoost algorithm based on misclassified samples oriented feature selection and weighted non-negative matrix factorization (WNMF) is proposed in this paper. Firstly, in order to consider the effects of sample weights, a misclassified samples oriented feature selection (called MOFS) is proposed to select the most discriminative features which occur in the samples with high weights. Secondly, the explicit features and the part-based features of the training samples are both considered, and the WNMF algorithm is introduced and combined with MOFS to reduce the dimension of the training sample set. Finally, the concept of misclassification degree is introduced and a fine grained sample weight updating method is proposed to distinguish the samples with different misclassification degrees. Numerical experiments show that the proposed MOFS method achieves higher accuracy compared to traditional feature selection methods, and the proposed MOFS and WNMF based AdaBoost method obtains significant improvement on classification accuracy when comparing with typical existing AdaBoost based algorithms using different classifiers. [ABSTRACT FROM AUTHOR]

• An advanced shuffled frog leaping algorithm (DSSRLFLA) is developed for model evaluation and feature selection. • A dynamic step size adjustment strategy based on historical information is designed to balance exploration and exploitation. • Specular reflection learning mechanism increases the possibility of searching for valid solutions and enhances the search ability. • An improved simulated annealing strategy is executed for each memetic to improve the efficiency of local exploitation. • DSSRLFLA can obtain higher quality optimal solutions compared to some competitive algorithms. The shuffled frog leaping algorithm is a new optimization algorithm proposed to solve the combinatorial optimization problem, which effectively combines the memetic algorithm based on a memetic algorithm and the particle swarm algorithm based on population behavior. The algorithm is widely used because it is easy to implement and requires few parameters to be adjusted. However, there are still some characteristics of this method that need to be improved because it is easy to fall into local optimization or poor search ability. To alleviate this limitation, a new version of the improved SFLA is proposed in this paper, which incorporates a dynamic step size adjustment strategy based on historical information, a specular reflection learning mechanism, and a simulated annealing mechanism based on chaotic mapping and levy flight. Firstly, the dynamic step size adjustment strategy based on historical information effectively helps to balance local exploration and global exploitation and alleviates the problem of falling into local optimum. Second, the specular reflection learning mechanism increases the possibility of searching for valid solutions in feasible domains and enhances the search ability of individuals in the population. Finally, an improved simulated annealing strategy is executed for each memetic, which improves the efficiency of local exploitation. In order to test the performance of the proposed algorithm, 31 test functions were selected from IEEE CEC2014 and 23 essential benchmark functions, and comparative experiments were carried out from the two dimensions of 30 and 100. Furthermore, a series of competing algorithms are selected, which involve nine classical standard algorithms, including PSO, BA, SSA, FA, SCA, WOA, GWO, MFO, and SFLA, as well as, six well-known improved algorithms, including LSFLA, DDSFLA, GOTLBO, ALCPSO, BLPSO, CLPSO. Furthermore, Wilcoxon signed-rank test and Friedman test are used as testing tools to illustrate the scalability of the proposed algorithm. From the analysis of the results, it can be seen that this proposed method has been effectively improved concerning stability and the quality of the optimal solution obtained from the search, both in low and high dimensions, and the ability to jump out of the local optimum has also been enhanced. In addition, to prove that this method has a reliable performance in discrete problems and continuous problems, DSSRLFLA is mapped into a discrete space, and 24 UCI data sets are also selected to evaluate the performance of the new feature selection method. The experimental results illustrate that this improved method can obtain fewer features and higher classification accuracy than some well-known feature selection methods. [ABSTRACT FROM AUTHOR]

Salient object detection is one of the basic challenging problems in the area of computer vision. Traditional saliency models usually utilize handcrafted features and various prior cues to locate and segment salient objects from complicated surroundings. Recently, with the development of convolutional neural networks (CNNs), most of existing approaches explore the combination of multi-level feature maps for improving detection performance. However, how to learn powerful feature is still a challenge, and the multi-level and multi-scale feature maps are usually aggregated without distinction. In this paper, we propose a novel salient object detection approach named residual attentive feature learning (RAFL) to uniformly highlight salient objects and effectively suppress background noises. In our proposed RAFL, a global perception (GP) module is designed to obtain rich global features of input image. A pyramid feature extraction (PFE) module is utilized to capture the rich context of side-output information. Moreover, an attentive feature selection (AFS) module is applied to refine these side-output features and generate attentive features, so as to find out the important feature channels having high responses to the salient objects. Finally, a residual feature learning (RFL) module is proposed to make the shallow layers generate more accurate saliency maps with the help of the deep layers which contain more global semantic information. Experimental results show that our proposed approach performs favorably against 18 state-of-the-art representative methods on five benchmark datasets. [ABSTRACT FROM AUTHOR]

The complexity of data has been increasing not only in the dimension of the data, but also in the interrelated class relationships. Pre-defined hierarchical structure of classes helps to model such relationships in a way, which divides the whole classification problem into a set of subtasks. Feature selection, known as a data preprocessing technique, aims to selecting useful feature to improve the performance of learning algorithms. However, the issue of close class relationships, especially under the same parent node in the hierarchical structure, is adverse to the feature selection. In this paper, we introduce a novel distance metric based feature selection method with large margin nearest neighbor strategy for the hierarchical classification problems. Different from other algorithms, our proposed method is to project the data into a low-dimensional feature space for learning correct metrics, and the weights of features will be indicated in the transformation matrix. A sparsity regularization and iterative optimization ensure that superior feature subsets can be selected. Experimental evaluation on several datasets with hierarchical class structure demonstrates that the proposed approach is comparable or better than some other state-of-the-art hierarchical feature selection methods. [ABSTRACT FROM AUTHOR]

Existing supervised Multi-Target Feature Selection (MTFS) methods seldom consider the nearest-neighbor relationship and statistical correlation of samples underlying the output space, which leads the result of feature selection to be easily interfered by the output noise, thus making it difficult to achieve satisfactory performance. This paper proposes a novel MTFS method to preserve both the global and local target correlations. Specifically, the low-rank constraint is introduced to achieve multi-layer regression structure to better decouple the inter-input and inter-target relationships. Moreover, the local nearest-neighbor relationships and variable correlations of the sample points in the output space are also explored through adaptive graph and manifold learning, to better utilize the target correlations to improve the MTFS performance. Following the above principle, the resulting objective function and the corresponding optimization algorithm are proposed. Extensive experiments on several public datasets show that the proposed method is superior to other state-of-the-art methods. • A new Embedded MTFS method is proposed to learn the feature redundancy and target correlations in MTR problems. • The nearest-neighbor relationships and target correlations are introduced to achieve superior performance in MTFS tasks. • Extensive experiments on several public datasets show that the proposed method is superior to the compared methods. [ABSTRACT FROM AUTHOR]

Intrusion detection technology can actively detect abnormal behaviors in the network and is important to the security of Industrial Internet of Things (IIOT). However, there are some issues with the current intrusion detection technology for IIOT, such as extreme imbalance in the number of samples of different classes in the dataset, redundant and meaningless features in the samples, and the inability of traditional intrusion detection methods to meet the requirements of detection accuracy in the increasingly complex IIOT. In view of the extreme imbalance of classes, this paper applies the hierarchical clustering algorithm to the under-sampling technology, which reduces the number of majority samples while reducing the loss of information of majority samples, and solves the problem of missing detection and false detection of minority samples caused by sample imbalance. In order to avoid feature redundancy and interference, this paper proposes an optimal feature selection algorithm based on greedy thought. This algorithm can obtain the optimal feature subset of each type of data in the data set, thus eliminating redundant and interfering features. Aiming at the problem of insufficient detection ability of traditional detection methods, this paper proposes a deep neural network intrusion detection model based on the parallel connection of global and local subnetworks. This model obtains the overall benchmark detection of the dataset through the deep neural network, and then strengthens the detection effect of each subclass through the parallel connection of subnetworks, greatly improving the performance of the intrusion detection algorithm. The experimental results show that the method described in this paper can improve the intrusion detection for IIOT. [ABSTRACT FROM AUTHOR]

Intelligent fault diagnosis is a rapidly evolving field within power engineering. Using gas-in-oil data is a reliable method for transformer fault diagnosis that has been widely adopted in the power industry. However, traditional machine learning methods often suffer from low diagnostic accuracy due to the lack of a clear and effective feature set for gas-in-oil data as well as an imbalance between classes of sample size. To overcome this challenge, this paper proposes a novel transformer fault diagnosis model that utilizes a Filter-Wrapper Combined Feature Selection method and an AdaBoost integrated weighted broad learning system (AdaBoost-WBLS). More specifically, the original data is expanded to extract meaningful features, and the Filter-Wrapper combined feature selection method is used to eliminate preliminary redundancy, relevance, and significance of current features. The Wrapper algorithm is then used for precise screening to obtain the optimal feature subset, which effectively improves the quality of transformer features. Furthermore, to address the issue of imbalanced transformer samples, an improved BLS and AdaBoost integration method is introduced, and a fault diagnosis model based on AdaBoost-WBLS is proposed. Compared with existing power transformer fault diagnosis methods, the proposed method has a more accurate and balanced effect on fault classification. Overall, this paper provides a comprehensive and effective approach to transformer fault diagnosis, which has important implications for the reliability and safety of power systems. [ABSTRACT FROM AUTHOR]

To describe objects more comprehensively and accurately, multi-view learning has attracted considerable attention. Recently, graph embedding based multi-view feature selection methods have been proposed and shown efficient in many real applications. The existing methods generally construct one common graph matrix to exploit the local structure of multi-view data via the linear weight fusion or learning one common graph matrix across all views. However, since all views share the identical graph structure, this emphasizes the consistency too much, resulting in restricting the diversity among different views. In this paper, a tensor low-rank constrained graph embedding method is proposed for multi-view unsupervised feature selection. To embody the view-specific information of each view, our model constructs the graph structure for each corresponding view, respectively. To capture the consistency across views, a tensor low-rank regularization constraint is imposed on the tensor data formed by these graph matrices. An efficient optimization algorithm with theoretical convergence guarantee is designed to solve the proposed method. Extensive experimental results validate that the proposed method outperforms some state-of-the-art methods. The code of our model can be found at https://www.researchgate.net/publication/353902948_demoTLR. [ABSTRACT FROM AUTHOR]

In hierarchical classification learning, the hierarchical feature selection algorithm plays an important role in overcoming the curse of dimensionality. Existing hierarchical feature selection algorithms, based on the granular computing framework, all use three basic search strategies to search for similar and dissimilar classes. These strategies compute the importance of features to the global label for feature selection. However, existing methods based on the sibling strategy can only stay at the fine-grained level for feature selection, often without considering that the fine-grained level is also continuously separated from the coarse-grained level. Thus, these methods do not take into account the features hidden below the coarse granularity, resulting in the selection of a top-heavy subset of features and the loss of many important features. Therefore, this paper proposes a Hierarchical Feature Selection Based on Neighborhood Interclass Spacing From Fine to Coarse (HFSNIS) algorithm, which aims to change the feature selection to the coarse-grained hierarchy. The framework of the HFSNIS algorithm is as follows: First, each fine-grained leaf node is coarsened to the coarsest hierarchy of granularity from fine to coarse, where the non-root ancestor node is located. Next, the search for similar and dissimilar nearest neighbors is performed at the coarsest granularity hierarchy. Finally, the features are filtered using the Neighborhood Interclass Spacing model to obtain a subset of features. Therefore, this HFSNIS algorithm based on the Coarsest Search Strategy (CSS) can reselect features that were previously ignored in the fine-grained hierarchy, resulting in a better feature subset. Finally, the proposed algorithm outperforms seven state-of-the-art feature selection algorithms on six datasets. [ABSTRACT FROM AUTHOR]

In recent years, Unsupervised Feature Selection (UFS) methods have attracted considerable interest in different research areas due to their wide application in problems where unlabeled data appears. Nevertheless, few UFS methods can process datasets described by both numerical and non-numerical features (mixed data). This paper introduces a new filter UFS method based on a new feature correlation measure for mixed data to select a relevant and non-redundant feature subset. The proposed method addresses the feature selection problem in two stages through a strategy that combines Spectral Feature Selection to identify relevant features and a Pair-wise Redundancy Analysis to remove those features with a high correlation with others. According to our experiments, the proposed feature selection method based on the introduced feature correlation measure provides better quality results than previous filter UFS methods for mixed data reported in the literature. • A filter Unsupervised Feature Selection method for mixed data is presented. • A new feature correlation measure for mixed data is introduced. • Spectral Feature Selection and Pair-wise Redundancy Analysis were applied. • A comprehensive evaluation of the proposed method is provided. [ABSTRACT FROM AUTHOR]

The number of fuzzy rules in a fuzzy neural network usually depends on the grid-type partition or clustering-based partition. However, the number of grids or clusters needs to be set in advance and the initial number of fuzzy sets on each dimension is usually the same. This usually does not fit the real situation (for example, when forecasting the weather, various features such as temperature, humidity, wind direction, wind speed, etc., may have different numbers of fuzzy sets). In this paper, a novel zero-order fuzzy neural network with an adaptive fuzzy partition (AdFPFNN) is proposed, which does not need to preset the number of fuzzy sets and owns various numbers of fuzzy sets on different dimensions. The trick is to use the mean shift algorithm to perform adaptive clustering for each feature individually. For common fuzzy neural networks, dealing with high-dimensional problems is challenging work on account of the "fuzzy rule curse" and "computation underflow". To overcome these issues, an improved version of AdFPFNN is devised to tackle high-dimensional problems, called PaCoAMF-based AdFPFNN. It adopts two techniques to overcome the "fuzzy rule curse" and "computation underflow". One is the construction of a partially combined fuzzy rule base, which can generate rules that are adequate but do not grow exponentially with the number of features. Another is the proposal of an adaptive membership function that can guarantee the elimination of numerical underflow due to the product T-norm. Without feature selection or other dimensional reduction methods, the proposed PaCoAMF-based AdFPFNN can be directly used to solve high-dimensional problems. Simulation results on one constructed dataset and 13 real-world datasets (including 5 high-dimensional problems) confirm the effectiveness of the proposed models. [ABSTRACT FROM AUTHOR]

Air pollution prediction is a burning issue, as pollutants can harm human health. Traditional machine learning models usually aim to improve the overall prediction accuracy but neglect the accuracy for peak values. Moreover, these models are not interpretable. They fail to explain the interactions between various determining factors and their impacts on air pollution. In this paper, we propose a new Hybrid Interpretable Predictive Machine Learning model for the Particulate Matter 2.5 prediction, which carries two novelties. First, a hybrid model structure is constructed with deep neural network and Nonlinear Auto Regressive Moving Average with Exogenous Input model. Second, automatic feature generation and feature selection procedures are integrated into this hybrid model. The experimental results demonstrate the superiority of our model over other models in prediction accuracy for peak values and model interpretability. The proposed model reveals how PM2.5 prediction is estimated by historical PM2.5, weather, and season. The accuracies (measured by correlation coefficients) of 1, 3 and 6-hour-ahead prediction are 0.9870, 0.9332 and 0.8587, respectively. More importantly, the proposed approach presents a new interpretable machine learning framework for time series data, enabling to explain complex dependence of multimode inputs, and to build reliable predictive models. [ABSTRACT FROM AUTHOR]

In multi-label learning, feature selection is a topical issue for addressing high-dimension data. However, most of existing methods adopt imperfect labels to perform feature selection. Although some graph-based multi-label feature selection methods are proposed to deal with the problem, they adopt the fixed graph Laplacian matrix so that the performances of these models are under-performing. To this end, this paper proposes a Dynamic Subspace dual-graph regularized Multi-label Feature Selection method named DSMFS. DSMFS decomposes the original label space into a low-dimensional subspace, and then both the dynamic label-level subspace graph and the feature-level graph are used to obtain a high-quality label subspace to conduct feature selection process. Seven state-of-the-art methods are compared to the proposed method on twelve multi-label benchmark data sets in the experiments. Experimental results demonstrate the superiority of DSMFS. [ABSTRACT FROM AUTHOR]

The technology for Visual Odometry (VO) that estimates the position and orientation of the moving object through analyzing the image sequences captured by on-board cameras, has been well investigated with the rising interest in autonomous driving. This paper studies monocular VO from the perspective of Deep Learning (DL). Unlike most current learning-based methods, our approach, called DeepAVO, is established on the intuition that features contribute discriminately to different motion patterns. Specifically, we present a novel four-branch network to learn the rotation and translation by leveraging Convolutional Neural Networks (CNNs) to focus on different quadrants of optical flow input. To enhance the ability of feature selection, we further introduce an effective channel-spatial attention mechanism to force each branch to explicitly distill related information for specific Frame to Frame (F2F) motion estimation. Experiments on various datasets involving outdoor driving and indoor walking scenarios show that the proposed DeepAVO outperforms the state-of-the-art monocular methods by a large margin, demonstrating competitive performance to the stereo VO algorithm and verifying promising potential for generalization. [ABSTRACT FROM AUTHOR]

Fuzzy rough set is a theoretical framework of fuzzy uncertainty management, and discernibility matrix offers a mathematical foundation for algorithm construction of feature learning. The approaches of fuzzy rough set and discernibility matrix have been successfully applied in single-label learning. However, few works have been done on investigating the foundation of fuzzy rough discernibility matrix on multi-label data. There will be two pivotal problems to be addressed when using fuzzy rough discernibility matrix for multi-label data analysis. One is how to extract sample-level and label-level correlations; and the other is how to utilize the discernibility matrix for algorithm construction. For this reason, in this paper the fuzzy rough discrimination matrix is introduced to deal with the problem of multi-label feature selection. First, the significance of labels in the label space is captured based on the label correlation. Labels with different significances contribute to different weights for measuring the similarity between samples. Hence, a sample similarity matrix in the label space is computed based on the label weighting strategy. Then, a framework of a fuzzy decision system is formalized, in which the discernibility matrix of fuzzy rough sets is introduced as a foundation to evaluate the sample discrimination ability of features. Under the discernibility matrix criterion, a multi-label learning algorithm is developed to select discriminative features from multi-label data. A series of experimental analysis verifies the effectiveness and efficiency of the proposed method. [ABSTRACT FROM AUTHOR]

Traditional approaches that represent observations in vector or matrix notation easily lead to structure information losses and dependency destruction between elements. Tensors are the generalizations of matrices, providing a natural representation of rich structure in real-world multiway arrays. Although tensor decomposition has been verified and used in classification tasks, the heterogeneity gap among different orders is often neglected. In this paper, we exploited tensor decomposition framework and presented a heterogeneous tensor decomposition for robust classification (HTDRC) approach, which integrates nuclear and ℓ 2 , 1 -norm for intrinsic representation learning. Specifically, to obtain the lowest-rank intrinsic representations of tensorial data from an underlying low-dimensional subspace, HTDRC simultaneously learns a set of orthogonality constrained factor matrices and a low-rank constrained representation matrix. To better guide the decomposition process, a robust discriminant feature selection scheme is utilized by imposing ℓ 2 , 1 -norm penalty on the constructed classification loss and the regularized terms simultaneously. Experiments on four datasets demonstrated the superior performance of our proposed approach. [ABSTRACT FROM AUTHOR]

Sparse Bayesian learning (SBL) provides state-of-the-art performance in accuracy, sparsity and probabilistic prediction for classification. In SBL, the regularization priors are automatically determined that avoids an exhaustive hyperparameter selection by cross-validation. However, scalability to large problems is a drawback of SBL because of the inversion of a potentially enormous covariance matrix for updating the regularization priors in every iteration. This paper develops an approximate SBL algorithm called ARP-SBL, where the regularization priors are approximated without inverting the covariance matrix. Therefore, our approach can easily scale up to problems with large data size or feature dimension. It alleviates the long training time and high memory complexity in SBL. Based on ARP-SBL, two scalable nonlinear SBL models: scalable relevance vector machine (ARP-RVM) and scalable sparse Bayesian extreme learning machine (ARP-SBELM) are developed for problems of large data size and large feature size respectively. Experiments on a variety of benchmarks have shown that the proposed models are with competitive accuracy compared to existing methods while i) converging faster; ii) requiring thousands of times less memory; iii) without exhaustive regularized hyperparameter selection; and iv) easily scaling up to large data size and high dimensional features. [ABSTRACT FROM AUTHOR]

In recent years, deep learning techniques have significantly improved the performance of single image super-resolution (SISR). However, this improvement is often achieved at the cost of introducing a large amount of parameters, which limits the real-world applications for SISR. In this paper, we propose a lightweight SISR network called Multi-scale Channel Attention Network for Image Super-Resolution (MCSN). Our contributions are threefold. First of all, the multi-scale feature fusion block (MSFFB) can extract multi-scale features by filters with different receptive fields. Secondly, the channel shuffle attention mechanism (CSAM) encourages the flow of the information across feature channels and enhances the ability of feature selection. Thirdly, the global feature fusion connection (GFFC) can effectively improve feature utilization. Extensive experiments demonstrate that the parameter amount of our method is reduced by 3/4 compared with the current state-of-the-art MSRN method, while both the subjective visual effect and objective quality of the reconstructed high-resolution images are significantly better. [ABSTRACT FROM AUTHOR]

This paper proposes a novel proximal projection neural network (PPNN) to deal with mixed variational inequalities. It is shown that the PPNN has a unique continuous solution under the condition of Lipschitz continuity and that the trajectories of the PPNN converge to the unique equilibrium solution exponentially under some mild conditions. In addition, we study the influence of different parameters on the convergence rate. Furthermore, the proposed PPNN is applied in solving nonlinear complementarity problems, min–max problems, sparse recovery problems and classification and feature selection problems. Finally, numerical and experimental examples are presented to validate the effectiveness of the proposed neurodynamic network. [ABSTRACT FROM AUTHOR]

Capturing rich contextual information is very helpful for semantic segmentation. In addition, the effective use of low-level details and high-level semantics is crucial for semantic segmentation. In this paper, we start from these two aspects, and we propose a self-attention feature fusion network for semantic segmentation (SA-FFNet) to improve semantic segmentation performance. Specifically, we introduced the vertical and horizontal compression attention module (VH-CAM) and the unequal channel pyramid pooling module (UC-PPM). The previous position attention module, such as the position attention module (PAM) in DANet, calculates the similarity between each pixel and all other pixels. However, the amount of information contained in a single isolated pixel is too small, so the resulting position attention weight is not perfect. Our approach is to compress a feature map from both vertical and horizontal directions so that the information contained in each pixel will be richer, and the spatial feature map obtained from this will be better. Experiments show that the position attention weight generated by our vertical and horizontal compression attention module is better than PAM in DANet. Additionally, our UC-PPM on each decoder can provide high-level rich semantic information to guide the selection of low-level feature maps. The proposed model achieves a 76.42% mean IoU on PASCAL VOC2012 and a 73.13% mean IoU on Cityscapes. [ABSTRACT FROM AUTHOR]

This paper investigates a new bacterial colony-based feature selection algorithm to improve the classification accuracy of customers for personalized products recommendation. An attribute learning strategy is developed in this study to update the feature related population. Specifically, the features can be weighted according to their historic contributions to both the individual- and group-based subsets. Additionally, the frequency of appearance is also recorded for the feature candidates to improve the diversity of feature distribution and avoid the over-fitting. Based on the weight-based feature indexes and frequency of appearance records, the performance of feature subsets are enhanced by replacing the features being repeatedly appeared in a same vector. To explore the feasibility of the proposed method for the missing feature problems, the objective of the optimization is to minimize the classification error using the acceptable number of features. K-Nearest Neighbor is employed as the learning technique to cooperate with the proposed feature selection method. The effectiveness of the proposed feature selection method is demonstrated by performing test on the datasets from UCI machine learning repository and real-world data from Amazon customer reviews of products. Compared with other seven feature selections methods, the proposed feature selection algorithm outperforms the other algorithms by achieving higher classification accuracy rate using smaller features. [ABSTRACT FROM AUTHOR]

• Generalized elastic net optimal scoring problem (GenOS) is proposed by utilizing l g norm. • Develop a new block coordinate descent (BCD) method for GenOS. • Convergence properties of the new method are established. • Preliminary experimental results show that our algorithm is efficient not only in classification accuracy but also in the ability of feature selection. Linear discriminant analysis (LDA) is a well-known tool for classification and dimensionality reduction. As an equivalent form of LDA, optimal scoring is presented to solve the classification problem by regression. In this paper, we propose a generalized elastic net optimal scoring problem (GenOS) to find sparse discriminant vectors, where the generalized elastic et is the combination of the ℓ 2 -norm and ℓ q -norm (0 < q ⩽ 1). In GenOS, ℓ q -norm is imposed to confer sparsity to discriminant vectors and ℓ 2 -norm is added to enhance the performance of the classifier. Then, a new efficient algorithm based on block coordinate descent (BCD) method is developed to solve GenOS approximately and the convergence is also established. Simulated data and real-world data are used to empirically illustrate the effectiveness of the proposed model. [ABSTRACT FROM AUTHOR]

Feature selection methods face new challenges in large-scale classification tasks because massive categories are managed in a hierarchical structure. Hierarchical feature selection can take full advantage of the dependencies among hierarchically structured classes. However, most of the existing hierarchical feature selection methods are not robust for dealing with the inevitable data outliers, resulting in a serious inter-level error propagation problem in the following classification process. In this paper, we propose a robust hierarchical feature selection method with a capped ℓ 2 -norm (HFSCN), which can reduce the adverse effects of data outliers and learn relatively robust and discriminative feature subsets for the hierarchical classification process. Firstly, a large-scale global classification task is split into several small local sub-classification tasks according to the hierarchical class structure and the divide-and-conquer strategy, which makes it easy for feature selection modeling. Secondly, a capped ℓ 2 -norm based loss function is used in the feature selection process of each local sub-classification task to eliminate the data outliers, which can alleviate the negative effects outliers and improve the robustness of the learned feature weighted matrix. Finally, an inter-level relation constraint based on the similarity between the parent and child classes is added to the feature selection model, which can enhance the discriminative ability of the selected feature subset for each sub-classification task with the learned robust feature weighted matrix. Compared with seven traditional and state-of-art hierarchical feature selection methods, the superior performance of HFSCN is verified on 16 real and synthetic datasets. [ABSTRACT FROM AUTHOR]

Supervised learning-based code smell detection has become one of the dominant approaches to identify code smell. Existing works optimize the process of code smell detection from multiple aspects, such as high-quality dataset, feature selection, and model, etc. Although the accuracy is improved continuously, researchers are confused about what model are the most suitable ones to detect code smell when considering dataset construction and feature selection. Furthermore, existing surveys for code smell mainly analyze the impact of code smell, categorize the concerns of code smell, and repair code smell. There is a lack of systematic analysis and classification of code smell detection based on supervised learning. To this end, we collect 86 papers of code smell detection based on supervised learning ranging from January 2010 to April 2023. A total of 7 research questions is empirically evaluated from different aspects, such as datasets construction, data pre-processing, feature selection, and model training, etc. We conclude that existing works suffer from issues such as sample imbalance, different attention to types of code smell, and limited feature selection. Finally, we suggest possible future research directions. [ABSTRACT FROM AUTHOR]

Feature selection is a vital preprocessing step in real applications of data mining and machine learning. With the prevalence of high-dimensional hybrid data sets in real-world scenarios, along with the presence of test costs and misclassification costs, the need for effective feature selection methods has become more prominent. However, existing feature selection approaches mainly focus on cost-sensitive data from a single granularity perspective, and they are primarily applicable to single-typed data sets. To address these limitations, this paper presents a novel feature selection approach specifically designed for hybrid data, considering variable test costs and misclassification costs. The proposed method is based on neighborhood multigranulation rough sets (NMRS), which provides more comprehensive and multi-angle data analysis for cost-sensitive hybrid data. First, a novel multigranulation model is developed to effectively process cost-sensitive hybrid data. Building upon this model, a cost-based multi-criteria measure is proposed to evaluate the significance of features. This measure takes into account the comprehensive information of candidate features, including their power in algebraic view, information view, and associated costs. Furthermore, a heuristic feature selection algorithm based on NMRS is proposed to handle hybrid data with the test costs and misclassification costs. This algorithm leverages the benefits of the proposed multigranulation model and cost-based measure to identify the most discriminative features efficiently. Finally, the experimental results on twelve different datasets show that the proposed heuristic algorithm outperforms other compared algorithms, especially in total cost and classification accuracy. [ABSTRACT FROM AUTHOR]

Partial Label Learning (PLL) aims to induce a multi-class classifier to deal with the problem that each training instance is associated with a set of candidate labels, among which only one is valid but unknown. Feature selection, which choses some features related to the task (e.g., classification) and omits the unrelated ones, is challenging for PLL due to the ambiguous labeling information. In this paper, a random forest based method, namely, RFUTE, is developed to deal with the feature selection challenge in PLL. Due to the fact that the ground-truth labels are inaccessible in the training process, RFUTE first disambiguates the candidate labels, and then operates feature selection by calculating and sorting the total change of information entropy over all trees in the forest for all features. More importantly, a supplement approach is proposed after disambiguation to avoid the class imbalance problem. Comprehensive experiments over both synthetic and real-world partial label data sets empirically justify the effectiveness of RFUTE in improving the generalization performance of well-established PLL methods. [ABSTRACT FROM AUTHOR]

[Display omitted] • A cosine transform-based similarity calculation method is proposed to cope with information loss in data discretization when the strategy of mutual information is adopted for feature selection. Feature selection (FS) based on mutual information (MI) metrics needs to discretize the data in preprocessing, which is a convenient way to identify correlation between features. However, information loss often occurs in data discretization. In order to solve this information loss problem, this paper proposes a FS algorithm based on cosine similarity coefficient and information measurement criterion (CSCIM_FS). First, the MI between features and tags is calculated, and features are sorted out according to the MI calculated. Then, a feature matrix is constructed to transform the one-dimensional feature sequence into a two-dimensional square matrix. Next, cosine transform is adopted to obtain the high-frequency components of the feature matrix, and sampling is conducted to derive the hash fingerprint of the feature matrix. After that, the similarity between every two features is calculated on the basis of the hash fingerprints of different features. Finally, the feature weight is calculated according to tags, the MI and similarity between features, and a key feature subset is obtained and used to conduct feature selection from the data. The experimental results on several UCI public datasets show that CSCIM_FS algorithm selected a feature subset with high accuracy, and that this algorithm performs better than MIM, CMIM, mRMR and other algorithms. [ABSTRACT FROM AUTHOR]

[Display omitted] • Particle Swarm Optimization can be used to perform feature selection for domain adaptation. • It has been previously applied using classifiers to evaluate the goodness of subsets of features. • We explore a novel option: using complexity measures instead of classifiers. • We compare both methods in terms of performance, speed and size of the resulting set of features. Particle Swarm Optimization is an optimization algorithm that mimics the behaviour of a flock of birds, setting multiple particles that explore the search space guided by a fitness function in order to find the best possible solution. We apply the Sticky Binary Particle Swarm Optimization algorithm to perform feature selection for domain adaptation, a specific type of transfer learning in which the source and the target domain have a common feature space, a common task, but different distributions. When applying Particle Swarm Optimization, classification error is usually employed in the fitness function to evaluate the goodness of subsets of features. In this paper, we aim to compare this approach with using complexity metrics instead, under the assumption that reducing the complexity of the problem will lead to results that are independent from the classifier used for testing while being less computationally demanding. Therefore, we carried out experiments to compare the performance of both approaches in terms of classification accuracy, speed and number of features selected. We found out that our proposal, although in some cases incurs in a slight degradation of classification performance, it is indeed faster and selects fewer features, making it a feasible trade-off. [ABSTRACT FROM AUTHOR]

Feature selection can help to reduce data redundancy and improve algorithm performance in actual tasks. Most of the embedded feature selection models are constructed based on square loss and hinge loss. However, these models based on the square loss cannot directly evaluate the discriminability of the samples in the feature subspace, and these methods based on the hinge loss are difficult to solve due to their complex objective functions. To deal with these problems, a Feature Selection method with Multi-class Logistic Regression (FSMLR) is proposed in this paper. Firstly, we construct a linear function to measure the difference between the distance from samples to their regression hyperplane and the distance from these samples to regression hyperplanes of other classes, which could be used to strengthen the discriminant property of the embedded model. Then, we design a re-weighting matrix with a ℓ 2 , 0 -norm sparse condition as well as a discrete condition, which is used to select features in the subspace. Considering that it is difficult to solve the re-weighting matrix with the discrete and sparse conditions in an optimization problem, we relax these two conditions and present a feature selection model via a re-weighted multi-class logistic regression with the two relaxed constraints. Finally, we add the F -norm regularization in our model to avoid overfitting, and its unconstrained equivalent transformation with ℓ 2 , p -norm regularization is derived to explore the function of the re-weighting matrix. The gradient descent algorithm could be used to solve the FSMLR. Especially, when the regularization term in the equivalence problem is set to ℓ 2 , 1 -norm, the global optimal solution can be obtained. Extensive experiments on multiple public data sets prove that FSMLR outperforms other competitors. [ABSTRACT FROM AUTHOR]

The applications of traditional statistical feature selection methods to high-dimension, low-sample-size data often struggle and encounter challenging problems, such as overfitting, curse of dimensionality, computational infeasibility, and strong model assumptions. In this paper, we propose a novel two-step nonparametric approach called Deep Feature Screening (DeepFS) that can overcome these problems and identify significant features with high precision for ultra high-dimensional, low-sample-size data. This approach first extracts a low-dimensional representation of input data and then applies feature screening on the original input feature space based on multivariate rank distance correlation recently developed by Deb and Sen (2021). This approach combines the strengths of both deep neural networks and feature screening, thereby having the following appealing features in addition to its ability of handling ultra high-dimensional data with small number of samples: (1) it is model free and distribution free; (2) it can be used for both supervised and unsupervised feature selection; and (3) it is capable of recovering the original input data. The superiority of DeepFS is demonstrated via extensive simulation studies and real data analyses. [ABSTRACT FROM AUTHOR]