Search

In the field of risk assessment, the traditional econometric models are generally used to assess credit risk. And with the introduction of the "dual-carbon" goals to promote the development of a low-carbon economy, the scale of green credit in China has rapidly expanded. But with the advent of the big data era, due to the poor interpretability of a traditional single machine learning model, it is difficult to capture nonlinear relationships, and there are shortcomings in prediction accuracy and robustness. This paper selects the adjusted ensemble learning model based on the homogeneous and heterogeneous factors for user default prediction, which can efficiently process large quantities of high-dimensional data. This article adjusts each model to adapt to the task and innovatively compares various models. In this paper, the missing value filling method, feature selection, and ensemble model are studied and discussed, and the optimal ensemble model is obtained. When comparing the predictions of single models and ensemble models, the accuracy, sensitivity, specificity, F1-Score, Kappa, and MCC of Categorical Features Gradient Boosting (CatBoost) and Random undersampling Boosting (RUSBoost) all reach 100%. The experimental results prove that the algorithm based on adjusted homogeneous and heterogeneous ensemble learning can predict the user default efficiently and accurately. This paper also provides some references for establishing a risk assessment index system. [ABSTRACT FROM AUTHOR]

In real-world applications, feature selection is crucial for enhancing the performance of data science and machine learning models. Typically, feature selection is a complex combinatorial optimization problem and a multi-objective optimization problem. Its primary goals are to reduce the dimensionality of the dataset and enhance the performance of machine learning algorithms. The selection of features in high-dimensional datasets is challenging due to the intricate relationships between features, which pose significant challenges to the performance and computational efficiency of algorithms. This paper introduces a Knowledge-Guided Competitive Co-Evolutionary Algorithm (KCCEA) for feature selection, especially for high-dimensional features. In the proposed algorithm, we make improvements to the foundational dominance-based multi-objective evolutionary algorithm in two aspects. First, the use of feature correlation as knowledge to guide evolution enhances the search speed and quality of traditional multi-objective evolutionary algorithm solutions. Second, a dynamically allocated competitive–cooperative evolutionary mechanism is proposed, integrating the improved knowledge-guided evolution with traditional evolutionary algorithms, further enhancing the search efficiency and diversity of solutions. Through rigorous empirical testing on various datasets, the KCCEA demonstrates superior performance compared to basic multi-objective evolutionary algorithms, providing effective solutions to multi-objective feature selection problems while enhancing the interpretability and effectiveness of prediction models. [ABSTRACT FROM AUTHOR]

SDN has the ability to transform network design by providing increased versatility and effective regulation. Its programmable centralized controller gives network administration employees more authority, allowing for more seamless supervision. However, centralization makes it vulnerable to a variety of attack vectors, with distributed denial of service (DDoS) attacks posing a serious concern. Feature selection-based Machine Learning (ML) techniques are more effective than traditional signature-based Intrusion Detection Systems (IDS) at identifying new threats in the context of defending against distributed denial of service (DDoS) attacks. In this study, NGBoost is compared with four additional machine learning (ML) algorithms: convolutional neural network (CNN), Stochastic Gradient Descent (SGD), Decision Tree, and Random Forest, in order to assess the effectiveness of DDoS detection on the CICDDoS2019 dataset. It focuses on important measures such as F1 score, recall, accuracy, and precision. We have examined NeTBIOS, a layer-7 attack, and SYN, a layer-4 attack, in our paper. Our investigation shows that Natural Gradient Boosting and Convolutional Neural Networks, in particular, show promise with tabular data categorization. In conclusion, we go through specific study results on protecting against attacks using DDoS. These experimental findings offer a framework for making decisions. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Engineering (17264073) is the property of Republic of Iraq Ministry of Higher Education & Scientific Research (MOHESR) and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

In the era of healthcare and its related research fields, the dimensionality problem of high-dimensional data is a massive challenge as it is crucial to identify significant genes while conducting research on diseases like cancer. As a result, studying new Machine Learning (ML) techniques for raw gene expression biomedical data is an important field of research. Disease detection, sample classification, and early disease prediction are all important analyses of high-dimensional biomedical data in the field of bioinformatics. Recently, machine-learning techniques have dramatically improved the analysis of high-dimension biomedical data sets. Nonetheless, researchers' studies on biomedical data faced the challenge of vast dimensions, i.e., the vast features (genes) with a very low sample space. In this paper, two-dimensionality reduction methods, feature selection, and feature extraction are introduced with a systematic comparison of several dimension reduction techniques for the analysis of high-dimensional gene expression biomedical data. We presented a systematic review of some of the most popular nature-inspired algorithms and analyzed them. The paper is mainly focused on the original principles behind each of the algorithms and their applications for cancer classification and prediction from gene expression data. Lastly, the advantages and disadvantages of nature-inspired algorithms for biomedical data are evaluated. This review paper may guide researchers to choose the most effective algorithm for cancer classification and prediction for the satisfactory analysis of high-dimensional biomedical data. [ABSTRACT FROM AUTHOR]

Background: Cancer diagnosis based on machine learning has become a popular application direction. Support vector machine (SVM), as a classical machine learning algorithm, has been widely used in cancer diagnosis because of its advantages in high-dimensional and small sample data. However, due to the high-dimensional feature space and high feature redundancy of gene expression data, SVM faces the problem of poor classification effect when dealing with such data. Methods: Based on this, this paper proposes a hybrid feature selection algorithm combining information gain and grouping particle swarm optimization (IG-GPSO). The algorithm firstly calculates the information gain values of the features and ranks them in descending order according to the value. Then, ranked features are grouped according to the information index, so that the features in the group are close, and the features outside the group are sparse. Finally, grouped features are searched using grouping PSO and evaluated according to in-group and out-group. Results: Experimental results show that the average accuracy (ACC) of the SVM on the feature subset selected by the IG-GPSO is 98.50%, which is significantly better than the traditional feature selection algorithm. Compared with KNN, the classification effect of the feature subset selected by the IG-GPSO is still optimal. In addition, the results of multiple comparison tests show that the feature selection effect of the IG-GPSO is significantly better than that of traditional feature selection algorithms. Conclusion: The feature subset selected by IG-GPSO not only has the best classification effect, but also has the least feature scale (FS). More importantly, the IG-GPSO significantly improves the ACC of SVM in cancer diagnostic. [ABSTRACT FROM AUTHOR]

A critical aspect of software engineering is Software fault prediction which aims to identify and prevent errors in software systems before their release which can cause failures or issues for its users. Various techniques and tools have been developed to detect software faults, including static code analysis, dynamic testing, and machine learning-based approaches. In past few years, the world has seen a growing interest in the use of ML models for predicting software faults, as they can effectively analyse high dimensional datasets and detect complex patterns which are difficult for human experts to detect. However, developing accurate and reliable software fault detection models requires careful selection of data, feature engineering, and model evaluation. This purpose of this paper is to present a comprehensive analysis of potential applications and future research directions in the field of software fault detection. The study emphasizes the importance of identifying and addressing software faults to ensure the reliability and efficiency of software systems. Additionally, the paper outlines various approaches and techniques that can be employed for effective software fault detection. [ABSTRACT FROM AUTHOR]

For the feature selection of network intrusion detection, the issue of numerous redundant features arises, posing challenges in enhancing detection accuracy and adversely affecting overall performance to some extent. Artificial rabbits optimization (ARO) is capable of reducing redundant features and can be applied for the feature selection of network intrusion detection. The ARO exhibits a slow iteration speed in the exploration phase of the population and is prone to an iterative stagnation condition in the exploitation phase, which hinders its ability to deliver outstanding performance in the aforementioned problems. First, to enhance the global exploration capabilities further, the thinking of ARO incorporates the mud ring feeding strategy from the bottlenose dolphin optimizer (BDO). Simultaneously, for adjusting the exploration and exploitation phases, the ARO employs an adaptive switching mechanism. Second, to avoid the original algorithm getting trapped in the local optimum during the local exploitation phase, the levy flight strategy is adopted. Lastly, the dynamic lens-imaging strategy is introduced to enhance population variety and facilitate escape from the local optimum. Then, this paper proposes a modified ARO, namely LBARO, a hybrid algorithm that combines BDO and ARO, for feature selection in the network intrusion detection model. The LBARO is first empirically evaluated to comprehensively demonstrate the superiority of the proposed algorithm, using 8 benchmark test functions and 4 UCI datasets. Subsequently, the LBARO is integrated into the feature selection process of the network intrusion detection model for classification experimental validation. This integration is validated utilizing the NSL-KDD, UNSW NB-15, and InSDN datasets, respectively. Experimental results indicate that the proposed model based on LBARO successfully reduces redundant characteristics while enhancing the classification capabilities of network intrusion detection. [ABSTRACT FROM AUTHOR]

When performing classification tasks on high-dimensional data, traditional machine learning algorithms often fail to filter out valid information in the features adequately, leading to low levels of classification accuracy. Therefore, this paper explores the high-dimensional data from both the data feature dimension and the model ensemble dimension. We propose a high-dimensional ensemble learning classification algorithm focusing on feature space reconstruction and classifier ensemble, called the HDELC algorithm. First, the algorithm considers feature space reconstruction and then generates a feature space reconstruction matrix. It effectively achieves feature selection and reconstruction for high-dimensional data. An optimal feature space is generated for the subsequent ensemble of the classifier, which enhances the representativeness of the feature space. Second, we recursively determine the number of classifiers and the number of feature subspaces in the ensemble model. Different classifiers in the ensemble system are assigned mutually exclusive non-intersecting feature subspaces for model training. The experimental results show that the HDELC algorithm has advantages compared with most high-dimensional datasets due to its more efficient feature space ensemble capability and relatively reliable ensemble operation performance. The HDELC algorithm makes it possible to solve the classification problem for high-dimensional data effectively and has vital research and application value. [ABSTRACT FROM AUTHOR]

Extracting effective features from high-dimensional datasets is crucial for determining the accuracy of regression and classification models. Model predictions based on causality are known for their robustness. Thus, this paper introduces causality into feature selection and utilizes Feature Selection based on NOTEARS causal discovery (FSNT) for effective feature extraction. This method transforms the structural learning algorithm into a numerical optimization problem, enabling the rapid identification of the globally optimal causality diagram between features and the target variable. To assess the effectiveness of the FSNT algorithm, this paper evaluates its performance by employing 10 regression algorithms and 8 classification algorithms for regression and classification predictions on six real datasets from diverse fields. These results are then compared with three mainstream feature selection algorithms. The results indicate a significant average decline of 54.02% in regression prediction achieved by the FSNT algorithm. Furthermore, the algorithm exhibits exceptional performance in classification prediction, leading to an enhancement in the precision value. These findings highlight the effectiveness of FSNT in eliminating redundant features and significantly improving the accuracy of model predictions. [ABSTRACT FROM AUTHOR]

In the era of big data, the demand for multivariate time series prediction has surged, drawing increased attention to feature selection and neural networks in machine learning. However, certain feature selection methods neglect the alignment between actual data sample differences and clustering results, while neural networks lack automatic parameter adjustment in response to changing target features. This paper presents the MBSSA-Bi-AESN model, a Bi-directional Adaptive Echo State Network that utilizes the modified salp swarm algorithm (MBSSA) and feature selection to address the limitations of manually set parameters. Initial feature subset selection involves assigning weights based on the consistency of clustering results with differences. Subsequently, the four critical parameters in the Bi-AESN model are optimized using MBSSA. The optimized Bi-AESN model and selected feature subset are then integrated for simultaneous model learning and optimal feature subset selection. Experimental analysis on eight datasets demonstrates the superior prediction accuracy of the MBSSA-Bi-AESN model compared to benchmark models, underscoring its feasibility, validity, and universality. [ABSTRACT FROM AUTHOR]

The salp swarm algorithm (SSA) is a well-known optimization algorithm that is increasingly being utilized to solve many sorts of optimization problems. However, SSA may converge to sub-optimal solutions when it is applied to discrete problems such as the feature selection (FS) problem. This paper presents the enhanced opposition-based learning salp swarm algorithm (EOSSA), which is an improved SSA algorithm for solving the FS problem in intrusion detection systems (IDS). EOSSA incorporates four improvements into the original SSA algorithm. Firstly, the opposition-based learning (OBL) method is used in the initialization step of SSA to boost its population diversity. Secondly, the Elite opposition-based learning (EOBL) is used in the improvement loop of SSA to improve its exploration ability. Third, a variable neighborhood search (VNS) method is used in the improvement loop of SSA to improve its exploration mechanism to improve the local search space. Lastly, the Sigmoid binary transform function is used to convert the continuous candidate solutions produced by SSA into discrete binary solutions. EOSSA was evaluated against eighteen popular optimization algorithms (e.g., improved salp swarm algorithm based on opposition-based learning (ISSA), SSA, particle swarm algorithm (PSO), cuckoo search (CS), bat algorithm (BA), and Harris Hawk Optimization (HHO)) using eleven popular intrusion detection datasets (CICIDS2017, CSE-CIC-IDS2018, CICDDOS2019, CIRA-CIC-DoH, Intrusion detection 2018, UNSW-NB15, NSL-KDD, Phishing Legitimate, Malmem2022, IoT, and LUFlow Network) to Detect IoT Botnet Attacks. Moreover, EOSSA was compared with four machine learning algorithms (Decision Tree (DT), logistic regression (LR), Naive Bayes (NB), and K-Nearest Neighborhood (KNN)). The overall simulation results suggested that the proposed method is superior to the other algorithms in terms of the accuracy and number of selected features. The statistical analysis of the simulation results using the Friedman and Wilcoxon signed-rank test confirms the superiority of the proposed method. [ABSTRACT FROM AUTHOR]

In Machine Learning (ML), Feature Selection (FS) plays a crucial part in reducing data's dimensionality and enhancing any proposed framework's performance. However, in real-world applications, FS work suffers from high dimensionality, computational and storage complexity, noisy or ambiguous nature, high performance, etc. The area of FS is very vast and challenging in its nature. There are lots of work that have been reported on FS over the various area of applications. This paper has discussed FS's framework and the multiple models of FS with detailed descriptions. We have also classified the various FS algorithms with respect to the data, i.e., structured or labeled data and unstructured data for the different applications of ML. We have also discussed what essential features are, the commonly used FS methods, the widely used datasets, and the widely used work done in the various ML fields for the FS task. Here we try to view the multiple comparison experimental results of FS work in different result discussions. This paper draws a descriptive survey on FS with the associated area of real-world problem domains. This paper's main objective is to understand the main idea of FS work and identify the core idea of how FS will be applicable in various problem domains. [ABSTRACT FROM AUTHOR]

The efficiency and the effectiveness of a machine learning (ML) model are greatly influenced by feature selection (FS), a crucial preprocessing step in machine learning that seeks out the ideal set of characteristics with the maximum accuracy possible. Due to their dominance over traditional optimization techniques, researchers are concentrating on a variety of metaheuristic (or evolutionary) algorithms and trying to suggest cutting-edge hybrid techniques to handle FS issues. The use of hybrid metaheuristic approaches for FS has thus been the subject of numerous research works. The purpose of this paper is to critically assess the existing hybrid FS approaches and to give a thorough literature review on the hybridization of different metaheuristic/evolutionary strategies that have been employed for supporting FS. This article reviews pertinent documents on hybrid frameworks that were published in the period from 2009 to 2022 and offers a thorough analysis of the used techniques, classifiers, datasets, applications, assessment metrics, and schemes of hybridization. Additionally, new open research issues and challenges are identified to pinpoint the areas that have to be further explored for additional study. [ABSTRACT FROM AUTHOR]

Sentiment analysis is considered one of the significant trends of the recent few years. Due to the high importance and increasing use of social media and electronic services, the need for reviewing and enhancing the provided services has become crucial. Revising the user services is based mainly on sentiment analysis methodologies for analyzing users' polarities to different products and applications. Sentiment analysis for Arabic reviews is a major concern due to high morphological linguistics and complex polarity terms expressed in the reviews. In addition, the users can present their orientation towards a service or a product by using a hybrid or mix of polarity terms related to slang and standard terminologies. This paper provides a comprehensive review of recent sentiment analysis methods based on lexicon or machine learning (ML). The comparison provides a clear vision of the number of classes, the used dialect, the annotated algorithms, and their performance. The proposed methodology is based on cross-validation of Arabic data using a k-fold mechanism that splits the dataset into training and testing folds; subsequently, the data preprocessing is executed to clean sentiments from unwanted terms that can affect data analysis. A vectorization of the dataset is then applied using TF–IDF for counting word and polarity terms. Furthermore, a feature selection stage is processed using Pearson, Chi2, and Random Forest (RF) methods for mapping the compatibility between input and target features. This paper also proposed an algorithm called the forward fusion feature for sentiment analysis (FFF-SA) to provide a feature selection that applied different machine learning (ML) classification models for each chunk of k features and accumulative features on the Arabic dataset. The experimental results measured and scored all accuracies between the feature importance method and ML models. The best accuracy is recorded with the Naïve Bayes (NB) model with the RF method. [ABSTRACT FROM AUTHOR]

Alzheimer's disease (AD) is the most common cause of dementia. Accurate prediction and diagnosis of AD and its prodromal stage, i.e., mild cognitive impairment (MCI), is essential for the possible delay and early treatment for the disease. In this paper, we adopt the data from the China Longitudinal Aging Study (CLAS), which was launched in 2011, and includes a joint effort of 15 institutions all over the country. Four thousand four hundred and eleven people who are at least 60 years old participated in the project, where 3,514 people completed the baseline survey. The survey collected data including demographic information, daily lifestyle, medical history, and routine physical examination. In particular, we employ ensemble learning and feature selection methods to develop an explainable prediction model for AD and MCI. Five feature selection methods and nine machine learning classifiers are applied for comparison to find the most dominant features on AD/MCI prediction. The resulting model achieves accuracy of 89.2%, sensitivity of 87.7%, and specificity of 90.7% for MCI prediction, and accuracy of 99.2%, sensitivity of 99.7%, and specificity of 98.7% for AD prediction. We further utilize the SHapley Additive exPlanations (SHAP) algorithm to visualize the specific contribution of each feature to AD/MCI prediction at both global and individual levels. Consequently, our model not only provides the prediction outcome, but also helps to understand the relationship between lifestyle/physical disease history and cognitive function, and enables clinicians to make appropriate recommendations for the elderly. Therefore, our approach provides a new perspective for the design of a computer-aided diagnosis system for AD and MCI, and has potential high clinical application value. [ABSTRACT FROM AUTHOR]

The Internet of Things (IoT) and network-enabled smart devices are crucial to the digitally interconnected society of the present day. However, the increased reliance on IoT devices increases their susceptibility to malicious activities within network traffic, posing significant challenges to cybersecurity. As a result, both system administrators and end users are negatively affected by these malevolent behaviours. Intrusion-detection systems (IDSs) are commonly deployed as a cyber attack defence mechanism to mitigate such risks. IDS plays a crucial role in identifying and preventing cyber hazards within IoT networks. However, the development of an efficient and rapid IDS system for the detection of cyber attacks remains a challenging area of research. Moreover, IDS datasets contain multiple features, so the implementation of feature selection (FS) is required to design an effective and timely IDS. The FS procedure seeks to eliminate irrelevant and redundant features from large IDS datasets, thereby improving the intrusion-detection system's overall performance. In this paper, we propose a hybrid wrapper-based feature-selection algorithm that is based on the concepts of the Cellular Automata (CA) engine and Tabu Search (TS)-based aspiration criteria. We used a Random Forest (RF) ensemble learning classifier to evaluate the fitness of the selected features. The proposed algorithm, CAT-S, was tested on the TON_IoT dataset. The simulation results demonstrate that the proposed algorithm, CAT-S, enhances classification accuracy while simultaneously reducing the number of features and the false positive rate. [ABSTRACT FROM AUTHOR]

CC's (Cloud Computing) networks are distributed and dynamic as signals appear/disappear or lose significance. MLTs (Machine learning Techniques) train datasets which sometime are inadequate in terms of sample for inferring information. A dynamic strategy, DevMLOps (Development Machine Learning Operations) used in automatic selections and tunings of MLTs result in significant performance differences. But, the scheme has many disadvantages including continuity in training, more samples and training time in feature selections and increased classification execution times. RFEs (Recursive Feature Eliminations) are computationally very expensive in its operations as it traverses through each feature without considering correlations between them. This problem can be overcome by the use of Wrappers as they select better features by accounting for test and train datasets. The aim of this paper is to use DevQLMLOps for automated tuning and selections based on orchestrations and messaging between containers. The proposed AKFA (Adaptive Kernel Firefly Algorithm) is for selecting features for CNM (Cloud Network Monitoring) operations. AKFA methodology is demonstrated using CNSD (Cloud Network Security Dataset) with satisfactory results in the performance metrics like precision, recall, F-measure and accuracy used. [ABSTRACT FROM AUTHOR]

With an increase in the use of software, the incidence of bugs and resulting maintenance costs also increase. In open source projects, developer reassignment accounts for approximately 50%. Software maintenance costs can be reduced if appropriate developers are recommended to resolve bugs. In this study, features are extracted by applying feature selection for each developer. These features are entered into CNN-LSTM algorithm to learn the model and recommend appropriate developers. To compare the performance of the proposed model, open source projects (Google Chrome, Mozilla Core, and Mozilla Firefox) were used to compare the performance of the proposed method with a baseline for developer recommendation. In this paper, the performance showed 54% for F-measure and 52% for accuracy in open source projects. The proposed model has improved and showed about a 13% more effective performance improvement than with DeepTriage. It was discovered that the performance of the proposed model was better. [ABSTRACT FROM AUTHOR]

Machine Learning and Data Mining algorithms are used extensively to enhance the performance of Intrusion Detection Systems. The number of training instances and the dimensionality of data are crucial factors affecting the performance of the model built during the training of any supervised learning algorithms. A sufficient proportion of instances having relevant features from all classes of attacks and normal traffic are considered most desirable while building the classification model that classifies the network traffic into attack and normal. This paper proposes a methodology to improve the accuracy of the model by giving importance to the relevant features that can contribute to model building. The feature selection using correlation‐based and information gain‐based techniques during training and testing contributes much to the detection of stealthier attacks and minority attacks. Then the features of the less detected attacks are identified as the second phase of the filter that is used to improve the performance. The relevant features of stealthy attacks are identified based on the correlation of corresponding features of the attack and normal data as the attacks are made stealthy mostly by making it resemble the normal traffic. Finally, the attacks that are rarely found in the training data are oversampled to improve their detection. CICIDS 2017 data set is employed as it comprises stealthier attacks generated using modern tools. NSL KDD data set is also used for evaluation to compare the proposed work with existing literature as it is used in most of the available literature. The results show superior performance with an accuracy of 99.8%, false positive rate of 0.2%, and a detection rate and 99.8%. [ABSTRACT FROM AUTHOR]

This paper proposes a novel network anomaly detection framework based on data balance and feature selection. Different from the previous binary classification of network intrusion, the network anomaly detection strategy proposed in this paper solves the problem of multiple classification of network intrusion. Regarding the common data imbalance of a network intrusion detection set, a resampling strategy generated by random sampling and Borderline SMOTE data is developed for data balance. According to the features of the intrusion detection dataset, feature selection is carried out based on information gain rate. Experiments are carried out on three basic machine learning algorithms (K-nearest neighbor algorithm (KNN), decision tree (DT), random forest (RF)), and the optimal feature selection scheme is obtained. [ABSTRACT FROM AUTHOR]

Relevant attribute selection in machine learning is a key aspect aimed at simplifying the problem, reducing its dimensionality, and consequently accelerating computation. This paper proposes new algorithms for selecting relevant features and evaluating and selecting a subset of relevant objects in a dataset. Both algorithms are mainly based on the use of a fuzzy approach. The research presented here yielded preliminary results of a new approach to the problem of selecting relevant attributes and objects and selecting appropriate ranges of their values. Detailed results obtained on the Sonar dataset show the positive effects of this approach. Moreover, the observed results may suggest the effectiveness of the proposed method in terms of identifying a subset of truly relevant attributes from among those identified by traditional feature selection methods. [ABSTRACT FROM AUTHOR]

A malicious URL is a link that is created to spread spams, phishing, malware, ransomware, spyware, etc. A user may download malware that can adversely affect the computer by clicking on an infected URL, or might be convinced to provide confidential information to a fraudulent website causing serious losses. These threats must be identified and handled in a decent time and in an effective way. Detection is traditionally done through the blacklist usage method, which relies on keyword matching with previously known malicious domain names stored in a repository. This method is fast and easy to implement, with the advantage of having low false-positive rates regarding previously recognized malicious URLs. However, this method cannot recognize newly created malicious URLs. To solve this problem, many machine-learning models have been used. In this paper, we introduce an effective machine learning approach that uses an ensemble learner algorithm called AdaBoost (Adaptive Boosting), combined with different algorithms that enhance detection. For datasets filtration, we used CfsSubsetEval technique, which is an algorithm that searches for a subset of features that work well together. Datasets were collected from the UNB repository; divided into four categories: spam, phishing, malware, and defacement URLs; combined with benign URLs, dataset content is based on lexical features. The experimental results indicate that the proposed approach was successful in enhancing the detection accuracy of malicious URLs with less false-positive rates for all experimental algorithms. [ABSTRACT FROM AUTHOR]

Copyright of Military Technical Courier / Vojnotehnicki Glasnik is the property of Military Technical Courier / Vojnotehnicki Glasnik and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

With the development of artificial intelligence, big data classification technology provides the advantageous help for the medicine auxiliary diagnosis research. While due to the different conditions in the different sample collection, the medical big data is often imbalanced. The class-imbalance problem has been reported as a serious obstacle to the classification performance of many standard learning algorithms. SMOTE algorithm could be used to generate sample points randomly to improve imbalance rate, but its application is affected by the marginalization generation and blindness of parameter selection. Focusing on this problem, an improved SMOTE algorithm based on Normal distribution is proposed in this paper, so that the new sample points are distributed closer to the center of the minority sample with a higher probability to avoid the marginalization of the expanded data. Experiments show that the classification effect is better when use proposed algorithm to expand the imbalanced dataset of Pima, WDBC, WPBC, Ionosphere and Breast-cancer-wisconsin than the original SMOTE algorithm. In addition, the parameter selection of the proposed algorithm is analyzed and it is found that the classification effect is the best when the distribution characteristics of the original data was maintained best by selecting appropriate parameters in our designed experiments. [ABSTRACT FROM AUTHOR]

Intrusion detection systems (IDS) play a crucial role in securing networks and identifying malicious activity. This is a critical problem in cyber security. In recent years, metaheuristic optimization algorithms and deep learning techniques have been applied to IDS to improve their accuracy and efficiency. Generally, optimization algorithms can be used to boost the performance of IDS models. Deep learning methods, such as convolutional neural networks, have also been used to improve the ability of IDS to detect and classify intrusions. In this paper, we propose a new IDS model based on the combination of deep learning and optimization methods. First, a feature extraction method based on CNNs is developed. Then, a new feature selection method is used based on a modified version of Growth Optimizer (GO), called MGO. We use the Whale Optimization Algorithm (WOA) to boost the search process of the GO. Extensive evaluation and comparisons have been conducted to assess the quality of the suggested method using public datasets of cloud and Internet of Things (IoT) environments. The applied techniques have shown promising results in identifying previously unknown attacks with high accuracy rates. The MGO performed better than several previous methods in all experimental comparisons. [ABSTRACT FROM AUTHOR]

Multi-label learning has widely applied in machine learning and data mining. The purpose of feature selection is to select an approximately optimal feature subset to characterize the original feature space. Similar to single-label data, feature selection is an import preprocessing step to enhance the performance of multi-label classification model. In this paper, we propose a multi-label feature selection approach with Pareto optimality for continuous data, called MLFSPO. It maps multi-label features to high-dimensional space to evaluate the correlation between features and labels by utilizing the Hilbert-Schmidt Independence Criterion (HSIC). Then, the feature subset obtains by combining the Pareto optimization with feature ordering criteria and label weighting. Eventually, extensive experimental results on publicly available data sets show the effectiveness of the proposed algorithm in multi-label tasks. [ABSTRACT FROM AUTHOR]

With the development of civil aviation, the number of flights keeps increasing and the flight delay has become a serious issue and even tends to normality. This paper aims to prove that Stacking algorithm has advantages in airport flight delay prediction, especially for the algorithm selection problem of machine learning technology. In this research, the principle of the Stacking classification algorithm is introduced, the SMOTE algorithm is selected to process imbalanced datasets, and the Boruta algorithm is utilized for feature selection. There are five supervised machine learning algorithms in the first-level learner of Stacking including KNN, Random Forest, Logistic Regression, Decision Tree, and Gaussian Naive Bayes. The second-level learner is Logistic Regression. To verify the effectiveness of the proposed method, comparative experiments are carried out based on Boston Logan International Airport flight datasets from January to December 2019. Multiple indexes are used to comprehensively evaluate the prediction results, such as Accuracy, Precision, Recall, F1 Score, ROC curve, and AUC Score. The results show that the Stacking algorithm not only could improve the prediction accuracy but also maintains great stability. [ABSTRACT FROM AUTHOR]

This paper is devoted to the problem of class imbalance in machine learning, focusing on the intrusion detection of rare classes in computer networks. The problem of class imbalance occurs when one class heavily outnumbers examples from the other classes. In this paper, we are particularly interested in classifiers, as pattern recognition and anomaly detection could be solved as a classification problem. As still a major part of data network traffic of any organization network is benign, and malignant traffic is rare, researchers therefore have to deal with a class imbalance problem. Substantial research has been undertaken in order to identify these methods or data features that allow to accurately identify these attacks. But the usual tactic to deal with the imbalance class problem is to label all malignant traffic as one class and then solve the binary classification problem. In this paper, however, we choose not to group or to drop rare classes but instead investigate what could be done in order to achieve good multi-class classification efficiency. Rare class records were up-sampled using SMOTE method (Chawla et al., 2002) to a preset ratio targets. Experiments with the 3 network traffic datasets, namely CIC-IDS2017, CSE-CIC-IDS2018 (Sharafaldin et al., 2018) and LITNET-2020 (Damasevicius et al., 2020) were performed aiming to achieve reliable recognition of rare malignant classes available in these datasets. Popular machine learning algorithms were chosen for comparison of their readiness to support rare class detection. Related algorithm hyper parameters were tuned within a wide range of values, different data feature selection methods were used and tests were executed with and without over-sampling to test the multiple class problem classification performance of rare classes. Machine learning algorithms ranking based on Precision, Balanced Accuracy Score, G ¯ , and prediction error Bias and Variance decomposition, show that decision tree ensembles (Adaboost, Random Forest Trees and Gradient Boosting Classifier) performed best on the network intrusion datasets used in this research. [ABSTRACT FROM AUTHOR]

Keeping computer reliability to confirm reliable, secure, and truthful correspondence of data between different enterprises is a major security issue. Ensuring information correspondence over the web or computer grids is always under threat of hackers or intruders. Many techniques have been utilized in intrusion detections, but all have flaws. In this paper, a new hybrid technique is proposed, which combines the Ensemble of Feature Selection (EFS) algorithm and Teaching LearningBased Optimization (TLBO) techniques. In the proposed, EFSTLBO method, the EFS strategy is applied to rank the features for choosing the ideal best subset of applicable information, and the TLBO is utilized to identify the most important features from the produced datasets. The TLBO algorithm uses the Extreme Learning Machine (ELM) to choose the most effective attributes and to enhance classification accuracy. The performance of the recommended technique is evaluated in a benchmark dataset. The experimental outcomes depict that the proposed model has high predictive accuracy, detection rate, false-positive rate, and requires less significant attributes than other techniques known from the literature. [ABSTRACT FROM AUTHOR]

In recent years, feature selection has emerged as a major challenge in machine learning. In this paper, considering the promising performance of metaheuristics on different types of applications, six physics-inspired metaphor algorithms are employed for this problem. To evaluate the capability of dimensionality reduction in these algorithms, six diverse-natured datasets are used. The performance is compared in terms of the average number of features selected (AFS), accuracy, fitness, convergence capabilities, and computational cost. It is found through experiments that the accuracy and fitness of the Equilibrium Optimizer (EO) are comparatively better than the others. Finally, the average rank from the perspective of average fitness, average accuracy, and AFS shows that EO outperforms all other algorithms. [ABSTRACT FROM AUTHOR]

Feature selection is one of the main issues in machine learning algorithms. In this paper, a new binary hyper-heuristics feature ranks algorithm is designed to solve the feature selection problem in high-dimensional classification data called the BFRA algorithm. The initial strong population generation is done by ranking the features based on the initial Laplacian Score (ILR) method. A new operator called AHWF removes the zero-importance or redundant features from the population-based solutions. Another new operator, AHBF, selects the key features in population-based solutions. These two operators are designed to increase the exploitation of the BFRA algorithm. To ensure exploration, we introduced a new operator called BOM, a binary counter-mutation that increases the exploration and escape from the BFRA algorithm's local trap. Finally, the BFRA algorithm was evaluated on 26 high-dimensional data with different statistical criteria. The BFRA algorithm has been tested with various meta-heuristic algorithms. The experiments' different dimensions show that the BFRA algorithm works like a robust meta-heuristic algorithm in low dimensions. Nevertheless, by increasing the dataset dimensions, the BFRA performs better than other algorithms in terms of the best fitness function value, accuracy of the classifiers, and the number of selected features compared to different algorithms. However, a case study of sentiment analysis of movie viewers using BFRA proves that BFRA algorithms demonstrate affordable performance. [ABSTRACT FROM AUTHOR]

The scenario is very important to smartphone-based pedestrian positioning services. The smartphone is equipped with MEMS(Micro Electro Mechanical System) sensors, which have low accuracy. Now, the methods for scenario recognition are mainly machine-learning methods. The recognition rate of a single method is not high. Multi-model fusion can improve recognition accuracy, but it needs to collect many samples, the computational cost is high, and it is heavily dependent on feature selection. Therefore, we designed the DT-BP(decision tree-Bayesian probability) scenario recognition algorithm by introducing the Bayesian state transition model based on experience design in the decision tree. The decision-tree rules and state transition probability assignment methods were respectively designed for smartphone mode and motion mode. We carried out experiments for each scenario and compared them with the methods in the references. The results showed that the method proposed in this paper has a high recognition accuracy, which is equivalent to the accuracy of multi-model machine learning, but it is simpler, easier to implement, requires less computation, and requires fewer samples. [ABSTRACT FROM AUTHOR]

Chronic kidney disease is a global health issue that affects millions of people worldwide and causes significant social, economic, and medical issues. Several automated detection systems can diagnose chronic kidney disease. This paper proposes the recursive random forest feature selection (RFFS) based ensemble learning algorithm to diagnose chronic kidney diseases (CKD). In the decision point, decision tree‐based classifiers are used. The accuracy and kappa scores are used to determine the classification results. According to the results of the proposed algorithm's performance analyses, the ensemble learning classifiers outperform other classifiers for classifying CKD. The proposed RFFS algorithm achieves 6%–40% improved prediction accuracy using the feature selection algorithm. Further, it attains 15%–39% of reduced mean square error. The performance metrics, precision, sensitivity, specificity, f1 score, and Jaccard scores, have been analysed and show greater results for the RFFS algorithm. Thus, the proposed RFE‐GB algorithm results prove it as a prominent model for CKD estimation and treatment. [ABSTRACT FROM AUTHOR]

Simple Summary: Single Cell RNA Sequencing generates gene expression data at a single cell resolution. While single cell RNA has many applications in biomedical research, the high dimensionality of the data produced poses a considerable computational challenge. This study proposes a novel algorithm using penalized regression methods to analyze single cell RNA sequencing data. The proposed algorithm reduces high dimensionality of the gene expression data using a sequence feature selection methods such as Ridge regression, LASSO, Elastic Net, Drop LASSO, and Sparse Group LASSO. The proposed algorithm successfully detected highly differentiated genes, including the marker genes, for 5 different single cell RNA sequencing datasets associated with the species mouse, plant, and human. With the emergence of single-cell RNA sequencing (scRNA-seq) technology, scientists are able to examine gene expression at single-cell resolution. Analysis of scRNA-seq data has its own challenges, which stem from its high dimensionality. The method of machine learning comes with the potential of gene (feature) selection from the high-dimensional scRNA-seq data. Even though there exist multiple machine learning methods that appear to be suitable for feature selection, such as penalized regression, there is no rigorous comparison of their performances across data sets, where each poses its own challenges. Therefore, in this paper, we analyzed and compared multiple penalized regression methods for scRNA-seq data. Given the scRNA-seq data sets we analyzed, the results show that sparse group lasso (SGL) outperforms the other six methods (ridge, lasso, elastic net, drop lasso, group lasso, and big lasso) using the metrics area under the receiver operating curve (AUC) and computation time. Building on these findings, we proposed a new algorithm for feature selection using penalized regression methods. The proposed algorithm works by selecting a small subset of genes and applying SGL to select the differentially expressed genes in scRNA-seq data. By using hierarchical clustering to group genes, the proposed method bypasses the need for domain-specific knowledge for gene grouping information. In addition, the proposed algorithm provided consistently better AUC for the data sets used. [ABSTRACT FROM AUTHOR]

Background: Machine learning can be used to predict the different onset of human cancers. Highly dimensional data have enormous, complicated problems. One of these is an excessive number of genes plus over-fitting, fitting time, and classification accuracy. Recursive Feature Elimination (RFE) is a wrapper method for selecting the best subset of features that cause the best accuracy. Despite the high performance of RFE, time computation and over-fitting are two disadvantages of this algorithm. Random forest for selection (RFS) proves its effectiveness in selecting the effective features and improving the over-fitting problem. Method: This paper proposed a method, namely, positions first bootstrap step (PFBS) random forest selection recursive feature elimination (RFS-RFE) and its abbreviation is PFBS- RFS-RFE to enhance cancer classification performance. It used a bootstrap with many positions included in the outer first bootstrap step (OFBS), inner first bootstrap step (IFBS), and outer/ inner first bootstrap step (O/IFBS). In the first position, OFBS is applied as a resampling method (bootstrap) with replacement before selection step. The RFS is applied with bootstrap = false i.e., the whole datasets are used to build each tree. The importance features are hybrid with RFE to select the most relevant subset of features. In the second position, IFBS is applied as a resampling method (bootstrap) with replacement during applied RFS. The importance features are hybrid with RFE. In the third position, O/IFBS is applied as a hybrid of first and second positions. RFE used logistic regression (LR) as an estimator. The proposed methods are incorporated with four classifiers to solve the feature selection problems and modify the performance of RFE, in which five datasets with different size are used to assess the performance of the PFBS-RFS-RFE. Results: The results showed that the O/IFBS-RFS-RFE achieved the best performance compared with previous work and enhanced the accuracy, variance and ROC area for RNA gene and dermatology erythemato-squamous diseases datasets to become 99.994%, 0.0000004, 1.000 and 100.000%, 0.0 and 1.000, respectively. Conclusion: High dimensional datasets and RFE algorithm face many troubles in cancers classification performance. PFBS-RFS-RFE is proposed to fix these troubles with different positions. The importance features which extracted from RFS are used with RFE to obtain the effective features. [ABSTRACT FROM AUTHOR]

Chimp Optimization Algorithm (ChOA), a novel meta-heuristic algorithm, has been proposed in recent years. It divides the population into four different levels for the purpose of hunting. However, there are still some defects that lead to the algorithm falling into the local optimum. To overcome these defects, an Enhanced Chimp Optimization Algorithm (EChOA) is developed in this paper. Highly Disruptive Polynomial Mutation (HDPM) is introduced to further explore the population space and increase the population diversity. Then, the Spearman's rank correlation coefficient between the chimps with the highest fitness and the lowest fitness is calculated. In order to avoid the local optimization, the chimps with low fitness values are introduced with Beetle Antenna Search Algorithm (BAS) to obtain visual ability. Through the introduction of the above three strategies, the ability of population exploration and exploitation is enhanced. On this basis, this paper proposes an EChOA-SVM model, which can optimize parameters while selecting the features. Thus, the maximum classification accuracy can be achieved with as few features as possible. To verify the effectiveness of the proposed method, the proposed method is compared with seven common methods, including the original algorithm. Seventeen benchmark datasets from the UCI machine learning library are used to evaluate the accuracy, number of features, and fitness of these methods. Experimental results show that the classification accuracy of the proposed method is better than the other methods on most data sets, and the number of features required by the proposed method is also less than the other algorithms. [ABSTRACT FROM AUTHOR]

With the rise of the data amount being collected and exchanged over networks, the threat of cyber-attacks has also increased significantly. Timely and accurate detection of any intrusion activity in networks has become a crucial task. While manual moderation and programmed logic have been used for this purpose, the use of machine learning algorithms for superior pattern mapping is desired. The system logs in a network tend to include many parameters, and not all of them provide indications of an impending network threat. The selection of the right features is thus important for achieving better results. There is a need for accurate mapping of high dimension features to low dimension intermediate representations while retaining crucial information. In this paper, an approach for feature reduction and selection when working on network threat detection is proposed. This approach modifies the traditional Principal Component Analysis (PCA) algorithm by working on its shortcomings and by minimizing false detection rates. Specifically, work has been done upon the calculation of symmetric uncertainty and subsequent sorting of features. The performance of the proposed approach is evaluated on four standard-sized datasets that are collected using the Microsoft SYSMON real-time log collection tool. The proposed method is found to be better than the standard PCA and FAST methods for data reduction. The proposed approach makes a strong case as a dimensionality reduction and feature selection technique for reducing time complexity and minimizing false detection rates when operating on real-time data. [ABSTRACT FROM AUTHOR]

In this paper, some significant efforts on fuzzy oblique decision tree (FODT) have been done to improve classification accuracy and decrease tree size. Firstly, to eliminate data redundancy and improve classification efficiency, a forward greedy fast feature selection algorithm based on neighborhood rough set (NRS_FS_FAST) is introduced. Then, a new fuzzy rule generation algorithm (FRGA) is proposed to generate fuzzy rules. These fuzzy rules are used to construct leaf nodes for each class in each layer of the FODT. Different from the traditional axis-parallel decision trees and oblique decision trees, the FODT takes dynamic mining fuzzy rules as decision functions. Moreover, the parameter δ, which can control the size of the tree, is optimized by genetic algorithm. Finally, a series of comparative experiments are carried out with five traditional decision trees (C4.5, Best First Tree (BFT), amulti-class alternating decision tree (LAD), Simple Cart (SC), Naive Bayes Tree (NBT)), and recently proposed decision trees (FRDT, HHCART, and FMMDT-HB) on UCI machine learning datasets. The experimental results demonstrate that the FODT exhibits better performance on classification accuracy and tree size than the chosen benchmarks. [ABSTRACT FROM AUTHOR]

In this data-driven era, where a large number of attributes are often publicly available, redundancy becomes a major problem, which leads to large storage and computational resource requirement. Feature selection is a method for reducing the dimensionality of the data by removing such redundant or misleading attributes. This leads to a selection of optimal feature subsets that can be used for further computation like the classification of data. Learning algorithms, when fitted on such optimal subsets of reduced dimensions, perform more efficiently and storing data also becomes easier. However, there exists a trade-off between the number of features selected and the accuracy obtained and the requirement for different tasks may vary. Thus, in this paper, a hybrid filter multi-objective evolutionary algorithm (HFMOEA) has been proposed based on the nondominated sorting genetic algorithm (NSGA-II) coupled with filter-based feature ranking methods for population initialization to obtain an optimal trade-off solution set to the problem. The two competing objectives for the algorithm are the minimization of the number of selected features and the maximization of the classification accuracy. The filter ranking methods used for population initialization help in faster convergence of the NSGA-II algorithm to the PF. The proposed HFMOEA method has been evaluated on 18 UCI datasets and 2 deep feature sets (features extracted from image datasets using deep learning models) to justify the viability of the approach with respect to the state-of-the-art. [ABSTRACT FROM AUTHOR]

Feature selection can efficiently improve classification accuracy and reduce the dimension of datasets. However, feature selection is a challenging and complex task that requires a high-performance optimization algorithm. In this paper, we propose an enhanced binary bat algorithm (EBBA) which is originated from the conventional binary bat algorithm (BBA) as the learning algorithm in a wrapper-based feature selection model. First, we model the feature selection problem and then transfer it as a fitness function. Then, we propose an EBBA for solving the feature selection problem. In EBBA, we introduce the Lévy flight-based global search method, population diversity boosting method and chaos-based loudness method to improve the BA and make it more applicable to feature selection problems. Finally, the simulations are conducted to evaluate the proposed EBBA and the simulation results demonstrate that the proposed EBBA outmatches other comparison benchmarks. Moreover, we also illustrate the effectiveness of the proposed improved factors by tests. [ABSTRACT FROM AUTHOR]

Feature selection becomes prominent, especially in the data sets with many variables and features. It will eliminate unimportant variables and improve the accuracy as well as the performance of classification. Random Forest has emerged as a quite useful algorithm that can handle the feature selection issue even with a higher number of variables. In this paper, we use three popular datasets with a higher number of variables (Bank Marketing, Car Evaluation Database, Human Activity Recognition Using Smartphones) to conduct the experiment. There are four main reasons why feature selection is essential. First, to simplify the model by reducing the number of parameters, next to decrease the training time, to reduce overfilling by enhancing generalization, and to avoid the curse of dimensionality. Besides, we evaluate and compare each accuracy and performance of the classification model, such as Random Forest (RF), Support Vector Machines (SVM), K-Nearest Neighbors (KNN), and Linear Discriminant Analysis (LDA). The highest accuracy of the model is the best classifier. Practically, this paper adopts Random Forest to select the important feature in classification. Our experiments clearly show the comparative study of the RF algorithm from different perspectives. Furthermore, we compare the result of the dataset with and without essential features selection by RF methods varImp(), Boruta, and Recursive Feature Elimination (RFE) to get the best percentage accuracy and kappa. Experimental results demonstrate that Random Forest achieves a better performance in all experiment groups. [ABSTRACT FROM AUTHOR]