Search

A critical aspect of software engineering is Software fault prediction which aims to identify and prevent errors in software systems before their release which can cause failures or issues for its users. Various techniques and tools have been developed to detect software faults, including static code analysis, dynamic testing, and machine learning-based approaches. In past few years, the world has seen a growing interest in the use of ML models for predicting software faults, as they can effectively analyse high dimensional datasets and detect complex patterns which are difficult for human experts to detect. However, developing accurate and reliable software fault detection models requires careful selection of data, feature engineering, and model evaluation. This purpose of this paper is to present a comprehensive analysis of potential applications and future research directions in the field of software fault detection. The study emphasizes the importance of identifying and addressing software faults to ensure the reliability and efficiency of software systems. Additionally, the paper outlines various approaches and techniques that can be employed for effective software fault detection. [ABSTRACT FROM AUTHOR]

Feature Selection (FS) is critically important for optimising the performance of Intrusion Detection Systems (IDSs) used in Wireless Sensor Networks (WSNs). However, selecting an optimal number of relevant features from massive IDS data sets has been an ongoing FS research problem. Many approaches use Machine Learning (ML), such as nature-inspired population-based and Differential Evolution (DE)-based algorithms for FS. However, the main drawback of DE is that it has a premature convergence issue, which leads to false convergence. In this paper, a novel mutation strategy is proposed to alleviate premature convergence and to develop a modified DE-based FS algorithm called DE with maturity extension (DE-ME). The novel mutation is achieved by changing the mutation factor in the DE\rnd\1 based on K-Nearest Neighbour (KNN) as the most efficient implementation without adding further complexity to the design process. The proposed DE-ME algorithm improves the FS operation with significant performance improvements in terms of overcoming premature convergence and increasing accuracy for selection. The DE-ME algorithm was used to select the most effective features from 41 features in the Network Security Laboratory Knowledge Discovery in Databases (NSL-KDD) dataset. The accuracy of DE-ME is high at 99.66 %, its False Positive Rate (FPR) is low at 0.464 %, and the true positive rate (TPR) achieved is 99.80 %. In addition, the number of features selected by DE-ME is 6 out of 41 features in the dataset. [ABSTRACT FROM AUTHOR]

• A feature selection method based on binary dandelion algorithm is proposed. • The algorithm applies a sine-cosine operator and a restart strategy. • Mutual information and quick bit mutation improve the performance of the algorithm. • The algorithm performs well on datasets with larger dimensions. Feature selection (FS) is an important data preprocessing technology for machine learning and data mining. Metaheuristic algorithm (MH) has been widely used in feature selection because of its powerful search function. This paper presents an improved Binary Dandelion Algorithm using Sine Cosine operator and Restart strategy (SCRBDA) for feature selection. First, the sine cosine operator is used in the radius formula of the core dandelions (CD), which significantly enhances the ability of algorithm development and exploration. Secondly, the algorithm uses a restart strategy to increase its ability to get rid of local optimum. Thirdly, mutual information is used to guide the generation of some dandelions, which pays more attention to the correlation between the selected features and categories. Finally, quick bit mutation is used as the mutation strategy to improve the diversity of the population. The SCRBDA proposed in this paper was tested on 18 datasets of different sizes from UCI machine learning database. The SCRBDA was compared with 8 other classical feature selection algorithms, and the performance of the proposed algorithm was evaluated through feature subset size, classification accuracy, fitness value, and F1-score. The experimental results show that SCRBDA achieves the best performance, which has stronger feature reduction ability and achieves better overall performance on most datasets. Especially on large-scale datasets, SCRBDA can obtain extremely smaller feature subsets while maintaining much higher classification accuracy, and satisfactory F1-score. [ABSTRACT FROM AUTHOR]

• MMODE is developed as a new hybrid gene selection method for tumor identification. • MMODE combines multi-tasking and multi-objective frameworks. • MMODE uses a new elite strategy and a new guidance strategy. • MMODE selects a few genes and achieves high classification accuracy. A key preprocessing step in tumor recognition based on microarray expression profile data and machine learning is to identify tumor marker genes. Gene selection aims to select the most relevant gene subset from the original ultra-high dimensional microarray expression profile data to improve tumor identification performance. Inspired by evolutionary multitasking (EMT) and multi-objective optimization, this paper puts forward a novel multitasking multi-objective differential evolution gene selection algorithm (MMODE) which uses new elite and guidance strategies to select the best gene subsets. MMODE initializes two different populations according to different filtering criteria to increase the diversity of the search. These two populations guide their respective populations to search in the optimal direction through knowledge transfer in the evolutionary process. In addition, MMODE employs new elite and guidance strategies that enables individuals to narrow the search range and jump out of local optima. The proposed algorithm is validated on 13 publicly available microarray expression datasets in comparison with state-of-the-art gene selection algorithms. The experimental results show that MMODE can find smaller gene subsets and achieve higher classification accuracy compared with other algorithms. [ABSTRACT FROM AUTHOR]

Attribute reduction can remove data noise and redundancy, thus reducing computational complexity, which is very important for machine learning. Because the difference between nominal attribute values is difficult to measure, attribute reduction for hybrid data faces challenges. In addition, most of the existing methods are sensitive to noise due to the lack of an anti-noise mechanism. Decision attribute contains the most important information of data. This paper proposes some techniques that consider the above problems from the perspective of fuzzy evidence theory. First of all, a new distance incorporating decision attributes is defined, and then a new fuzzy relation with an anti-noise mechanism is defined. Furthermore, fuzzy belief and fuzzy plausibility are defined based on the defined new distance and new fuzzy relation. In this framework, two anti-noise attribute reduction algorithms for hybrid data are proposed. Experiments on 12 data sets of various types show that compared with the other 8 state-of-the-art algorithms, the proposed algorithms improve the classification accuracy by at least 2% and the anti-noise ability by at least 11%. Therefore, it can be concluded that the proposed algorithms have excellent anti-noise ability while maintaining good feature selection ability. [ABSTRACT FROM AUTHOR]

Feature selection (FS) is considered a challenging machine learning problem that handles the large size of features. The main purpose of FS is to remove irrelevant and redundant variables to improve the performance of the learning algorithms. Consequently, the FS process is considered as an optimization problem where metaheuristics approaches prove efficiency in solving it. In this paper, we propose new binary versions of a new ancient inspired metaheuristic approach called Giza Pyramids Construction (GPC) to select the most relevant subset of features. The proposed binary versions of the algorithm called BGPC-S and BGPC-V are implemented with two transfer functions, with the main objective of maximizing classification accuracy and minimizing the number of selected features. The two versions of BGPC were compared to six well-known binary metaheuristics for feature selection problem, namely Binary Atom Search Optimisation (BASO), Binary Bat Algorithm (BBA), Binary Differential Evolution (BDE), Binary Grey Wolf Optimizer (BGWO), Binary Particle Swarm Optimization (BPSO), and Binary Harris Hawks Optimizer (BHHO), and evaluated over 20 datasets from the UCI repository. Experiments have demonstrated that the proposed approaches outperformed the other algorithms in terms of classification accuracy and the number of selected features. [ABSTRACT FROM AUTHOR]

Feature selection (FS) is an important pre-processing step in data mining and pattern recognition. It can effectively compress the dimensionality of the feature space to reduce computation time and improve classification performance. The meta-heuristic algorithm-based feature selection method by finding the optimal set of features in the solution space has been widely used. However, this method is prone to trap into local optimality in a sufficiently large solution space. In this paper, we first propose a binary dandelion algorithm (BDA) to improve classification accuracy. In addition, to improve the performance of the algorithm, a binary dandelion algorithm using an improved seeding strategy and chaotic populations (SBDA) is proposed in this paper. Firstly, the strategy of optimizing the seeding radius by using the vibrational function and the historical optimal population increases the complexity of the search process and improves the search performance of the algorithm in the solution space. Secondly, when generating seeds, chaotic populations are generated using chaotic operators, which improves the ability of the algorithm to jump out of the local optimum and improves the stability of the algorithm. In this paper, 15 well-established datasets collected from the UCI machine learning database were adopted to compare four variants of BDA using only chaotic population improvement and in the next experiments, both mechanisms are verified to be effective in improving the performance of the algorithm. In addition, this paper compares the proposed BDA algorithm and SBDA algorithm with eight other classical algorithms. The experimental results show that SBDA can obtain fewer features with higher classification accuracy in most cases. • Proposed a BDA algorithm for solving binary problems. • Proposed a SBDA algorithm by incorporating chaos mapping. • Presented an improved seeding strategy in relation to the historical optimal population. • Studied and applied the feature selection problem with extreme learning machine. [ABSTRACT FROM AUTHOR]

Age estimation has become inordinately significant for human beings for many reasons, such as detecting legal and criminal responsibility and other social events like a marriage license, birth certificate, etc. This paper aims to decide on the most desirable machine learning algorithm (from conventional machine learning algorithms to deep learning) for dental age estimation based on buccal bone level. The database consisted of 150 CBCT images (73 males and 77 females) from an existing base of the Faculty of Dental Medicine with Clinics, University of Sarajevo, aged 20-69. Results were obtained using the Waikato Environment for Knowledge Analysis (Weka), machine learning software in Java. Left and Right Buccal Alveolar Bone Levels are increasing with age, so they showed to be the most important attributes, especially the latter. Random Forest classifier provided the greatest result with the correlation coefficient of 0.803 and the mean absolute error of 6.022. We have also shown that considering sinus-related features can be a significant addition to the databases. Our paper is probably one of the first studies where regression algorithms based on the Support Vector Machines and Random Forest were utilized. [ABSTRACT FROM AUTHOR]

• 3 layer neural networks outperform other ML algorithms in automated assessments. • In subjective assignments, deep neural networks can improve score prediction. • Word embeddings and linguistic feature results in more precise text mining models. • Extraction and selection of appropriate features can improve results. Machine learning (ML) has recently gained popularity in a variety of domains for automating tasks. This has also been used in the evaluation of student responses/ assignments. This paper presents a qualitatively enhanced methodology for automated score prediction of subjective assignments. To consider all the major aspects of a human grader, 21 linguistic features related to syntactical, grammatical, sentimental, and readability are analyzed from the assignments. This study makes use of the ASAP competition dataset from Kaggle. The evaluation metric used is Quadratic Weighted Kappa (QWK), which measures the agreement between the human graded score and the predicted score. The effect of appropriate feature selection has been observed using Mutual Information Regression. Four ML algorithms are investigated on identified linguistic features. 3 Layer Neural Network with feature selection performed well among all chosen ML algorithms with average QWK of 0.678. To include the benefits of deep learning, a new hybrid model (LF-BiLSTM-att-FS) is proposed that combines a higher level deep neural network (DNN) with the selected features. Pre-trained Glove embedding is used to include contextual information of the text. The proposed model results demonstrated an improvement in overall accuracy, with an average QWK value of 0.768. [ABSTRACT FROM AUTHOR]

Reduced computational time and cost, reduced skilled professional resources, and diagnostic accuracy have made medical diagnosis using computer aided systems (CAD) increasingly popular and can be comfortably employed in the diagnosis of many acute and chronic diseases in ophthalmology, cardiology, cancer detection, etc. There seems to be a growing necessity for the computational algorithms to be robust enough, to identify the abnormality in each of the cases, aiding in early diagnosis. In this paper, a glaucoma diagnostic approach based on the wrapper method employing bio-inspired algorithms, and a Kernel-Extreme Learning Machine (KELM) classifier is proposed. The bio-inspired algorithms are deployed to select feature sub-sets, generating three feature sub-sets from the pre-processed fundus images by adopting a correlation-based feature selection (CFS) approach. The selected features are utilized to train the salp-swarm optimization based KELM, which finds the optimal parameters of the KELM classifier network. The proposed methodology is evaluated on the public and private retinal fundus datasets containing 7280 images. The experimental outcome revealed that the system is able to attain a maximum overall accuracy of 99.61% with 99.89% sensitivity and 100% specificity. A 5-fold cross validation showed 98.78% accuracy ensuring a bias-free classification. Further, by experimenting on degraded images (Gaussian, salt-pepper noise images) of the original dataset, the model achieved extreme robustness with 99.3% accuracy. The proposed method is compared with other similar methods, which showed the efficiency of our method. The framework proposed can aid in making clinical decisions for various pathologies like lung infection, diabetic retinopathy, etc. • A predictive framework involving bio-inspired algorithms and Kernel-ELM is presented. • Feature Selection based on Correlation (CFS) is employed to find the optimal feature set. • KELM parameters are optimized by bio-inspired evolutionary algorithm, salp-swarm optimization algorithm (SSA) for improved diagnosis. • The proposed method works well on degraded (gaussian and salt-pepper noise) images showing the robustness of the method.. • Superior performance with statistically significant improvements on varied datasets compared to competitive algorithms. [ABSTRACT FROM AUTHOR]

Projected gradient descent has been proved efficient in many optimization and machine learning problems. The weighted ℓ 1 ball has been shown effective in sparse system identification and features selection. In this paper we propose three new efficient algorithms for projecting any vector of finite length onto the weighted ℓ 1 ball. The first two algorithms have a linear worst case complexity. The third one has a highly competitive performances in practice but the worst case has a quadratic complexity. These new algorithms are efficient tools for machine learning methods based on projected gradient descent such as compressed sensing, feature selection. We illustrate this effectiveness by adapting an efficient compressed sensing algorithm to weighted projections. We demonstrate the efficiency of our new algorithms on benchmarks using very large vectors. For instance, it requires only 8 ms, on an Intel I7 3rd generation, for projecting vectors of size 107. [ABSTRACT FROM AUTHOR]

Feature selection (FS) is recognized as one of the majority public and challenging problems in the Machine Learning domain. FS can be examined as an optimization problem that needs an effective optimizer to determine its optimal subset of more informative features. This paper proposes a wrapper FS method that combines chaotic maps (CMs) and binary Group Search Optimizer (GSO) called CGSO, which is used to solve the FS problem. In this method, five chaotic maps are incorporated with the GSO algorithm's main procedures, namely, Logistic, Piecewise, Singer, Sinusoidal, and Tent. The GSO algorithm is used as a search strategy, while k-NN is employed as an induction algorithm. The objective function is to integrate three main objectives: maximizing the classification accuracy value, minimizing the number of selected features, and minimizing the complexity of generated k-NN models. To evaluate the proposed methods' performance, twenty well-known UCI datasets are used and compared with other well-known published methods in the literature. The obtained results reveal the superiority of the proposed methods in outperforming other well-known methods, especially when using binary GSO with Tent CM. Finally, it is a beneficial method to be utilized in systems that require FS pre-processing. • A novel feature selection method is proposed for classification task. • Five Chaotic Group Search Optimizers are investigated. • Twenty UCI datasets were used in the experiments. • Our methods got a higher accuracy results using a fewer number of features. [ABSTRACT FROM AUTHOR]

Sine Cosine Algorithm (SCA) that is one of the population-based metaheuristic optimization algorithms basically consists of the updating mechanism based on sine and cosine functions. In this algorithm, a few random and adaptive variables are also utilized for more effective motions of the candidate solutions. SCA has some drawbacks like other some metaheuristic algorithms. SCA tends to be stuck into the local regions in the search space and this affects negatively on the computational effort required to find the best solution point in the search space. This paper presents four different improved versions of SCA. The proposed improvements on original SCA are the innovations on the updating mechanism of SCA. To evaluate the performances of Improved Sine Cosine Algorithms (ImpSCAs), well-known numerical optimization problems including CEC 2014 test suite are used. Firstly, different analyses of the proposed ImpSCAs are dealt with such as the convergence analysis, search history analysis, trajectory analysis, average distance analysis, and computational complexity analysis. Secondly, the proposed four versions of ImpSCAs are compared with the original SCA for CEC 2014 benchmark problems with dimension sizes of 10D, 30D and 50D. Finally, original SCA and ImpSCAs are adapted to select optimal feature combination and they are tested for 10 feature selection datasets taken from the UCI machine learning repository. The benchmark results show that the performances of the ImpSCA 1 , ImpSCA 2 , and ImpSCA 4 are better than that of the original SCA. From the feature selection results, it is observed that three versions of ImpSCAs (except ImpSCA 3) outperform the original SCA in 80% of the datasets. Source codes of ImpSCAs are publicly available at https://github.com/uguryuzgec/ImpSCAs. • We focused to improve the exploration and exploitation ability of the original SCA. • The basic idea is based on the innovations on the update mechanism. • The ISCAs were evaluated using CEC2014 benchmark test suite. • The proposed ISCAs were tested for 10 popular UCI datasets. • Three versions of ISCAs (except ISCA 3) outperform the SCA in 80% of the datasets. [ABSTRACT FROM AUTHOR]

• FCS is proposed as a complementary step for principal component analysis. • PCA-FCS identifies the most relevant components based on their level of contribution. • PCA-FCS simultaneously performs dimension reduction and component selection. • PCA-FCS has substantially improved the accuracy of the random forest algorithm. Principal component analysis (PCA) is one the most effective and widely used dimensionality-reduction techniques which aggregates the maximal variance in the first few components. In many applications, the first few components of PCA are used in place of the original variables in statistical and machine learning models; nevertheless, such use does not guarantee the selection of the most relevant components to the target variable. This paper presents an efficient approach in which all components, rather than simply the first few, are considered as input for the random forest (RF) model, and a feature selection algorithm is integrated with the RF to select the most relevant components, hence called fully component selection (FCS). The proposed method was evaluated on spectroscopic data comprising 70 soil samples with 2050 spectral bands to estimate soil organic carbon (SOC). The FCS approach was compared to the RF model once considering the original features, then again using only the first few components, and again using all components without feature selection. The results showed that RF-FCS substantially outperformed other approaches, such that the R2 was increased between 25.2% and 55.5%. Furthermore, the findings indicated that the most relevant principal components (PCs) to the target variable were not the first few, but PC6, PC7, PC8, PC15, and PC43. Using the FCS approach increased model performance substantively, and coupling it with machine learning and statistical models is strongly recommended for high dimensional data applications. [ABSTRACT FROM AUTHOR]

Feature selection in real-world data mining problems is essential to make the learning task efficient and more accurate. Identifying the best feature selection algorithm, among the many available, is a complex activity that still relies heavily on human experts or some random trial-and-error procedure. Thus, the automated machine learning community has taken some steps towards the automation of this process. In this paper, we address the metalearning challenge of recommending feature selection algorithms by proposing a novel meta-feature engineering model. Our model considers a broad collection of meta-features that enable the study of the relationship between the dataset properties and the feature selection algorithm performance in terms of several criteria. We arrange the input meta-features into eight categories: (i) simple, (ii) statistical, (iii) information-theoretical, (iv) complexity, (v) landmarking, (vi) based on symbolic models, (vii) based on images, and (viii) based on complex networks (graphs). The target meta-features emerge from a multi-criteria performance measure, based on five individual performance indexes, that assesses feature selection methods grounded in information, distance, dependence, consistency, and precision measures. We evaluate our proposal using a recently developed framework that extracts the input meta-features from 213 benchmark datasets, and ranks the assessed feature selection algorithms, to fill in the target meta-features in meta-bases. This evaluation uses five state-of-the-art classification methods to induce recommendation models from meta-bases: C4.5, Random Forest, XGBoost, ANN, and SVM. The results showed that it is possible to reach an average accuracy of up to 90% applying our meta-feature engineering model. This work is the first to use an extensive empirical evaluation to provide a careful discussion of the strengths and limitations of more than 160 meta-features. These meta-features, while designed to aid the task of feature selection algorithm recommendation, can readily be employed in other metalearning scenarios. Therefore, we believe our findings are a valuable contribution to the fields of automated machine learning and data mining, as well as to the feature extraction and pattern recognition communities. • A novel meta-feature engineering model recommends feature selection algorithms. • The proposal obtains promising results from 213 datasets with hit rates of up to 90%. • Some simple, landmarking, image, and graph-based input meta-features highlighted. • A multi-criteria performance measure rigorously assesses candidate algorithms. • Chains of binary or multiclass classifiers can efficiently rank candidate algorithms. [ABSTRACT FROM AUTHOR]

Feature selection is considered as a necessary and significant pre-processing step in many fields, especially in machine learning. However, in some real problems, in which features flow one by one, many existing approaches do not work well on the online streaming features, and most online streaming feature selection (OSFS) methods face the challenge of requiring domain knowledge before setting optimal parameters in advance. Therefore, an effective feature selection method for online streaming features, named OFS-Gapknn, is proposed in this paper. A new neighborhood rough set relation is firstly defined, which combines the advantages of both the k -nearest and the Gap neighborhood. The proposed neighborhood relation can not only work well on the unevenly distributed sample space, but also need not any parameters and domain knowledge. Then, the relevance and redundancy features are analyzed by using the dependency based on the neighborhood rough set. Finally, one of the optimal feature subsets is obtained. To validate the effectiveness of the proposed algorithm, four traditional methods and three OSFS methods are compared with it on 11 datasets. Experimental results indicate the dominance and significance of the proposed method. • Need not any prior information about the feature space in advance. • Parameters are automatically computed. [ABSTRACT FROM AUTHOR]

• A machine learning network traffic analysis system has been designed. • Experiments of our proposed machine learning classifiers on a real-world P2P network traffic classification problem. • P2P network traffic classification using various machine learning techniques. • Comparison Performance of all Ensemble and Single models was performed using VPN netoworks. Network traffic classification is a significant and problematic aspect of network resource management arising from an investigation of network developments, planning, and design for 5G and beyond. Recently, traffic investigation systems for network monitoring and user access restrictions to Virtual Private Networks (VPN) and non-Virtual Private Networks (non-VPN) have gained widespread attention. In this paper, different algorithms for classifying and detecting VPN traffic are considered. A few existing machine learning procedures were tested concerning their performance in network traffic classification and security. The purpose is to improve Precision, Recall, and F1-score in VPN Network Traffic using Ensemble Classifiers. Therefore, the parameters of the ensemble classifier were changed to obtain high Precision, Recall, and F1-score. Bagging Decision Tree and Gradient Boosting algorithms were used for classification which produced promising results when compared to single classifiers like k -Nearest Neighbors (kNN), Multilayer Perceptron (MLP), and Decision Tree. The proposed classifier demonstrates recognition accuracy on a test sample of up to 93.80 % which outperforms all other single algorithms used in previous work. The MLP, Random Forest (RF), and Gradient Boosting (GB) algorithms had almost identical performance in all experiments. Furthermore, the proposed classifiers are found to perform better when the network traffic flows are generated using different values of time parameters (timeout). Our results show that the ensemble algorithms (Random Forest and the Gradient Boosting) outperform the single machine learning classifier previously used by other researchers, and we achieved the highest accuracy with the random forest classifier with better results while using non-VPN traffic and VPN traffic. The novelty lies in the application of an ensemble algorithm to secure a network traffic classification performed in comparison with single classifiers to determine Accuracy, Precision, and F1-score of a given dataset, contrary to the known process of selection of features and generation. [ABSTRACT FROM AUTHOR]