Search

This paper describes an approach for credit risk evaluation based on linear Support Vector Machines classifiers, combined with external evaluation and sliding window testing, with focus on application on larger datasets. It presents a technique for optimal linear SVM classifier selection based on particle swarm optimization technique, providing significant amount of focus on imbalanced learning issue. It is compared to other classifiers in terms of accuracy and identification of each class. Experimental classification performance results, obtained using real world financial dataset from SEC EDGAR database, lead to conclusion that proposed technique is capable to produce results, comparable to other classifiers, such as logistic regression and RBF network, and thus be can be an appealing option for future development of real credit risk evaluation models. [ABSTRACT FROM AUTHOR]

The production of a motion picture is an expensive, risky endeavor. During the five-year period from 2008 through 2012, approximately 90 films were released in the United States with production budgets in excess of $100 million. The majority of these films failed to recoup their production costs via gross domestic box office revenues. Existing decision support systems for pre-production analysis and green-lighting decisions lack sufficient accuracy to meaningfully assist decision makers in the film industry. Established models focus primarily upon post-release and post-production forecasts. These models often rely upon opening weekend data and are reasonably accurate but only if data up until the moment of release is included. A forecast made immediately prior to the debut of a film, however, is of limited value to stakeholders because it can only influence late-stage adjustments to advertising or distribution strategies and little else. In this paper we present the development of a model based upon a dynamic artificial neural network (DAN2) for the forecasting of movie revenues during the pre-production period. We first demonstrate the effectiveness of DAN2 and show that DAN2 improves box-office revenue forecasting accuracy by 32.8% over existing models. Subsequently, we offer an alternative modeling strategy by adding production budgets, pre-release advertising expenditures, runtime, and seasonality to the predictive variables. This alternative model produces excellent forecasting accuracy values of 94.1%. [ABSTRACT FROM AUTHOR]

In this paper, we present a fraud detection framework for publicly traded firms using an optimization approach integrated with a meta-heuristic algorithm known as Beetle Antennae Search (BAS). Existing techniques include human resources, like financial experts and audit teams, to determine the ambiguities or financial frauds in the companies based on financial and non-financial ratios. It is a laborious task, time-consuming, and prone to errors. We designed an optimization problem to minimize the loss function based on a non-linear decision function combined with the maximization of recall (Sensitivity and Specificity). We solved the optimization problem iteratively using the BAS. It is a nature-inspired algorithm and mimics the beetle's food-searching nature. It includes a single searching particle to find an optimal solution to the optimization problem in n -dimensional space. We used a benchmark dataset collected from SEC's Accounting and Auditing Enforcement Releases (AAERs) for the simulation. It includes 28 raw financial variables and the data collected between 1991–2008. For the comparison, we evaluated the performance of BAS with the recently proposed approach using RUSBoost. We also compared it with some additional algorithms, i.e., Logit and SVM-FK. The results showed that BAS is comparable with these algorithms and outperformed them in time consumption. • Fraud detection in US-Public firms using BAS algorithm. • BAS is computationally and time-wise efficient. • Unbalanced Public firms' dataset is tackled with SMOTE and BAS. • Achieved state-of-the-art accuracy in comparison with other algorithms. • Time efficiency of BAS outperformed other algorithms. [ABSTRACT FROM AUTHOR]

• Deep learning algorithm for identification of type and flight mode of detected drones. • Novel deep learning algorithm for detection of multiple drones in radio frequency domain. • Results of three indoor experiments concerning collecting and analyzing radio signals from drones. • Public available database with radio signals of drones from two different radio frequency bands. Unmanned aerial systems, especially drones have gone through remarkable improvement and expansion in recent years. Drones have been widely utilized in many applications and scenarios, due to their low price and ease of use. However, in some applications drones can pose a malicious threat. To diminish risks to public security and personal privacy, it is necessary to deploy an effective and affordable anti-drone system in sensitive areas to detect, localize, identify, and defend against intruding malicious drones. This research article presents a new publicly available radio frequency drone dataset and investigates detection and identification methodologies to detect single or multiple drones and identify a single detected drone's type. Moreover, special attention in this paper has been underlined to examine the possibility of using deep learning algorithms, particularly fully connected deep neural networks as an anti-drone solution within two different radio frequency bands. We proposed a supervised deep learning algorithm with fully-connected deep neural network models that use raw drone signals rather than features. Regarding the research results, the proposed algorithm shows a lot of potentials. The probability of detecting a single drone is 99.8%, and the probability of type identification is 96.1%. Moreover, the results of multiple drones detection demonstrate an average accuracy of 97.3%. There have not been such comprehensive publications, to this time, in the open literature that have presented and enlightened the problem of multiple drones detection in the radio frequency domain. [ABSTRACT FROM AUTHOR]

In this paper, we propose a binary classifier ensemble-based one-class classifier (BCE-OC) for one-class classification. Given a training set comprising of only target class instances, it is partitioned into several clusters. Multiple binary classifiers are then trained with the clusters in a one-against-rest fashion, in which each classifier treats one cluster as a pseudo non-target class and is responsible for distinguishing the cluster from the other clusters. The binary classifiers are finally combined to constitute a one-class classifier, which is used to classify unknown instances. BCE-OC allows the use of any supervised classification algorithms for one-class classification. Accordingly, it allows extensive comparison of various learning algorithms to obtain a more competent one-class classifier for the problem. The effectiveness of BCE-OC is demonstrated through experimental validation using benchmark datasets. • A binary classifier ensemble-based one-class classifier is proposed. • The training set of the target class is partitioned into several clusters. • Multiple binary classifiers are built on the clusters in a one-against-rest fashion. • Any supervised classification algorithm can be used for the binary classifiers. • A one-class classifier is constructed by combining the binary classifiers. [ABSTRACT FROM AUTHOR]

Hierarchical multi-label text classification (HMTC) has been gaining popularity in recent years thanks to its applicability to a plethora of real-world applications. The existing HMTC algorithms largely focus on the design of classifiers, such as the local, global, or a combination of them. However, very few studies have focused on hierarchical feature extraction and explore the association between the hierarchical labels and the text. In this paper, we propose a L abel-based A ttention for H ierarchical Multi-label Text C lassification Neural N etwork (LA-HCN), where the novel label-based attention module is designed to hierarchically extract important information from the text based on the labels from different hierarchy levels. Besides, hierarchical information is shared across levels while preserving the hierarchical label-based information. Separate local and global document embeddings are obtained and used to facilitate the respective local and global classifications. In our experiments, LA-HCN outperforms other state-of-the-art neural network-based HMTC algorithms on four public HMTC datasets. The ablation study also demonstrates the effectiveness of the proposed label-based attention module as well as the novel local and global embeddings and classifications. By visualizing the learned attention (words), we find that LA-HCN is able to extract meaningful information corresponding to the different labels which provides explainability that may be helpful for the human analyst. • Proposed label-based attention can improve performance and provide interpretability. • Information can be shared across hierarchical layers while avoiding error propagation. • Respective document embedding is generated for local and global classification to prevent error propagation. [ABSTRACT FROM AUTHOR]

• Discretization is an important process in terms of both machine learning and data mining. • The effect of discretization on ensemble methods was analyzed. • Four Chi-square based discretization algorithms were used. • The Ensemble methods performed better on discrete data sets. Classification is one of the important tasks in data mining and machine learning. Classification performance depends on many factors as well as data characteristics. Some algorithms are known to work better with discrete data. In contrast, most real-world data contain continuous variables. For algorithms working with discrete data, these continuous variables must be converted to discrete ones. In this process called discretization, continuous variables are converted to their corresponding discrete variables. In this paper, four Chi-square based supervised discretization algorithms ChiMerge(ChiM), Chi2, Extended Chi2(ExtChi2) and Modified Chi2(ModChi2) were used. In the literature, the performance of these algorithms is often tested with decision trees and Naïve Bayes classifiers. In this study, differently, four sets of data discretized by these algorithms were classified with ensemble methods. Classification accuracies for these data sets were obtained through using a stratified 10-fold cross-validation method. The classification performance of the original and discrete data sets of the methods is presented comparatively. According to the results, the performance of the discrete data is more successful than the original data. [ABSTRACT FROM AUTHOR]

• Some ELMs are used in parallel as experts to map data into discriminative space. • The training data are easily grouped into some clusters in the new feature space. • An index is calculated per group to help finding corresponding groups of a sample. • A tree is used to find corresponding groups of a sample and applying kNN on it. • The method is evaluated on 4 big datasets in absence and presence of noise. kNN algorithm, as an effective data mining technique, is always attended for supervised classification. On the other hand, the previously proposed kNN finding methods cannot be considered as efficient methods for dealing with big data. As there is daily generated and expanded big datasets on different online and offline servers, the efficient methods for such data must be introduced to find kNN. Moreover, massive amounts of data contain more noise and imperfection data samples that significantly increase the need for a robust kNN finding method. In this paper, a new fast and robust kNN finding framework is introduced to deal with the big datasets. In this method, a group of most relevant data samples to an input data sample are detected and the original kNN method is applied on them for finding the final nearest neighbors. The main goal of this method is dealing with the big datasets in an accurate, fast, and robust manner. Here, the training data samples of each label are grouped into some partitions based on the output of some mini-classifiers (i.e. ELM classifier). In fact, the behavior of the mini-classifiers is the basis of partitioning the training data samples. These mini-classifiers are trained using non-overlapping subsets of the training set in the form of each mini-classifier a subset. Here, an index is calculated for each partition to make the corresponding partition finding faster using a tree structure in which each partition index is fallen into a leaf. Then, outputs of the mini-classifiers for an input test sample are used to find the corresponding group of most relevant data samples to the input data sample on the tree. Experimental results indicate that the proposed method has better performance in most cases and comparable performance on other cases of original and noisy big data problems. [ABSTRACT FROM AUTHOR]

In this paper, we develop topological data analysis methods for classification tasks on univariate time series. As an application, we perform binary and ternary classification tasks on two public datasets that consist of physiological signals collected under stress and non-stress conditions. We accomplish our goal by using persistent homology to engineer stable topological features after we use a time delay embedding of the signals and perform a subwindowing instead of using windows of fixed length. The combination of methods we use can be applied to any univariate time series and allows us to reduce noise and use long window sizes without incurring an extra computational cost. We then use machine learning models on the features we algorithmically engineered to obtain higher accuracies with fewer features. [ABSTRACT FROM AUTHOR]

• Improved oversampling algorithm (G-SOMO) to address the imbalance-learning problem. • It uses a Self-Organizing Map to identify optimal areas to generate artificial data. • The Geometric-SMOTE algorithm is used to generate the artificial instances. • G-SOMO is tested in 69 datasets and consistently outperforms the benchmarks. Traditional supervised machine learning classifiers are challenged to learn highly skewed data distributions as they are designed to expect classes to equally contribute to the minimization of the classifiers cost function. Moreover, the classifiers design expects equal misclassification costs, causing a bias for overrepresented classes. Different strategies have been proposed to correct this issue. The modification of the data set has become a common practice since the procedure is generalizable to all classifiers. Various algorithms to rebalance the data distribution through the creation of synthetic instances were proposed in the past. In this paper, we propose a new oversampling algorithm named G-SOMO. The algorithm identifies optimal areas to create artificial data instances in an informed manner and utilizes a geometric region during the data generation process to increase their variability. Our empirical results on 69 datasets, validated with different classifiers and metrics against a benchmark of commonly used oversampling methods show that G-SOMO consistently outperforms competing oversampling methods. Additionally, the statistical significance of our results is established. [ABSTRACT FROM AUTHOR]

• We identify foreground, background, and hybrid deep features for scene images. • We aggregate all three types of deep features to represent the scene images. • We evaluate our proposed method using two commonly used benchmark datasets. Previous methods for representing scene images based on deep learning primarily consider either the foreground or background information as the discriminating clues for the classification task. However, scene images also require additional information (hybrid) to cope with the inter-class similarity and intra-class variation problems. In this paper, we propose to use hybrid features in addition to foreground and background features to represent scene images. We suppose that these three types of information could jointly help to represent scene images more accurately. To this end, we adopt three VGG-16 architectures pre-trained on ImageNet, Places, and Hybrid (both ImageNet and Places) datasets for the corresponding extraction of foreground, background and hybrid information. All these three types of deep features are further aggregated to achieve our final features for the representation of scene images. Extensive experiments on two large benchmark scene datasets (MIT-67 and SUN-397) show that our method produces the state-of-the-art classification performance. [ABSTRACT FROM AUTHOR]

• Thorough preprocessing steps to cope with the limited on board resources. • Classifier selection for deployment must evaluate different stability matrices. • Pipeline validation using real engine data from a dedicated test bench environment. • Data driven predictive maintenance offers satisfying performance for prognostic. • Mismatch matrix as novel visual representation of classification results. Nowadays, cars are instrumented with thousands of sensors continuously collecting data about its components. Thanks to the concept of connected cars, this data can be now transferred to the cloud for advanced analytics functionalities, such as prognostic or predictive maintenance. In this paper, we dissect a data-driven prognostic pipeline and apply it in the automotive scenario. Our pipeline is composed of three main steps: (i) selection of most important signals and features describing the scenario for the target problem, (ii) creation of machine learning models based on different classification algorithms, and (iii) selection of the model that works better for a deployment scenario. For the development of the pipeline, we exploit an extensive experimental campaign where an actual engine runs in a controlled test bench under different working conditions. We aim to predict failures of the High-Pressure Fuel System, a key part of the diesel engine responsible for delivering high-pressure fuel to the cylinders for combustion. Our results show the advantage of data-driven solutions to automatically discover the most important signals to predict failures of the High-Pressure Fuel System. We also highlight how an accurate model selection step is fundamental to identify a robust model suitable for deployment. [ABSTRACT FROM AUTHOR]

[Display omitted] • Two novel DR methods are proposed to extract critical discriminative information. • Neighborhood size parameters of DR models are decided adaptively via data similarity. • Outside data can be projected directly through SD-IsoP that prevents over-fitting. • Margins between classes in reduced space can be maximized by proposed methods. As one of the most popular nonlinear dimensionality reduction methods, Isomap has been widely used in pattern recognition and machine learning. However, Isomap has the following problems: (1) Isomap is an unsupervised dimensionality reduction method, it cannot use class label information to obtain discriminative low dimensional embedding for classification; (2) The embedding performance of Isomap is sensitive to neighborhood size parameter; (3) Isomap cannot deal with outside new data by direct embedding. In this paper, a novel dimensionality reduction method called supervised discriminant Isomap is proposed to solve the first two problems mentioned above. Specifically, first, raw data points are partitioned into different manifolds by using their class label information. Then, supervised discriminant Isomap aims at seeking an optimal nonlinear subspace to preserve the geometrical structure of each manifold according to the Isomap criterion, and to enhance the discriminating capability by maximizing the distances between data points of different classes and the maximum margin graph regularization term. Finally, the corresponding optimization problems are solved by using eigen-decomposition algorithm. Further, we extend supervised discriminant Isomap to a linear dimensionality reduction method called supervised discriminant Isomap projection for handling the above three problems. Moreover, our approaches have three important characteristics: (1) Proposed methods adaptively estimate the local neighborhood surrounding each sample based on data density and similarity; (2) The objective functions of proposed methods can maximize margins between the each classes in the dimension-reduced feature space; (3) The objective functions of proposed methods have closed-form solutions. Furthermore, our methods can capture more discriminative information from raw data than other Isomap based methods. Extensive experiments on nine data sets demonstrate that the proposed methods are superior to the related state-of-the-art methods. [ABSTRACT FROM AUTHOR]

• The first automated machine learning approach to imbalanced domains. • 95% reduction of computational efforts concerning workflow selection and optimisation. • Competitive results in comparison to state-of-the-art AutoML methods. Imbalanced learning is one of the most relevant problems in machine learning. However, it faces two crucial challenges. First, the amount of methods proposed to deal with such problem has grown immensely, making the validation of a large set of methods impractical. Second, it requires specialised knowledge, hindering its use by those without such level of experience. In this paper, we propose the Automated Imbalanced Classification method, ATOMIC. Such a method is the first automated machine learning approach for imbalanced classification tasks. It provides a ranking of solutions most likely to ensure an optimal approximation to a new domain, drastically reducing associated computational complexity and energy consumption. We carry this out by anticipating the loss of a large set of predictive solutions in new imbalanced learning tasks. We compare the predictive performance of ATOMIC against state-of-the-art methods using 101 imbalanced data sets. Results demonstrate that the proposed method provides a relevant approach to imbalanced learning while reducing learning and testing efforts of candidate solutions by approximately 95%. [ABSTRACT FROM AUTHOR]

• The first public dataset dedicated to detecting abusive Turkish messages. • 10,528 abusive, 19,826 not-abusive Instagram comments have been collected. • CNN, NB, SVM, DT, RF, LR, AdaBoost, and XGBoost classifiers were evaluated. • The best performance (F1-score: 0.974) was achieved by the CNN model. Instagram is a free photo-sharing platform where each user has a profile and can upload photos for followers to view, like, and comment. Abusive comments on images can be humiliating and harmful to those who share photos. Developing a comment filter in languages other than English is difficult and time-consuming. This paper proposes a dataset called Abusive Turkish Comments (ATC) to detect abusive Instagram comments in Turkish. It is composed of a large number of Instagram comments posted to tabloid and sports accounts (i.e., 10,528 abusive and 19,826 not-abusive). It is the first public dataset dedicated to detecting abusive Turkish messages, as far as we know. The sentiment annotation has been done in sentence-level by assigning polarity to each comment. The performance of the abusive message detection models was evaluated using several performance metrics: Convolutional Neural Network (CNN), five well-known classifiers (i.e., Naive Bayes, Support Vector Machine, Decision Tree, Random Forest, and Logistic Regression), and two reweighted classifiers (i.e., Adaptive Boosting (AdaBoost), eXtreme Gradient Boosting (XGBoost)) were compared in terms of F1-score, precision, and recall. The results showed that the best performance (i.e., Micro-averaged F1-score: 0.974, Macro-averaged F1-score: 0.973, Kappa-value: 0.946) was yielded by the CNN model on the oversampled ATC dataset. The abusive message detection model proposed in this study can contribute to the development of Turkish comment filters on Instagram. Different model combinations are considered to select the best model that gives better recognition accuracy. [ABSTRACT FROM AUTHOR]

• Feature selection improves performance of machine learning algorithms. • Feature selection with more n-tier techniques is simpler and more stable. • A feature selection model that is not specific to any data set is widely applied. Feature Selection and classification have previously been widely applied in various areas like business, medical and media fields. High dimensionality in datasets is one of the main challenges that has been experienced in classifying data, data mining and sentiment analysis. Irrelevant and redundant attributes have also had a negative impact on the complexity and operation of algorithms for classifying data. Consequently, the algorithms record poor results or performance. Some existing work use all attributes for classification, some of which are insignificant for the task, thereby leading to poor performance. This paper therefore develops a hybrid filter model for feature selection based on principal component analysis and information gain. The hybrid model is then applied to support classification using machine learning techniques e.g. the Naïve Bayes technique. Experimental results demonstrate that the hybrid filter model reduces data dimensions, selects appropriate feature sets, and reduces training time, hence providing better classification performance as measured by accuracy, precision and recall.. [ABSTRACT FROM AUTHOR]

Novelty detection is a binary task aimed at identifying whether a test sample is novel or unusual compared to a previously observed training set. A typical approach is to consider distance as a criterion to detect such novelties. However, most previous work does not focus on finding an optimum distance for each particular problem. In this paper, we propose to detect novelties by exploiting non-linear distances learned from multi-class training data. For this purpose, we adopt a kernelization technique jointly with the Large Margin Nearest Neighbor (LMNN) metric learning algorithm. The optimum distance tries to keep each known class' instances together while pushing instances from different known classes to remain reasonably distant. We propose a variant of the K-Nearest Neighbors (KNN) classifier that employs the learned distance to detect novelties. Besides, we use the learned distance to perform multi-class classification. We show quantitative and qualitative experiments conducted on synthetic and real data sets, revealing that the learned metrics are effective in improving novelty detection compared to other metrics. Our method also outperforms previous work regularly used for novelty detection. • A novelty detection method based on learning non-linear distances is proposed. • Non-linear distances are learnt from datasets using a metric learning algorithm. • Normal data is attracted to clusters, while abnormalities become isolated. • Our approach is shown to outperform previous work on novelty detection. [ABSTRACT FROM AUTHOR]

• Providing an extensive overview about the use of machine learning in manufacturing. • Reviewing state-of-the-art studies relevant to data mining in manufacturing. • Presenting manufacturing tasks grouped under supervised and unsupervised learning. • Addressing a number of research questions that are unanswered in literature. • Discussing benefits, challenges and possible further research directions in the area. Manufacturing organizations need to use different kinds of techniques and tools in order to fulfill their foundation goals. In this aspect, using machine learning (ML) and data mining (DM) techniques and tools could be very helpful for dealing with challenges in manufacturing. Therefore, in this paper, a comprehensive literature review is presented to provide an overview of how machine learning techniques can be applied to realize manufacturing mechanisms with intelligent actions. Furthermore, it points to several significant research questions that are unanswered in the recent literature having the same target. Our survey aims to provide researchers with a solid understanding of the main approaches and algorithms used to improve manufacturing processes over the past two decades. It presents the previous ML studies and recent advances in manufacturing by grouping them under four main subjects: scheduling, monitoring, quality, and failure. It comprehensively discusses existing solutions in manufacturing according to various aspects, including tasks (i.e., clustering, classification, regression), algorithms (i.e., support vector machine, neural network), learning types (i.e., ensemble learning, deep learning), and performance metrics (i.e., accuracy, mean absolute error). Furthermore, the main steps of knowledge discovery in databases (KDD) process to be followed in manufacturing applications are explained in detail. In addition, some statistics about the current state are also given from different perspectives. Besides, it explains the advantages of using machine learning techniques in manufacturing, expresses the ways to overcome certain challenges, and offers some possible further research directions. [ABSTRACT FROM AUTHOR]

• Confident itemsets are used to discretize the whole model into subspaces. • Concise instance-wise explanations approximate the behavior of the black-box. • Class-wise explanations approximate the black-box's behavior in different subspaces. • Confident itemsets explanations improve the fidelity by 9.3% over other methods. • Confident itemsets explanations improve the interpretability by 8.8% Black-box Artificial Intelligence (AI) methods, e.g. deep neural networks, have been widely utilized to build predictive models that can extract complex relationships in a dataset and make predictions for new unseen data records. However, it is difficult to trust decisions made by such methods since their inner working and decision logic is hidden from the user. Explainable Artificial Intelligence (XAI) refers to systems that try to explain how a black-box AI model produces its outcomes. Post-hoc XAI methods approximate the behavior of a black-box by extracting relationships between feature values and the predictions. Perturbation-based and decision set methods are among commonly used post-hoc XAI systems. The former explanators rely on random perturbations of data records to build local or global linear models that explain individual predictions or the whole model. The latter explanators use those feature values that appear more frequently to construct a set of decision rules that produces the same outcomes as the target black-box. However, these two classes of XAI methods have some limitations. Random perturbations do not take into account the distribution of feature values in different subspaces, leading to misleading approximations. Decision sets only pay attention to frequent feature values and miss many important correlations between features and class labels that appear less frequently but accurately represent decision boundaries of the model. In this paper, we address the above challenges by proposing an explanation method named Confident Itemsets Explanation (CIE). We introduce confident itemsets, a set of feature values that are highly correlated to a specific class label. CIE utilizes confident itemsets to discretize the whole decision space of a model to smaller subspaces. Extracting important correlations between the features and the outcomes of the classifier in different subspaces, CIE produces instance-wise and class-wise explanations that accurately approximate the behavior of the target black-box. Conducting a set of experiments on various black-box classifiers, and different tabular and textual data classification tasks, we show that our CIE method performs better than the previous perturbation-based and rule-based explanators in terms of the descriptive accuracy (an improvement of 9.3%) and interpretability (an improvement of 8.8%) of the explanations. Subjective evaluations demonstrate that the users find the explanations of CIE more understandable and interpretable than those of the other comparison methods. [ABSTRACT FROM AUTHOR]

• A study on learning comprehensible and accurate hybrid trees. • Hybrid trees – a combination of comprehensible and black-box classifieirs. • A novel algorithm evaluated on 23 UCI datasets. • The algorithm provides trade-off between accuracy and comprehensibility. • The algorithm generates hybrid-trees faster that a baseline search algorithm. Finding the best classifiers according to different criteria is often performed by a multi-objective machine learning algorithm. This study considers two criteria that are usually treated as the most important when deciding which classifier to apply in practice: comprehensibility and accuracy. A model that offers a broad range of trade-offs between the two criteria is introduced because they conflict; i.e., increasing one decreases the other. The choice of the model is motivated by the fact that domain experts often formalize decisions based on knowledge that can be represented by comprehensible rules and some tacit knowledge. This approach is mimicked by a hybrid tree that consists of comprehensible parts that originate from a regular classification tree and incomprehensible parts that originate from an accurate black-box classifier. An empirical evaluation on 23 UCI datasets shows that the hybrid trees provide trade-offs between the accuracy and comprehensibility that are not possible using traditional machine learning models. A corresponding hybrid-tree comprehensibility metric is also proposed. Furthermore, the paper presents a novel algorithm for learning MAchine LeArning Classifiers with HybrId TrEes (MALACHITE), and it proves that the algorithm finds a complete set of nondominated hybrid trees with regard to their accuracy and comprehensibility. The algorithm is shown to be faster than the well-known multi-objective evolutionary optimization algorithm NSGA-II for trees with moderate size, which is a prerequisite for comprehensibility. On the other hand, the MALACHITE algorithm can generate considerably larger hybrid-trees than a naïve exhaustive search algorithm in a reasonable amount of time. In addition, an interactive iterative data mining process based on the algorithm is proposed that enables inspection of the Pareto set of hybrid trees. In each iteration, the domain expert analyzes the current set of nondominated hybrid trees, infers domain relations, and sets the parameters for the next machine learning step accordingly. [ABSTRACT FROM AUTHOR]

• Proposed method jointly optimizes both feature selection and classification. • Proposed method can reduce extremely high numbers of features. • This study uses multiple LS algorithms, running in parallel and sharing information. Over the last decade, the number of studies on machine learning has significantly increased. One of the most widely researched areas of machine learning is data classification. Most big data systems require a large amount of information storage for analytic purposes; however, this involves some disadvantages, such as the costs of processing and collecting data. Thus, many researchers and practitioners are working on effectively reducing the number of features used in classification. This paper proposes a method which jointly optimizes both feature selection and classification. A survey of the relevant literature shows that the vast majority of studies focus on either feature selection or classification. In this study, the proposed parallel local search algorithm both selects features and finds a classifier with high rates of accuracy. Moreover, the proposed method is capable of finding solutions for problems that have extremely high numbers of features within a reasonable computation time. [ABSTRACT FROM AUTHOR]

• Proposed an intelligent diagnosis framework for skin lesion classification. • Designed a multi-class multi-level algorithm to enhance the accuracy. • Proposed improved techniques for noise removal from the images. • Deep learning and other machine learning approaches developed and compared. • So far, the best multi-class result (∼96.5% accuracy) achieved using deep learning. Skin diseases remain a major cause of disability worldwide and contribute approximately 1.79% of the global burden of disease measured in disability-adjusted life years. In the United Kingdom alone, 60% of the population suffer from skin diseases during their lifetime. In this paper, we propose an intelligent digital diagnosis scheme to improve the classification accuracy of multiple diseases. A Multi-Class Multi-Level (MCML) classification algorithm inspired by the "divide and conquer" rule is explored to address the research challenges. The MCML classification algorithm is implemented using traditional machine learning and advanced deep learning approaches. Improved techniques are proposed for noise removal in the traditional machine learning approach. The proposed algorithm is evaluated on 3672 classified images, collected from different sources and the diagnostic accuracy of 96.47% is achieved. To verify the performance of the proposed algorithm, its metrics are compared with the Multi-Class Single-Level classification algorithm which is the main algorithm used in most of the existing literature. The results also indicate that the MCML classification algorithm is capable of enhancing the classification performance of multiple skin lesions. [ABSTRACT FROM AUTHOR]