Search

Automatic clustering and dimension reduction are two of the most intriguing topics in the field of clustering. Affinity propagation (AP) is a representative graph-based clustering algorithm in unsupervised learning. However, extracting features from high-dimensional data and providing satisfactory clustering results is a serious challenge for the AP algorithm. Besides, the clustering performance of the AP algorithm is sensitive to preference. In this paper, an improved affinity propagation based on optimization of preference (APBOP) is proposed for automatic clustering on high-dimensional data. This method is optimized to solve the difficult problem of determining the preference of affinity propagation and the poor clustering effect for non-convex data distribution. First, t-distributed stochastic neighbor embedding is introduced to reduce the dimensionality of the original data to solve the redundancy problem caused by excessively high dimensionality. Second, an improved hybrid equilibrium optimizer based on the crisscross strategy (HEOC) is proposed to optimize preference selection. HEOC introduces the crisscross strategy to enhance local search and convergence efficiency. The benchmark function experiments indicate that the HEOC algorithm has better accuracy and convergence rate than other swarm intelligence algorithms. Simulation experiments on high-dimensional and real-world datasets show that APBOP has better effectiveness. [ABSTRACT FROM AUTHOR]

• Prior Knowledge Encoding. We propose a novel knowledge-aware prompt-tuning into verbalizer for biomedical relation extraction that the rich semantic knowledge to solve the problem, which simultaneously transfers entity-node-level and relation-link-level structures across graphs. • Efficient Prompt Design. To the best of our knowledge, this is the first work to successfully perform the knowledge jointly optimizing the representation of a learnable words and learnable relation words in the biomedical graphs. • SOTA Results. BioKnowPrompt outperforms baseline models in five relation extraction benchmark datasets in the biomedical domain, showing its potential to serve as an effective general-purpose representation learning algorithm for biomedical graph data. Domain tuning pre-trained language models (PLMs) with task-specific prompts have achieved great success in different domains. By using cloze-style language prompts to stimulate the versatile knowledge of PLMs, which directly bridges the gap between pre-training tasks and various downstream tasks. Large unlabelled corpora in the biomedical domain have been created in the last decade(i.e., PubMed, PMC, MIMIC, and ScienceDirect). In this paper, we introduce BioKnowPrompt, a prompt-tuning PLMs model that has been incorporating imprecise knowledge into verbalizer for biomedical text relation extraction. In particular, we use learnable words and learnable relation words to infuse entity and relation information into quick creation, and we use biomedical domain knowledge constraints to synergistically improve their representation. By using additional prompts to fine-tune PLMs, we can further stimulate the rich knowledge distributed in PLMs to better serve downstream tasks such as relation extraction. BioKnowPrompt has a lot of significant potential in few-shot learning, which outperforms the previous models and achieves state-of-the-art on the 5 datasets. [ABSTRACT FROM AUTHOR]

Federated learning is a privacy-preserving machine learning paradigm that can train a model with decentralized data. Classical federated learning systems are vulnerable to attacks from malicious clients. Although a number of efforts have been made to improve robustness of federated learning, existing solutions are either too costly or attack-specific. In this paper, we propose the Robust Ternary Gradients Aggregation (RTGA) algorithm, which can efficiently handle different attacks using two novel mechanisms. At the client-side, the ternary quantization mechanism compresses gradients using only two bits to store each coordinate of the gradient vector. We adopt the error-feedback mechanism to compensate for the difference between the compressed and actual gradients. At the server-side, the robust aggregation mechanism is designed to detect malicious clients according to the risk score of each client. The risk score is calculated by solving an optimization problem of the divergence among clients. Furthermore, we provide analysis of the compression and robustness properties of our proposed algorithm. Extensive experiments show that, compared with existing algorithms, RTGA can mitigate different types of attacks while costing much less communications. [ABSTRACT FROM AUTHOR]

Positive and unlabeled learning (PU learning) is a kind of problem whose goal is learning a two-classes classifier with little proportion of positive samples and numerous unlabeled samples. A series of studies focus on how to extract most likely negative samples from the unlabeled samples, and then train a classifier with the labeled samples as supervised learning. Previous PU learning methods always ignore the additional information called privileged information, which is just provided during the training process while unavailable during testing. In this paper, we propose a novel self-paced algorithm for PU learning with privileged information (SPUPI). The proposed SPUPI extracts some reliable negative samples from unlabeled samples at first, and then generates weights for the unlabeled samples according to the similarity with each class. After that, it builds a more accurate classifier based on privileged information and similarity weights by self-paced learning. By taking the self-paced learning into training, we can build the model with a few labeled samples from easy to complex. We also solve the problem by transforming the primal problem of the proposed model into its dual problem and achieving the PU classifier. Various experiments on the practical datasets indicate that the SPUPI has a better performance compared with previous methods. [ABSTRACT FROM AUTHOR]

[Display omitted] • A two-phase optimization system for solving classification problems. • Combining Auto-ML pipelines to improve overall performance. • Intelligent selection of ensemble methods. • Best results delivered by double-fault measure and 20 or 50 maximum number of base models. Automatic Machine Learning (Auto-ML) tools enable the automatic solution of real-world problems through machine learning techniques. These tools tend to be more time consuming than standard machine learning libraries, therefore, exploiting all the available resources to the full is a valuable feature. This paper presents a two-phase optimization system for solving classification problems. The system is designed to produce more robust classifiers by exploiting the different architectures that are generated while solving classification problems with Auto-ML tools, particularly AutoGOAL. In the first phase, the system follows a probabilistic strategy to find the best combination of algorithms and hyperparameters to generate a collection of base models according to certain diversity criteria; and in the second, it follows similar Auto-ML strategies to ensemble those models. The HAHA 2019 challenge corpus and the Adult dataset were used to evaluate the system. The experimental results show that: i) a better solution can be built by ensembling a subset of the already tested models; ii) the performance of ensemble methods depends on the collection of base models used; and, iii) ensuring diversity using the double-fault measure produces better results than the disagreement measure. The source code is available online for the research community. [ABSTRACT FROM AUTHOR]

K-nearest neighbors search (K-NNS) is a fundamental problem in many areas of machine learning and data mining. In an attempt to solve NNS problems by locality-sensitive hashing (LSH)-based algorithms and avoid the F1-trap, in this paper, we theoretically prove the necessity of maintaining close edges of each dimension, the independence of the partition location and the precision of retrieval. Based on the theories, a new method is proposed for constructing the hash function family. Then, a volume-balancing locality-sensitive hashing (VBLSH) algorithm is implemented. In this algorithm, the problem of constructing the hash function family is transformed into an optimization problem under a fixed number of hypercubes. By solving the optimization problem, different hyperplane intervals are set for each dimension to ensure that the hash functions can evenly segment the data space, where each segmented hypercube has a close volume and close side lengths. Finally, the analysis of the algorithm shows that the VBLSH algorithm has a high index coding efficiency and hash code calculation efficiency. The experimental results show that the VBLSH algorithm can simultaneously take into account the efficiency, precision and recall rate, and avoid the F1-trap; it shows excellent effectiveness on 4 datasets. [ABSTRACT FROM AUTHOR]

Feature selection plays a crucial role in machine learning and data mining, and improves the performance of learning models by selecting a distinguishing feature subset and eliminating irrelevant features. Existing feature selection methods are mainly used for single-label learning and multi-label learning; however, there are only a few feature selection methods for label distribution learning. Label distribution learning has the "curse of dimensionality" problem, similar to that in multi-label learning. In label distribution learning, the related labels of each sample have different levels of importance. Therefore, multi-label feature selection algorithms can not be directly applied to label distribution data, and discretizing the label distribution data into multi-label data would result in the loss of some important supervised information. To solve this problem, a novel feature selection algorithm for label distribution learning is proposed in this paper. The proposed method utilizes neighborhood granularity to explore feature similarity, and it uses a correlation coefficient to generate the label correlations. In addition, sparse learning is used to improve the robustness and control complexity. Experimental results indicate that our proposed method is more effective than five state-of-art feature selection algorithms on twelve datasets, with respect to six representative evaluation measures. [ABSTRACT FROM AUTHOR]

Dynamic adaptive streaming over the HTTP (DASH) standard has been widely adopted by many content providers for online video transmission and greatly improve the performance. Designing an efficient DASH system is challenging because of the inherent large fluctuations characterizing both encoded video sequences and network traces. In this paper, a reinforcement learning (RL)-based DASH technique that addresses user quality of experience (QoE) is constructed. The DASH adaptive bitrate (ABR) selection problem is formulated as a Markov decision process (MDP) problem. Accordingly, an RL-based solution is proposed to solve the MDP problem, in which the DASH clients act as the RL agent, and the network variation constitutes the environment. The proposed user QoE is used as the reward by jointly considering the video quality and buffer status. The goal of the RL algorithm is to select a suitable video quality level for each video segment to maximize the total reward. Then, the proposed RL-based ABR algorithm is embedded in the QoE-oriented DASH framework. Experimental results show that the proposed RL-based ABR algorithm outperforms state-of-the-art schemes in terms of both temporal and visual QoE factors by a noticeable margin while guaranteeing application-level fairness when multiple clients share a bottlenecked network. [ABSTRACT FROM AUTHOR]

• Propose an auto-weighted manifold embedding for multi-view semi-supervised classification. • Present an accelerated method with theoretic guarantee and low computational complexity. • Design practical optimization procedures to solve the problems with fast convergence. Multi-view semi-supervised learning has gained much attention since a great number of unlabeled multi-view data are easy to obtain while few labeled data are available. Accordingly, how to utilize the relationship between labeled and unlabeled data is of significance in multi-view semi-supervised learning. In this paper, we propose an auto-weighted manifold embedding model to address multi-view semi-supervised classification problems, where only a small percentage of labeled data points are used for model training. In the proposed model, data points close to each other in the feature space will be assigned to similar classes in the label space through manifold embedding. Accordingly, the class information of labeled data will be employed in the prediction process for unlabeled data. Moreover, an optimal weight for each view of multi-view data is learned automatically, which enables an encouraging fusion quality for multi-view manifold embedding. To further speed up the calculation process and reduce the computational complexity of the proposed model, an effective accelerated auto-weighted manifold embedding scheme is developed. Besides, theoretical analyses are then provided to indicate a tight approximation bound for the primary manifold embedding method, while the accelerated method is designed by an order decrease of magnitude of computational complexity. Finally, comprehensive experiments on eight publicly available data sets demonstrate the superiority of the proposed models over compared state-of-the-art semi-supervised methods and fully supervised classifiers. Furthermore, the experimental results are suggestive of positive robustness and promising generalization capacity of the proposed methods. [ABSTRACT FROM AUTHOR]

Nowadays, bike-sharing is available in many cities, solving the problem of the last mile, and it is an environmental-friendly way to commute. However, there is a tidal phenomenon in the bike-sharing system, and the rents/returns of bikes at different stations are unbalanced. Thus, bikes at different stations need to be rebalanced regularly and station-level demand prediction plays an essential role in bike-sharing rebalancing. In this paper, a novel deep graph convolutional network (GCN) model with temporal attention (TAGCN) is proposed for bike check-out/in number prediction of each station. TAGCN can not only model the spatial and temporal dependency between varying stations, but also reflect the influence of different time granularity, which are hour-level, day-level and week-level time periodicity. With the help of well-designed temporal attention mechanism, our model can capture the dynamical temporal correlations and comprehensive spatial patterns in bike check-out/in flow effectively. The proposed model consistently outperforms state-of-the-art methods on four real-world bike-sharing datasets that are four seasons data of Divvy Bike System in Chicago. [ABSTRACT FROM AUTHOR]

Although multi-objective evolutionary subspace clustering approaches have shown promise in handling high-dimensional datasets, their performance is restricted by two main drawbacks. First, their local search strategies have not been well investigated. Second, while exploring the search space, they neglect the useful knowledge from previously solved problems. To tackle these issues, this paper proposes a transfer learning-assisted multi-objective evolutionary clustering framework with decomposition. Firstly, we provide a decomposition-based local search strategy. To capture a comprehensive data structure, this strategy updates the weights of features by considering both the within-class compactness and between-class separation, and spontaneously balances the two properties. Secondly, we develop a knowledge transfer strategy. By transferring search experience from a previously solved clustering problem, the strategy improves the search efficiency, consequently enhances the clustering accuracy of the current problem. It has a closed-form solution and can transfer knowledge across both homogeneous and heterogeneous problems from either different or the same domains. Finally, we conduct an extensive experimental study on the framework by comparing with six representative subspace clustering approaches on a wide range of benchmarks and real-world applications. Results demonstrate the superiority of our framework. [ABSTRACT FROM AUTHOR]

Abstract: Rapidly increasing complexity of vehicle systems is calling for technologies to promptly analyze field problems, and effectively identify weakness in vehicle engineering design, in order to enhance product quality. Recent vehicular communication technologies allow for remote access to extensive amount of vehicle data in a cost-effective way, which enables in-depth field issue analysis. However, practical solutions are still lacking to effectively turn the massive amount of raw data into actionable design enhancement suggestions. In this paper, we propose a general framework, named Automatic Field Data Analyzer (AFDA), and related algorithms that analyze large volumes of field data, and identify root causes of faults by systematically making use of signal processing, machine learning, and statistical analysis approaches. AFDA evaluates vehicle system performance, generates feature vectors that represent different root causes of faults, and identifies the features that are most relevant to system performance fluctuation, which eventually reveals the underlying reasons for the faults. This paper presents a case study of AFDA in the application of vehicle battery, where gigabytes of real vehicle data are sifted through, and the root causes of field issues are identified. The results well match the findings from experts with years of experiences. The proposed data-based scheme and approaches can be generally applied to any vehicle systems. [Copyright &y& Elsevier]

Abstract: Teaching–Learning-Based Optimization (TLBO) seems to be a rising star from amongst a number of metaheuristics with relatively competitive performances. It is reported that it outperforms some of the well-known metaheuristics regarding constrained benchmark functions, constrained mechanical design, and continuous non-linear numerical optimization problems. Such a breakthrough has steered us towards investigating the secrets of TLBO’s dominance. This paper reports our findings on TLBO qualitatively and quantitatively through code-reviews and experiments, respectively. Our findings have revealed three important mistakes regarding TLBO: (1) at least one unreported but important step; (2) incorrect formulae on a number of fitness function evaluations; and (3) misconceptions about parameter-less control. Additionally, unfair experimental settings/conditions were used to conduct experimental comparisons (e.g., different stopping criteria). The experimental results for constrained and unconstrained benchmark functions under fairly equal conditions failed to validate its performance supremacy. The ultimate goal of this paper is to provide reminders for metaheuristics’ researchers and practitioners in order to avoid similar mistakes regarding both the qualitative and quantitative aspects, and to allow fair comparisons of the TLBO algorithm to be made with other metaheuristic algorithms. [Copyright &y& Elsevier]

Abstract: The results reported in this paper create a step toward the rough set-based foundations of data mining and machine learning. The approach is based on calculi of approximation spaces. In this paper, we present the summarization and extension of our results obtained since 2003 when we started investigations on foundations of approximation of partially defined concepts (see, e.g., ). We discuss some important issues for modeling granular computations aimed at inducing compound granules relevant for solving problems such as approximation of complex concepts or selecting relevant actions (plans) for reaching target goals. The problems discussed in this article are crucial for building computer systems that assist researchers in scientific discoveries in many areas such as biology. In this paper, we present foundations for modeling of granular computations inside of system that is based on granules called approximation spaces. Our approach is based on the rough set approach introduced by Pawlak . Approximation spaces are fundamental granules used in searching for relevant complex granules called as data models, e.g., approximations of complex concepts, functions or relations. In particular, we discuss some issues that are related to generalizations of the approximation space introduced in . We present examples of rough set-based strategies for the extension of approximation spaces from samples of objects onto a whole universe of objects. This makes it possible to present foundations for inducing data models such as approximations of concepts or classifications analogous to the approaches for inducing different types of classifiers known in machine learning and data mining. Searching for relevant approximation spaces and data models are formulated as complex optimization problems. The proposed interactive, granular computing systems should be equipped with efficient heuristics that support searching for (semi-)optimal granules. [Copyright &y& Elsevier]

Rule learning is a special type of machine learning approaches, and its key advantage is the generation of interpretable models, which provides a transparent process of showing how an input is mapped to an output. Traditional rule learning algorithms are typically based on Boolean logic for inducing rule antecedents, which are very effective for training models on data sets that involve discrete attributes only. When continuous attributes are present in a data set, traditional rule learning approaches need to employ crisp intervals. However, in reality, problems usually show shades of grey, which motivated the development of fuzzy rule learning approaches by employing fuzzy intervals for handling continuous attributes. While a data set contains a large portion of discrete attributes or even no continuous attributes, fuzzy approaches cannot be used to learn rules effectively, leading to a drop in the performance. In this paper, a multi-stage approach of mixed rule learning is proposed, which involves strategic combination of both traditional and fuzzy approaches to handle effectively various types of attributes. We compare our proposed approach with existing algorithms of rule learning. Our experimental results show that our proposed approach leads to significant advances in the performance compared with the existing algorithms. [ABSTRACT FROM AUTHOR]

Large-scale optimization, solving real high-dimensional problems, has attracted many research interests. Large-scale optimization problems are far more difficult than traditional optimization problems due to their numerous local optimum. In this paper, a principle of maximizing the fitness difference between learners and exemplars is proposed to improve the performance of the optimization algorithm. Then based on the principle, a improved particle swarm optimization algorithm called the "ranking-based biased learning swarm optimizer for large-scale optimization" (RBLSO) is proposed. The proposed RBLSO contains two types of learning strategies, namely, ranking paired learning (RPL) and biased center learning (BCL). In RPL, the worse particles learn peer to peer from the better particles according to their ranks, so then the convergence speed will be accelerated. In BCL, each particle learns from the biased center that is defined as the fitness weighted center of the whole swarm. This operator is utilized to strengthen the explorative ability of the algorithm. To test the performances of the proposed algorithm, we conduct some experiments on the proposed learning mechanism. RBLSO is compared with several state-of-the-art large-scale optimization algorithms on two widely used benchmark function sets, CEC2010 and CEC2013. These two function sets were proposed for the special session and competition on large-scale global optimization held under the Congress on Evolutionary Computation (CEC) 2010 and 2013. Experimental results show that RBLSO is effective in solving large-scale optimization problems. [ABSTRACT FROM AUTHOR]

Abstract In this contribution we consider methods for improving the quality of classification by the k nearest neighborhood classifiers in the case of large number of missing values. The classical version of such classifier is applied, moreover the classifier with the aggregation of certainty coefficients of the individual classifiers with the use of the arithmetic mean, as well as new versions of the classifier with the use of diverse interval-valued aggregation functions are considered. After adding missing values to the data, the quality of the classical method decreases. By contrast, the interval modeling of missing values and application of interval-valued aggregation functions in the classifiers entail a much slower decrease of their quality. The paper includes the results of experiments performed on the data sets from the machine learning repository UCI. [ABSTRACT FROM AUTHOR]

Abstract The decision-theoretic rough set (DTRS) can be regarded as a type of cost-sensitive learning method that incorporates cost functions. Three-way decisions for an object are made in the DTRS model to obtain the minimum Bayesian decision cost, including acceptance, rejection and deferment decisions. However, because the cost matrix in traditional cost-sensitive learning problems does not include the cost for deferment decisions, DTRS cannot be directly used to solve traditional cost-sensitive learning problems. In this paper, we present a multiclass three-way decision-theoretic rough set model that can be directly applied to traditional cost-sensitive learning problems. This model can calculate the corresponding cost for deferment decisions based on the traditional cost matrix. In addition, we propose a multiphase cost-sensitive learning method based on the multiclass three-way decision-theoretic rough set model. The deferment and rejection samples are reduced and eliminated through multiphase iterative learning. Compared with the existing multiclass cost-sensitive learning methods, our proposed method can achieve higher classification accuracy and lower misclassification cost. [ABSTRACT FROM AUTHOR]

Abstract: This paper proposes the construction of a centralized hybrid metaheuristic cooperative strategy to solve optimization problems. Knowledge (intelligence) is incorporated into the coordinator to improve performance. This knowledge is incorporated through a set of rules and models obtained from a knowledge extraction process applied to the records of the results returned by individual metaheuristics. The effectiveness of the approach is tested in several computational experiments in which we compare the results obtained by the individual metaheuristics, by several non-cooperative and cooperative strategies and by the strategy proposed in this paper. [Copyright &y& Elsevier]

Multicriteria classification refers to classify objects evaluated by a set of criteria to preference-ordered decision classes. Dominance-based rough set approach has been successfully introduced to express and reason inconsistencies with a dominance principle in multicriteria classification problems. Hierarchical attribute values exist extensively within many real-world applications, which provide a hierarchical form to organize, view and analyze data from different perspectives for accommodating the preference variability. In this study, we consider an extension of dominance-based rough set approach by applying an incremental learning technique for hierarchical multicriteria classification while attribute values dynamically vary across different levels of granulations. We formalize the dynamic characteristics of knowledge granules with the cut refinement and coarsening through attribute value taxonomies in the hierarchical multicriteria decision systems. In consequence, incremental algorithms for computing dominance-based rough approximations of preference-ordered decision classes are developed by applying the resulted prior-knowledge as the input, and only recomputing those outputs which depend on the changed attribute values. This paper presents the theoretical foundation of the proposed approach. Example analysis and experimental evaluation are also provided for illustration of the feasibility and efficiency. [ABSTRACT FROM AUTHOR]

Multiple-Instance Learning (MIL) refers to the problem wherein each object is a bag consisting of multiple instances and only the bags' labels are provided. MIL data can contain irrelevant, redundant, and noisy components, which makes feature-extraction preprocessing essential for performance improvement. In this paper, we propose a Multiple-Instance Feature Extraction (MIFE) framework to design algorithms at both the bag and instance levels based on the Maximum Trace-Difference criterion, which simultaneously maximizes between-class scattering and minimizes within-class scattering. MIFE not only treats the existing Multiple-Instance Discriminant Analysis algorithm as an instance-level realization but also enables us to adopt different bag-level distances to design corresponding bag-level algorithms. In particular, we introduce the Class-to-Bag (C2B) and Bag-to-Bag (B2B) distances into the MIFE framework and obtain the MIFE-C2B and MIFE-B2B algorithms, respectively. The experimental results show that both MIFE-C2B and MIFE-B2B obtain competitive classification performance, and MIFE-B2B obtains the best performance on most tested datasets. The dimensionality reduction results show that both MIFE-C2B and MIFE-B2B obtain their best performance with no more than approximately 30% of the original dimensions on most tested datasets. [ABSTRACT FROM AUTHOR]

In this paper, a novel recombination operator, called adaptive hybrid crossover operator (AHX), is designed for tackling continuous multiobjective optimization problems (MOPs), which works effectively to enhance the search capability of multiobjective evolutionary algorithms (MOEAs). Different from the existing hybrid operators that are commonly operated on chromosome level, the proposed operator is executed on gene level to combine the advantages of simulated binary crossover (SBX) with local search ability and differential evolution (DE) with strong global search capability. More opportunities are assigned to DE in the early evolutionary stage for gene-level global search in decision space; whereas, with the generation grows, more chances are gradually allocated to SBX for gene-level local search. The balance between the gene-level global and local search is well maintained by an adaptive control approach in AHX. To validate the effectiveness of AHX, it is studied by substituting the original recombination operators in the four state-of-the-art MOEAs (i.e., NSGA-II, SPEA2, SMS-EMOA, and MOEA/D), and the performance of revised algorithms is significantly improved. Furthermore, AHX is also compared to three recently proposed recombination operators, such as a newly DE inspired (DEI) recombination operator, a learning paradigm based on jumping genes (JGBL) and a bandit-based adaptive operator selection approach (FRRMAB). The experimental studies validate that AHX can be effectively integrated into different frameworks of MOEAs, and performs better than SBX, DE, DEI, JGBL and FRRMAB in solving various kinds of MOPs. [ABSTRACT FROM AUTHOR]

When memory-based evolutionary algorithms are applied in dynamic environments, the certainly use of uncertain prior knowledge for future environments may mislead the evolutionary algorithms. To address this problem, this paper presents a new, memory-based evolutionary approach for applying the Bayesian optimization algorithm (BOA) in dynamic environments. Our proposed method, unlike existing memory-based methods, uses the knowledge of former environments probabilistically in future environments. For this purpose, the run of BOA is modeled as the movements in a Markov chain, in which the states become the Bayesian networks that are learned in every generation. When the environment changes, a stationary distribution of the Markov chain is defined on the basis of the retrieved prior knowledge. Then, the transition probabilities of BOA in the Markov chain are modified (biased) to comply with the defined stationary distribution. To this end, we employ the Metropolis algorithm and modify the K2 algorithm for learning the Bayesian network in BOA in order to reflect the obtained transition probabilities. Experimental results show that the proposed method achieves improved performance compared to conventional methods, especially in random environments. [ABSTRACT FROM AUTHOR]

Feedforward neural networks with random weights (FNNRWs), as random basis function approximators, have received considerable attention due to their potential applications in dealing with large scale datasets. Special characteristics of such a learner model come from weights specification, that is, the input weights and biases are randomly assigned and the output weights can be analytically evaluated by a Moore–Penrose generalized inverse of the hidden output matrix. When the size of data samples becomes very large, such a learning scheme is infeasible for problem solving. This paper aims to develop an iterative solution for training FNNRWs with large scale datasets, where a regularization model is employed to potentially produce a learner model with improved generalization capability. Theoretical results on the convergence and stability of the proposed learning algorithm are established. Experiments on some UCI benchmark datasets and a face recognition dataset are carried out, and the results and comparisons indicate the applicability and effectiveness of our proposed learning algorithm for dealing with large scale datasets. [ABSTRACT FROM AUTHOR]

In this paper, a dynamic mentoring scheme along with a self-regulation scheme have been incorporated in the standard Particle Swarm Optimization (PSO) algorithm to empower the searching particles with human-like characteristics. The algorithm is referred to as a Dynamic Mentoring and Self-Regulation based Particle Swarm Optimization (DMeSR-PSO) algorithm. Based on their experiences, the particles are divided into three groups, viz., the mentor group, the mentee group and the independent learner group where the number of particles in each group is dynamically changing in every iteration. In human learning psychology, mentoring is regarded as a powerful and effective learning process and independent learners are the ones who do not need mentoring and are capable of performing self-regulation of their own knowledge. Therefore, the particles in each of the above three groups have different learning strategies for their velocity updates where the mentors are equipped with a strong self-belief based search, the mentees are taking guidance from the mentors and the independent learners employ self-perception strategy. The DMeSR-PSO algorithm has been extensively evaluated using the simple unimodal and multimodal benchmark functions from CEC2005, more complex shifted and rotated benchmark functions fromCEC2013 and also based on eight real- world problems from CEC2011. The results have been compared with six state-of-the-art PSO variants and five meta-heuristic algorithms for the CEC2005 problems. Further, a comparative analysis on CEC2013 benchmark functions with different PSO variants has also been presented. Finally, DMeSR-PSO's performance on the real-world problems is compared with the top two algorithms from the CEC2011 competition. The results indicate that the proposed learning strategies help DMeSR-PSO to achieve faster convergence and provide better solutions in most of the problems with a 95% confidence level, yielding an effective optimization algorithm for real-world applications. [ABSTRACT FROM AUTHOR]

Data clustering aims to segment a database into groups of objects based on the similarity among these objects. Due to its unsupervised nature, the search for a good-quality solution can become a complex process. There is currently a wide range of clustering algorithms, and selecting the best one for a given problem can be a slow and costly process. In 1976, Rice formulated the Algorithm Selection Problem (ASP), which postulates that the algorithm performance can be predicted based on the structural characteristics of the problem. Meta-learning brings the concept of learning about learning; that is, the meta-knowledge obtained from the algorithm learning process allows the improvement of the algorithm performance. Meta-learning has a major intersection with data mining in classification problems, in which it is normally used to recommend algorithms. The present paper proposes new ways to obtain meta-knowledge for clustering tasks. Specifically, two contributions are explored here: (1) a new approach to characterize clustering problems based on the similarity among objects; and (2) new methods to combine internal indices for ranking algorithms based on their performance on the problems. Experiments were conducted to evaluate the recommendation quality. The results show that the new meta-knowledge provides high-quality algorithm selection for clustering tasks. [ABSTRACT FROM AUTHOR]

Modeling dynamical systems is a commonly used technique to solve partially observable problems in the artificial intelligence field. Predictive state representations (PSRs) have been proposed as an alternative to partially observable Markov decision processes (POMDPs) to model dynamical systems. Although POMDPs and PSRs provide general frameworks to solve partially observable problems, they rely heavily on a known and accurate model of the environment. However, in real world applications it is extremely difficult to build an accurate model. In this paper, we propose an algorithm to solve partially observable problems using an inaccurate PSR model which is learned from samples. The proposed algorithm can also improve the accuracy of the learned model. Given the inaccurate PSR model, the PSR state is identified firstly. Then the traditional Markov decision processes (MDP) techniques are used to solve the partially observable problem. Furthermore, the learned model, which may get off-track as often happens when the model is learned from samples, can be reset. The effectiveness of our proposed algorithm is demonstrated based on a standard set of POMDP test problems. [ABSTRACT FROM AUTHOR]

Abstract: In recent years, particle swarm optimization (PSO) algorithm has been used to solve global optimization problems. This algorithm is widely used as an effective optimization tool in various applications. However, traditional PSO consists of only two searching layers and thus often results in premature convergence into the local minima. Thus, multi-layer particle swarm optimization (MLPSO) is proposed in this paper to improve the performance of traditional PSO by increasing the two layers of swarms to multiple layers. The MLPSO strategy increases the diversity of searching swarms to improve its performance when solving complex problems. The experiment indicates that the novel approach improves the final results and the convergence speed. [Copyright &y& Elsevier]

Abstract: The Multi-Layer Perceptron (MLP), as one of the most-widely used Neural Networks (NNs), has been applied to many practical problems. The MLP requires training on specific applications, often experiencing problems of entrapment in local minima, convergence speed, and sensitivity to initialization. This paper proposes the use of the recently developed Biogeography-Based Optimization (BBO) algorithm for training MLPs to reduce these problems. In order to investigate the efficiencies of BBO in training MLPs, five classification datasets, as well as six function approximation datasets are employed. The results are compared to five well-known heuristic algorithms, Back Propagation (BP), and Extreme Learning Machine (ELM) in terms of entrapment in local minima, result accuracy, and convergence rate. The results show that training MLPs by using BBO is significantly better than the current heuristic learning algorithms and BP. Moreover, the results show that BBO is able to provide very competitive results in comparison with ELM. [Copyright &y& Elsevier]

Abstract: Classifying program modules as fault-prone or not fault-prone is a valuable technique for guiding the software development process, so that resources can be allocated to components most likely to have faults. The rule-based classification and the case-based learning techniques are commonly used in software quality classification problems. However, studies show that these two techniques share some complementary strengths and weaknesses. Therefore, in this paper we propose a new multi-strategy classification model, RB2CBL, which integrates a rule-based (RB) model with two case-based learning (CBL) models. RB2CBL possesses the merits of both the RB model and CBL model and restrains their drawbacks. In the RB2CBL model, the parameter optimization of the CBL models is critical and an embedded genetic algorithm optimizer is used. Two case studies were carried out to validate the proposed method. The results show that, by suitably choosing the accuracy of the RB model, the RB2CBL model outperforms the RB model alone without overfitting. [Copyright &y& Elsevier]

Abstract: Decision support systems (DSSs) perform complex computations to provide suggestions regarding decision-making and problem solving. Quite often, the DSS solutions are not fully accepted by users because DSSs work as a black box so that the users cannot fully understand where the results came from and how they were derived. Explanations of the generated DSSs solutions are expected to mitigate this situation. In this paper, two machine-learning techniques, called rough set analysis (RSA) and dependency network analysis (DNA), are proposed for mining DSS solutions. The mining results are provided to the users as explanations for those solutions. Two parts of research results are described. First, a framework applying RSA and DNA for generating explanations for DSS solutions is presented. This framework is generic and applicable to many other DSSs. Second, as a proof-of-concept, the applications of RSA and DNA techniques are demonstrated through a case study of mining patterns from input-output pairs of ReleasePlanner™, a specific DSS for product release planning. Our evaluation indicates that the explanations generated by RSA and DNA improve the overall user acceptance of results provided by this specific DSS. [Copyright &y& Elsevier]

Abstract: Non-stationary classification problems concern the changes on data distribution over a classifier lifetime. To face this problem, learning algorithms must conciliate essential, but difficult to gather, attributes like good classification performance, stability and low associated costs, like processing time and memory. This paper presents an extension of the K-associated optimal graph learning algorithm to cope with classification over non-stationary domains. The algorithm relies on a graph structure consisting of many disconnected components (subgraphs). Such graph enhances data representation by fitting locally groups of data according to a purity measure, which, in turn, quantifies the overlapping between vertices of different classes. As a result, the graph can be used to accurately estimate the probability of unlabeled data to belong to a given class. The proposed algorithm is benefited from the dynamical evolution of the graph by updating its set of components when new data is presented along time, by removing old components as new components arise. Experimental results on artificial and real domains and further statistical analysis show that the proposed algorithm is an effective solution to non-stationary classification problems. [Copyright &y& Elsevier]

Abstract: This paper proposes the Simplified Structure Evolving Method (SSEM) for fuzzy system identification, which improves our earlier work on the Structure Evolving Learning Method for fuzzy systems (SELM). The improvement is that SSEM applies a scheme that starts with the simplest fuzzy rule set with only one fuzzy rule (instead of 2 n fuzzy rules as in SELM, where n is the number of input variables), whilst retaining all the advantages of SELM. SELM is able to solve the problem of the exponential increase of fuzzy rules, however, it requires a basic fuzzy rule set which is exponential to the number of input variables (2 n fuzzy rules) as a starting point. The improvement offered by SSEM enables automatic feature selection and system structure identification, and avoids inefficient rules and inefficient variable involvement for system identification. This improvement enables fuzzy systems to be applicable to problems of any input dimension. Three benchmark examples with high dimension inputs are given to illustrate the advantages of the proposed algorithm. [Copyright &y& Elsevier]

Abstract: This paper discusses a multi-period portfolio selection problem when security returns are given by experts’ evaluations. The security return rates are regarded as uncertain variables and an uncertain risk index adjustment model is proposed. Optimal portfolio adjustments are determined with the objective of maximizing the total incremental wealth within the constraints of controlling the cumulative risk index value over the investment horizon and satisfying self-financing at each period. To enable the users to solve the model problem with currently available programming tools, an equivalent of the model is provided. In addition, a method of obtaining the uncertainty distributions of the security returns is given based on experts’ evaluations, and a selection example is presented. [Copyright &y& Elsevier]

Abstract: The machine learning community has traditionally used correct classification rates or accuracy (C) values to measure classifier performance and has generally avoided presenting classification levels of each class in the results, especially for problems with more than two classes. C values alone are insufficient because they cannot capture the myriad of contributing factors that differentiate the performance of two different classifiers. Receiver Operating Characteristic (ROC) analysis is an alternative to solve these difficulties, but it can only be used for two-class problems. For this reason, this paper proposes a new approach for analysing classifiers based on two measures: C and sensitivity (S) (i.e., the minimum of accuracies obtained for each class). These measures are optimised through a two-stage evolutionary process. It was conducted by applying two sequential fitness functions in the evolutionary process, including entropy (E) for the first stage and a new fitness function, area (A), for the second stage. By using these fitness functions, the C level was optimised in the first stage, and the S value of the classifier was generally improved without significantly reducing C in the second stage. This two-stage approach improved S values in the generalisation set (whereas an evolutionary algorithm (EA) based only on the S measure obtains worse S levels) and obtained both high C values and good classification levels for each class. The methodology was applied to solve 16 benchmark classification problems and two complex real-world problems in analytical chemistry and predictive microbiology. It obtained promising results when compared to other competitive multi-class classification algorithms and a multi-objective alternative based on E and S. [Copyright &y& Elsevier]

Abstract: Model trees are a particular case of decision trees employed to solve regression problems. They have the advantage of presenting an interpretable output, helping the end-user to get more confidence in the prediction and providing the basis for the end-user to have new insight about the data, confirming or rejecting hypotheses previously formed. Moreover, model trees present an acceptable level of predictive performance in comparison to most techniques used for solving regression problems. Since generating the optimal model tree is an NP-Complete problem, traditional model tree induction algorithms make use of a greedy top-down divide-and-conquer strategy, which may not converge to the global optimal solution. In this paper, we propose a novel algorithm based on the use of the evolutionary algorithms paradigm as an alternate heuristic to generate model trees in order to improve the convergence to globally near-optimal solutions. We call our new approach evolutionary model tree induction (E-Motion). We test its predictive performance using public UCI data sets, and we compare the results to traditional greedy regression/model trees induction algorithms, as well as to other evolutionary approaches. Results show that our method presents a good trade-off between predictive performance and model comprehensibility, which may be crucial in many machine learning applications. [Copyright &y& Elsevier]

Abstract: Stacking is a general ensemble method in which a number of base classifiers are combined using one meta-classifier which learns their outputs. Such an approach provides certain advantages: simplicity; performance that is similar to the best classifier; and the capability of combining classifiers induced by different inducers. The disadvantage of stacking is that on multiclass problems, stacking seems to perform worse than other meta-learning approaches. In this paper we present Troika, a new stacking method for improving ensemble classifiers. The new scheme is built from three layers of combining classifiers. The new method was tested on various datasets and the results indicate the superiority of the proposed method to other legacy ensemble schemes, Stacking and StackingC, especially when the classification task consists of more than two classes. [Copyright &y& Elsevier]

Abstract: In real life, information about the world is uncertain and imprecise. The cause of this uncertainty is due to: deficiencies on given information, the fuzzy nature of our perception of events and objects, and on the limitations of the models we use to explain the world. The development of new methods for dealing with information with uncertainty is crucial for solving real life problems. In this paper three interval type-2 fuzzy neural network (IT2FNN) architectures are proposed, with hybrid learning algorithm techniques (gradient descent backpropagation and gradient descent with adaptive learning rate backpropagation). At the antecedents layer, a interval type-2 fuzzy neuron (IT2FN) model is used, and in case of the consequents layer an interval type-1 fuzzy neuron model (IT1FN), in order to fuzzify the rule’s antecedents and consequents of an interval type-2 Takagi–Sugeno–Kang fuzzy inference system (IT2-TSK-FIS). IT2-TSK-FIS is integrated in an adaptive neural network, in order to take advantage the best of both models. This provides a high order intuitive mechanism for representing imperfect information by means of use of fuzzy If–Then rules, in addition to handling uncertainty and imprecision. On the other hand, neural networks are highly adaptable, with learning and generalization capabilities. Experimental results are divided in two kinds: in the first one a non-linear identification problem for control systems is simulated, here a comparative analysis of learning architectures IT2FNN and ANFIS is done. For the second kind, a non-linear Mackey–Glass chaotic time series prediction problem with uncertainty sources is studied. Finally, IT2FNN proved to be more efficient mechanism for modeling real-world problems. [Copyright &y& Elsevier]