Search

Proportional–Integral–Derivative controller technique continues to provide the easiest and effective solutions to most of the industrial applications in recent years. However PID controller is poorly tuned in practice compared to most other tuning methods and is complicated with poor performance. This research presents a multi objective optimization approach involving Genetic Algorithm, Evolutionary Programming, Particle Swarm Optimization and Bacterial foraging optimization. The proposed multi objective optimization algorithm is used to tune the PID controller parameters and their performances have been compared with the conventional methodologies like Ziegler Nichols method. The results proved that the use of multi objective optimization approach based controller tuning improves the performance of process in terms of time domain specifications and performance index, set point tracking and regulatory changes and also provides stability. This paper describes the various multi objective optimization algorithms and its implementation to tune the PID Controller used in paper machine DCS as real time processing of a Pulp and paper industry processes.

Identifying the key risk factors of disease from a large amount of clinical data is a prerequisite for further scientific decision-making. In medical practice, the clinical symptom information of patients usually includes various types of data. Meanwhile, the occurrence and development of diseases are joint result of the mutual influence factors. Therefore, there is usually a correlation between attributes. In this paper, we discuss a kind of hybrid attribute feature selection problem considering the correlation between attributes. Firstly, we take the identification of disease pathogenic factors in medical decision as the background, and construct a hybrid attribute decision system. Secondly, by introducing kernel alignment, the uncertain relationship between attributes is defined. Based on this, a three-way clustering model in attribute space is established. Furthermore, a feature selection method for hybrid attribute data based on three-way clustering in attribute space is proposed. Finally, we applied the proposed model to identify the pathogenic factors of stroke and used 279 clinical random samples for simulation analysis. The results verified the applicability and validity of the model. The main contributions of this paper include two aspects. In terms of theory, by introducing kernel alignment, a three-way clustering algorithm in attribute space is established. Meanwhile, a hybrid attribute feature selection method based on three-way clustering is proposed. In terms of application, the proposed method is applied to identify risk factors of stroke.

In recent years, tensor data appear more frequently in machine learning. Tucker decomposition is a powerful tool for processing tensor data. However, from the perspective of dimensionality reduction, Tucker decomposition without any regularization is only a tensor version of principal component analysis (PCA) that learning a linear subspace to minimize the distance between the data and their projections on the subspace. However, many high-dimensional tensor data usually reside in low-dimensional submanifolds, rather than low-dimensional linear subspaces. It is necessary to fill the gap between submanifolds and linear subspaces to achieve superior performance. In this paper, we proposed a new dimensionality reduction algorithm of tensor data based on manifold regularized Tucker decomposition (called MRTD for short), in which manifold regularization is in addition to Tucker decomposition. Furthermore, unlike many other similar algorithms that ignore the matrices of Tucker decomposition by absorbing them into a whole matrix, in this paper, we proposed an iterative solution to MRTD that the matrices are calculated based on each other iteratively. Matrices in Tucker decomposition represent different dimensionality reduction operations for each dimension of tensor. Therefore, MRTD is a dimensionality reduction algorithm specially designed for tensor data, not for vector data. The experimental results of MRTD and 5 other related state-of-the-art algorithms on 6 commonly-used real-world datasets are presented, showing the better performance of MRTD.

Hierarchical reinforcement learning (HRL) is a promising method, which decomposes complex tasks into a series of sub-tasks. However, at present, most HRL methods have slow convergence speed and are difficult to be widely applied to such scenarios in real life. In this paper, we propose an efficient hierarchical reinforcement learning algorithm with fuzzy rules (HFR), a novel framework for integrating human prior knowledge with hierarchical policy network, which can effectively accelerate the optimization of policy. The model presented in this paper uses fuzzy rules to represent the human prior knowledge, making the rules trainable because of the derivability of the fuzzy rules. In addition, a switch module that adaptively adjusts the decision-making frequency of the upper-level policy is proposed to solve the limitation of manual tuning. Experiment results demonstrate that HFR has a faster convergence rate than the current state-of-the-art HRL algorithms, especially in complex scenarios, such as robot control tasks.

In emotional computing related IoT system, emotional sensors, as the IoT devices, are usually deployed to collect the emotional data from humans. The IoT devices need wireless connections to send the collected data to the server, that conducts the prediction to give user instructions. Mobile edge computing (MEC) is a promising technology to fit into this scenario. However, the IoT devices are usually short of energy supply and the local computation gives less accurate emotional computing results. To solve the problem, this paper intends to maximize the total energy efficiency of communication and computation within the MEC servers and sensors by jointly optimizing the allocation of channels and computing resources. The formulated problem is non-convex and usually solved through the successive convex approximation (SCA) method. Compared to SCA, deep Q network (DQN) method is used in this paper, which involves less computation cost to be more practically deployed. The simulation results show that the DQN solution outperforms the other benchmarking solutions, and the total energy consumption of the system is effectively reduced with a guaranteed emotional computing accuracy.

Federated learning (FL) is an emerging distributed artificial intelligence (AI) algorithm. It can train a global model with multiple participants and at the same time ensure the privacy of the participants’ data. Thus, FL provides a solution for the problems faced by data silos. Existing federated learning algorithms face two significant challenges when dealing with (1) non-independent and identically distributed (non-IID) data, and (2) data with noise or without preprocessing. To address these challenges, a novel federated learning approach based on the confidence of federated Kalman filters is proposed and is referred to as FedCK in this paper. Firstly, this paper proposes a deep Generative Adversarial Network with an advanced auxiliary classifier as a pre-training module. The Non-IID increases the discreteness of the parameters of local models, it is difficult for FL to aggregate an excellent global model. The pre-training module proposed in this paper can deeply mine hidden features and increase the correlation between local model parameters. Secondly, a federated learning framework based on Federated Kalman Filter (FKF) is proposed in this paper. Because the general federation average aggregation algorithm cannot identify the model parameters with noise. This paper uses the idea of FKF to propose a set of adaptive confidence to improve the fault tolerance of FL. Experiments carried out on the MNIST, CIFAR-10 and SVHN datasets demonstrate that FedCK has better robustness and accuracy than classical federated learning methods.

A decision-making (DM) agent models its environment and quantifies its DM preferences. An adaptive agent models them locally nearby the realisation of the behaviour of the closed DM loop. Due to this, a simple tool set often suffices for solving complex dynamic DM tasks. The inspected Bayesian agent relies on a unified learning and optimisation framework, which works well when tailored by making a range of case-specific options. Many of them can be made off-line. These options concern the sets of involved variables, the knowledge and preference elicitation, structure estimation, etc. Still, some meta-parameters need an on-line choice. This concerns, for instance, a weight balancing exploration with exploitation, a weight reflecting agent’s willingness to cooperate, a discounting factor, etc. Such options influence, often vitally, DM quality and their adaptive tuning is needed. Specific ways exist, for instance, a data-dependent choice of a forgetting factor serving to tracking of parameter changes. A general methodology is, however, missing. The paper opens a pathway to it. The solution uses a hierarchical feedback exploiting a generic, DM-related, observable, mismodelling indicator. The paper presents and justifies the theoretical concept, outlines and illustrates its use.

To deal with semi-supervised feature selection tasks, this paper presents a recursive feature retention (RFR) method based on a neighborhood discriminant index (NDI) method (a supervised feature selection method) and a forward iterative Laplacian score (FILS) method (an unsupervised method), where FILS is designed specially for RFR. The goal of RFR is to determine an optimal feature subset that has not only a high discriminant ability but also a strong ability to maintain the local structure of data. The discriminant ability of a feature is measured by NDI, and the ability of a feature to maintain the local structure of data is described by FILS. RFR compromises these two scores to give a balanced score for a feature. RFR iteratively selects a feature with the smallest balanced score and moves it into the current optimal feature subset. This paper also shows theoretical analysis to speed up iterations. Extensive experiments are conducted on toy and real-world data sets. Experimental results confirm that RFR can achieve a better performance compared with the state-of-the-art semi-supervised methods.

Network representation learning (NRL) maps vertices into latent vector space for further network inference. The existing algorithms concern more about whether the vectors of two similar nodes be close in latent vector space while the hierarchy proximity has been largely neglected by them. The distribution of the representation vectors needs to reflect the hierarchical structural properties which widely exist in networks. In this paper, we propose a novel network representation learning framework that can encode the interpretable hierarchical structural semantics into the representation vectors. Specifically, we measure the distance and importance degree of nodes in the original network and map the nodes to a tree space. This makes the hierarchical structural relations in the original network be clearly revealed by the tree which is also of good interpretability. In this paper, the local structural proximities and the interpretable hierarchy knowledge are encoded into vector space by optimizing the objective function. Extensive experiments conducted on the realistic data sets demonstrate that the proposed approach outperforms the existing state-of-the-art approaches on tasks of node classification, link prediction, and visualization. Finally, a case study is conducted for further analysis about how the proposed model works.

Big data is usually massive, diverse, time-varying, and high-dimensional. The focus of this paper is on the domain description of big data, which is the basis for solving the above problems. This paper has three main contributions. Firstly, one hyperellipsoid model is proposed to analyze domain description of big data. The parameters of the hyperellipsoid model can be adaptively adjusted according to the proposed objective function without relying on manual parameter selection, which expands the application range of the model. Secondly, an improved FDPC algorithm is proposed to generate multiple hyperellipsoid models to approximate the spatial distribution of big data, thus improving the accuracy of domain description. Multiple hyperellipsoid models can not only greatly eliminate the spatial redundancy of the domain description based on one hyperellipsoid model, but also provide a feasible method for describing complex spatial distribution. Thirdly, an online domain description algorithm based on hyperellipsoid models is proposed, which improves the robustness of hyperellipsoid models on time-varying data. The parallel processing flow of the algorithm is given. In the experiment, synthetic instances and real-world datasets were applied to test the performance of hyperellipsoid models. By comparing LOF, OneClassSVM, SVDD and isolation forest, the performance of the proposed method is competitive and promising.

Formal concept analysis has shown much potential in supporting personalized learning. This paper is motivated by several application scenarios of concept lattices in supporting mathematics education. The aim of the paper is to develop a scheme for designing problem-chains based on concept lattices and provide evaluation methods towards steadiness of the corresponding learning processes. Particularly, a model is developed to study the sensitivity of conversion from effort on problems to efficacy on knowledge/skills. Then the notion of successive trails is introduced to describe means-end chains of learning states. Towards steadiness of the corresponding learning processes, evaluation methods are developed and data experiments are conducted from aspects of effort on problems, efficacy on knowledge/skills and conversion sensitivity. Moreover, the notion of conceptual paths is introduced in order to describe personalized learning strategies. Subsequently, a scheme for designing problem-chains is given based on conceptual paths and similarity measuring. Finally, a method of evaluating the steadiness of problem-chains is proposed by taking advantage of linear regression analysis on problem-chains.

The main purpose of this paper is to provide a solution, through which one can efficiently reduce the population of cancer cells by injecting the lowest dose of the drug; therefore, reducing the side effects of the drug on healthy cells. In this paper, a mathematical model of stem Chronic Myelogenous Leukemia (CML) is used. To this aim, a hybrid method is used, that is a combination of the Eligibility Traces algorithm and Neural Networks. The eligibility traces algorithm is one of the well-known methods for solving problems under the Reinforcement Learning (RL) approach. The reason is that the population of cancer cells can be controlled with a higher accuracy and will have a significant impact on dosage of injection. The eligibility traces algorithm has the advantage of backward view, meaning it will investigate previous states, as well. That will result in improving the learning procedure, speed of reduction in cancer cells population and the total dosage of the injected drug during the treatment period, in patients with CML. Combination of the mentioned method and neural networks has provided continuous states in the considered problem. Hence, there will be no limitation for considering all possible states for solving the problem. Moreover, this can accelerate obtaining the optimal dosage with a high accuracy, which is a significant advantage of the proposed method. To show the effectiveness of the proposed method to control the population of cancer cells and obtaining the optimal dosage, it is compared with four different cases: when only the eligibility traces algorithm is employed, in the case only the Q-learning algorithm is used, when the Optimal Control is applied and in the case no dosage is injected. Finally, it is revealed that the combinatory method of the eligibility traces algorithm and neural networks can control the population of cancer cells more quickly, with a higher accuracy as well as applying a lower dosage of the drug.

Integrated modular avionics is the core system of modern aircraft, which hosts almost all kinds of electrical functions. The performance of integrated modular avionics has an immediate influence on flight mission. Remaining useful life prediction is an effective manner to guarantee the safety and reliability of airplane. To satisfy the real-time requirement of integrated modular avionics, the prediction algorithm should have fast learning speed. This paper proposes an ensemble enhanced online sequential parallel extreme learning machine to predict the remaining useful life of integrated modular avionics. Firstly, a network with parallel hidden layers is designed to improve feature extraction. Secondly, to enhance the learning stability, the input weights of the network are determined by using extreme learning machine autoencoder. Thirdly, an updating method is developed for online prediction and an adaptive weight is designed to construct the ensemble online sequential prediction method. The effectiveness and superiority of the proposed method are verified through the standard datasets. Finally, this paper regards intermittent faults as the feature of integrated modular avionics and builds a degradation model by using Levy Process. The proposed method is applied to remaining useful life prediction of integrated modular avionics.

This paper studies the fuzzy hierarchical multimodal hub location problem for cargo delivery systems. It differs from traditional hub location problem in two ways. First, this paper constructs a hierarchical multimodal hub-and-spoke distribution network for the cargo delivery systems, which involves two transportation modes (road and air), two types of hubs (ground and airport) and three corresponding layers. Second, this paper develops a credibility-based fuzzy programming model capturing the uncertainty in travel time and handling time of the cargo delivery systems. This new model aims to minimize the latest arrival time (travel time plus handling time) for delivering cargoes from each pair of origin and destination nodes under diverse credibility of chance constraints. Under mild assumptions, the original model can be turned into an equivalent deterministic integer linear programming model. However, even for small instances of the problem, the equivalent model becomes too hard to be tackled by a general solver, e.g., CPLEX. This fact motivates the development of a two-stage heuristic procedure, wherein the first stage for the hub location subproblem is solved by a variable neighborhood search algorithm. These location solutions are then embedded into the second-stage process for the link assignment subproblem based on a shortest path method. To verify the proposed model and method, extensive numerical experiments are conducted on the well-known Turkish network data set.

In this paper, a novel feature selection method is introduced to tackle the problem of high-dimensional features in the text clustering application. Text clustering is a prevailing direction in big text mining; in this manner, documents are grouped into cohesive groups by using neatly selected informative features. Swarm-based optimization techniques have been widely used to select the relevant text features and shown promising results on multi-sized datasets. The performance of traditional optimization algorithms tends to fail miserably when using large-scale datasets. A novel parallel membrane-inspired framework is proposed to enhance the performance of the krill herd algorithm combined with the swap mutation strategy (MHKHA). In which the krill herd algorithm is hybridized the swap mutation strategy and incorporated within the parallel membrane framework. Finally, the k-means technique is employed based on the results of feature selection-based Krill Herd Algorithm to cluster the documents. Seven benchmark datasets of various characterizations are used. The results revealed that the proposed MHKHA produced superior results compared to other optimization methods. This paper presents an alternative method for the text mining community through cohesive and informative features.

Clinical quantitative information contains crucial measurable expressions of patients’ diseases and treatment conditions, which are commonly exist in free-text electronic medical records. Although the clinical quantitative information is of considerable significance in assisting the analysis of health care, few researches have yet focused on the topic and it remains an ongoing challenge. Focusing on Chinese electronic medical records, this paper proposed an extended Bi-LSTM-CRF model, which integrated domain knowledge information and position characteristics of quantitative information as external features to improve the effectiveness of clinical quantitative information recognition. In addition, to associate the extracted entities and quantities more effectively, this paper presented an automatic approach for entity-quantity association using machine learning strategy. Based on 1359 actual Chinese electronic medical records from burn department of a domestic public hospital, we compared our model with a number of widely-used baseline methods. The evaluation results showed that our model outperformed the baselines with an F1-measure of 94.27% for quantitative information recognition and an accuracy of 94.60% for entity-quantity association, demonstrating its effectiveness.

5G cellular networks are expected to be the key infrastructure to deliver the emerging services. These services bring new requirements and challenges that obstruct the desired goal of forthcoming networks. Mobile operators are rethinking their network design to provide more flexible, dynamic, cost-effective and intelligent solutions. This paper starts with describing the background of the 5G wireless networks then we give a deep insight into a set of 5G challenges and research opportunities for machine learning (ML) techniques to manage these challenges. The first part of the paper is devoted to overview the fifth-generation of cellular networks, explaining its requirements as well as its key technologies, their challenges and its forthcoming architecture. The second part is devoted to present a basic overview of ML techniques that are nowadays applied to cellular networks. The last part discusses the most important related works which propose ML solutions in order to overcome 5G challenges.

Learning low-dimensional vector representations of words from a large corpus is one of the basic tasks in natural language processing (NLP). The existing universal word embedding model learns word vectors mainly through grammar and semantic information from the context, while ignoring the sentiment information contained in the words. Some approaches, although they model sentiment information in the reviews, do not consider certain words in different domains. In a case where the emotion changes, if the general word vector is directly applied to the review sentiment analysis task, then this will inevitably affect the performance of the sentiment classification. To solve this problem, this paper extends the CBoW (continuous bag-of-words) word vector model and proposes a cross-domain sentiment aware word embedding learning model, which can capture the sentiment information and domain relevance of a word at the same time. This paper conducts several experiments on Amazon user review data in different domains to evaluate the performance of the model. The experimental results show that the proposed model can obtain a nearly 2% accuracy improvement compared with the general word vector when modeling only the sentiment information of the context. At the same time, when the domain information and the sentiment information are both included, the accuracy and Macro-F1 value of the sentiment classification tasks are significantly improved compared with existing sentiment word embeddings.

In this paper, our focus is on using a rule-based approach to develop agents with artificial general intelligence. In rule-based systems, developing effective rules is a huge challenge, and coding rules for agents requires a large amount of manual work. Association rules mining (ARM) can be used for discovering specific rules from data sets and determining relationships between data sets. In this paper, we introduce a modified ARM method and use it to discover rules that analyse the surrounding environment and determine movements for an agent-guided vehicle that has been designed to achieve autonomous parking. The rules are created by our ARM-based method from training data gained during manual training in customised parking scenarios. In this system, data are represented in terms of fuzzy symbolic elements. We have tested our system by simulation in a virtual environment to demonstrate the effectiveness of this new approach.

Attribute reduction in formal decision contexts has become one of the key issues in the research and development of formal concept analysis (FCA) and its applications. As far as we know, however, most of the existing reduction methods for formal decision contexts are time-consuming especially for the large-scale data. This paper investigates the attribute reduction method for large-scale formal decision contexts. The computation of a discernibility matrix is an important step in the development of the corresponding reduction method. A simple and powerful method to efficiently calculate the discernibility matrix of formal decision contexts is first presented. In addition, a heuristic algorithm for searching the optimal reduct is then proposed. Thirdly, as an application of the new results, we discuss the problem of finding the minimal subbases of finite topological spaces. It has shown that the method of attribute reduction in formal decision contexts can be used to obtain all the minimal subbases of a finite topological space. Furthermore, we present an algorithm for computing the minimal subbase of a topological space, based on the attribute reduction method proposed in this paper. Finally, two groups of experiments are carried out on some large-scale data sets to verify the effectiveness of the proposed algorithms.

As an emerging and applicable method, deep learning (DL) has attracted much attention in recent years. With the development of DL and the massive of publications and researches in this direction, a comprehensive analysis of DL is necessary. In this paper, from the perspective of bibliometrics, a comprehensive analysis of publications of DL is deployed from 2007 to 2019 (the first publication with keywords “deep learning” and “machine learning” was published in 2007). By preprocessing, 5722 publications are exported from Web of Science and they are imported into the professional science mapping tools: VOS viewer and Cite Space. Firstly, the publication structures are analyzed based on annual publications, and the publication of the most productive countries/regions, institutions and authors. Secondly, by the use of VOS viewer, the co-citation networks of countries/regions, institutions, authors and papers are depicted. The citation structure of them and the most influential of them are further analyzed. Thirdly, the cooperation networks of countries/regions, institutions and authors are illustrated by VOS viewer. Time-line review and citation burst detection of keywords are exported from Cite Space to detect the hotspots and research trend. Finally, some conclusions of this paper are given. This paper provides a preliminary knowledge of DL for researchers who are interested in this area, and also makes a conclusive and comprehensive analysis of DL for these who want to do further research on this area.

In this paper, the nonlinear multi-agent system which contains uncertainty and is controlled by state-constraint impulsive protocol is taken into consideration. For the uncertainty of the multi-agent system, it is replaced by fuzzy logic system approximately and a judgement strategy which only contains the relative information with neighbors is proposed in this paper. In order to do research in state-constraint impulsive protocol, three kinds of impulsive control protocols which conclude partial input saturation, double actuator saturation and single actuator saturation are discussed. Then, some sufficient conditions of the system are obtained to reach consensus. Finally, some numerical simulation examples are provided to prove the effectiveness of the theoretical analysis.

Distance/Similarity learning is a fundamental problem in machine learning. For example, kNN classifier or clustering methods are based on a distance/similarity measure. Metric learning algorithms enhance the efficiency of these methods by learning an optimal distance function from data. Most metric learning methods need training information in the form of pair or triplet sets. Nowadays, this training information often is obtained from the Internet via crowdsourcing methods. Therefore, this information may contain label noise or outliers leading to the poor performance of the learned metric. It is even possible that the learned metric functions perform worse than the general metrics such as Euclidean distance. To address this challenge, this paper presents a new robust metric learning method based on the Rescaled Hinge loss. This loss function is a general case of the popular Hinge loss and initially introduced in Xu et al. (Pattern Recogn 63:139–148, 2017) to develop a new robust SVM algorithm. In this paper, we formulate the metric learning problem using the Rescaled Hinge loss function and then develop an efficient algorithm based on HQ (Half-Quadratic) to solve the problem. Experimental results on a variety of both real and synthetic datasets confirm that our new robust algorithm considerably outperforms state-of-the-art metric learning methods in the presence of label noise and outliers.

Despite the benefits mentioned in previous works of automatically acquiring skills for using them in hierarchical reinforcement learning algorithms such as solving the curse of dimensionality, improving exploration, and speeding up value propagation, they have not paid much attention to evaluating the effect of each skill on these factors. In this paper, we show that depending on the given task, a skill may be useful for learning it or not. In addition, the focus of the related work of automatically acquiring skills is on detecting subgoals, i.e., the skill termination condition, but there is not a precise method for extracting the initiation set of skills. In this paper, we propose not only two methods for evaluating skills but also two other methods for pruning the initiation set of them. Experimental results show significant improvements in learning different test domains after evaluating and pruning skills.

This paper researches the value of the margin distribution in binary classifier. The central idea of large margin distribution machine (LDM) is to optimize the margin distribution, such as maximizing the margin mean and minimizing the margin variance. Compared to support vector machine (SVM), LDM demonstrates the good generalization performance. In order to improve the generalization performance of twin support vector machine (TSVM), a twin support vector machine based on adjustable large margin distribution (ALD-TSVM) is proposed in this paper. Firstly, the margin distribution is redefined to construct a pair of adjustable supporting hyperplanes. Then, the redefined margin distribution is introduced onto TSVM to obtain the models of ALD-TSVM, including linear case and nonlinear case. ALD-TSVM is a general learning method which can be used in any place where TSVM and LDM can be applied. Finally, the novel method is compared with other classification algorithms by doing experiments on toy dataset, UCI datasets and image datasets. The experimental results show that ALD-TSVM obtains better classification performance.

Recently, heterogeneous information network (HIN) embedding is wide studied due to its various applications. In general, network embedding is a way of representation network’s nodes into a low-dimensional space. Most of previous embedding techniques concentrate on the homogeneous networks only in which all nodes are considered as a single type. Heterogeneous network embedding is a challenging problem due to the complexity of different node’s types and link’s types. Recent heterogeneous network embedding studies are based on meta-path and meta-graph to guide the random walks over the networks. These heterogeneous network embedding approaches outperform state-of-the-art homogeneous embedding models in multiple heterogeneous network mining tasks. However, recent meta-graph-based approaches are ineffective in capturing topic similarity between nodes. There is no doubt that most of common HINs (DBLP, Facebook, etc.) are rich-text which contain many text-based nodes, such as paper, comment, post, etc. In this paper, we propose a novel embedding approach, namely W-MetaGraph2Vec. The W-MetaGraph2Vec uses the topic-driven meta-graph-based random walk mechanism in weighted HIN to guide the generation of heterogeneous neighborhood of a node. Extensive experiments on real-world datasets demonstrate that our proposed model not only leverage HIN mining tasks, such as node similarity search, clustering, classification, etc. in performance accuracy but also discern the problems of topic relevance between text-based nodes.

With the fast developments of three-way decisions (3WD), this paper systematically summarizes the development track and evolution process of 3WD in recent decades. Firstly, the historical context, internal connections and relations between 3WD and rough sets are carefully investigated. Then, we discuss the methodology of 3WD via granular computing with “multi-level” strategy and “multi-view” strategy. Two novel 3WD generalized models with multilevel structure and multiview structure, as well as an integration framework of multilevel and multiview, are analyzed and discussed detailedly. Finally, this paper presents the research status and future research topics of 3WD.

Psychological stress may cause various health problems. To prevent the potential chronic illness that long-term psychological stress could cause, it is important to detect and monitor the psychological stress at its initial stage. In this paper, we present a framework for remotely detecting and classifying human stress by using a KINECT sensor that is portable and affordable enough for ordinary users in everyday life. Unlike most of emotion recognition tasks in which respiratory signals (RSPS) are usually used only as an aiding analysis, the whole task proposed is based on RSPS. Thus, the main contribution of this paper is that not only the non-contact devices is used to identify human stress, but also the relationship between RSPS and stress recognition is analyzed in detail. Experimental results on 84 volunteers show that the recognition accuracy for recognizing psychological stress, physical stress, and relaxing state are 93.90%, 93.40%, and 89.05% respectively. These results suggest that the proposed framework is effective for monitoring human stress, and RSPS could be used for stress recognition.

In this paper, complex networks are used to analyze the dataset of three-way decision articles published before December 18, 2019 and downloaded from ISI Web of Science. The scientific collaboration network, university collaboration network, networks of scientific papers (i.e., citation network, bibliographic coupling network, co-citation network) and keywords network are constructed to reveal the relationships between authors, affiliations, papers and keywords, respectively. Some interesting results are obtained and used to answer the following questions: (1) which authors play a key role in developing three-way decision; (2) which affiliations actively promote the development of three-way decision; (3) which papers are important or influential in the field of three-way decision; (4) what are the closely related research issues around three-way decision.

Due to the discovery of innovative and practical neural architectures, deep learning has achieved bright successes in many fields, such as computer vision, natural language processing, recommendation systems, etc. To reach high performance, researchers have to adjust neural architectures and choose training tricks very carefully. The manual trial-and-error process for discovering the best neural network configuration consumes plenty of manpower. The neural architecture search (NAS) aims to alleviate this issue by automatically configuring neural networks. Recently, the rapid development of NAS has shown significant achievements. Novel neural network architectures that outperform the state-of-the-art handcrafted networks have been discovered in image classification benchmarks. In this paper, we survey NAS from a technical view. By summarizing the previous NAS approaches, we drew a picture of NAS for readers including problem definition, search approaches, progress towards practical applications and possible future directions. We hope that this paper can help beginners start their researches on NAS.

Recently, many feature selection algorithms based on non-negative matrix factorization have been proposed. However, many of these algorithms only consider unilateral information about global or local geometric structure normally. To this end, this paper proposes a new feature selection algorithm called double feature selection algorithm based on low-rank sparse non-negative matrix factorization (NMF-LRSR). Firstly, to reduce the dimensions effectively, NMF-LRSR uses non-negative matrix factorization as the framework to further reduce the dimension of the feature selection which is originally a dimension reduction problem. Secondly, the low-rank sparse representation with the self-representation is used to construct the graph, so both the global and intrinsic geometric structure information of the data could be taken into account in the process of feature selection, which makes full use of the information and makes the feature selection more accurate. In addition, the double feature selection theory is used to this paper, which makes the result of feature selection more accurate. NMF-LRSR is tested on the baseline and the other six algorithms in the literature and evaluated them on 11 publicly available benchmark datasets. Experimental results show that NMF-LRSR is more effective than the other six feature selection algorithms.

COVID-19 has resulted in a significant impact on individual lives, bringing a unique challenge for face retrieval under occlusion. In this paper, an occluded face retrieval method which consists of generator, discriminator, and deep hashing retrieval network is proposed for face retrieval in a large-scale face image dataset under variety of occlusion situations. In the proposed method, occluded face images are firstly reconstructed using a face inpainting model, in which the adversarial loss, reconstruction loss and hash bits loss are combined for training. With the trained model, hash codes of real face images and corresponding reconstructed face images are aimed to be as similar as possible. Then, a deep hashing retrieval network is used to generate compact similarity-preserving hashing codes using reconstructed face images for a better retrieval performance. Experimental results show that the proposed method can successfully generate the reconstructed face images under occlusion. Meanwhile, the proposed deep hashing retrieval network achieves better retrieval performance for occluded face retrieval than existing state-of-the-art deep hashing retrieval methods.

Federated learning (FL) is a secure distributed machine learning paradigm that addresses the issue of data silos in building a joint model. Its unique distributed training mode and the advantages of security aggregation mechanism are very suitable for various practical applications with strict privacy requirements. However, with the deployment of FL mode into practical application, some bottlenecks appear in the FL training process, which affects the performance and efficiency of the FL model in practical applications. Therefore, more researchers have paid attention to the challenges of FL and sought for various effective research methods to solve these current bottlenecks. And various research achievements of FL have been made to promote the intelligent development of all application areas with privacy restriction. This paper systematically introduces the current researches in FL from five aspects: the basics knowledge of FL, privacy and security protection mechanisms in FL, communication overhead challenges and heterogeneity problems of FL. Furthermore, we make a comprehensive summary of the research in practical applications and prospect the future research directions of FL.

Robust principal component analysis (RPCA) based methods via decomposition into low-rank plus sparse matrices offer a wide range of applications for image processing, video processing and 3D computer vision. Most of the time the observed imagery data is often arbitrarily corrupted by anything such as large sparse noise, small dense noise and other unknown fraction, which we call mixed noise in this paper. However, low rank matrix recovery by RPCA is born for the existence of large sparse noise, so its performance and applicability are limited in the presence of mixed noise. In this paper, a generalized robust principal component analysis with norm $$l_{2,1}$$ model is proposed to solve the problem of low-rank matrix recovery under mixed large sparse noise and small dense noise. The corrupted matrix is written as a combination that minimizes the nuclear norm, the 1-norm and the norm $$l_{2,1}$$, which has high efficiency, flexibility and robustness for low rank matrix recovery from mixed noise. Then a novel and efficient algorithm called random permutation alternative direction of multiplier method is applied to solve the model. Experiments with simulations and real datasets demonstrate efficiency and robustness of this model and algorithm.

Support vector machine (SVM) is a widely used algorithm in the field of machine learning, and it is a research hotspot in the field of data mining. In order to fully understand the historical progress and current situation of SVM researches, as well as its future development trend in China, this paper conducts a comprehensive bibliometric study based on the publications from web of science database by Chinese scholars in this field. First, this paper focuses on some of the basic characteristics of the research publications of SVM in China, including important journals, research institutions and countries/regions, most cited publications, and so on. Then, based on the knowledge mapping software VOSviewer, the cooperation between other countries and China as well as the cooperation between research institutions in China are explored. Finally, VOSviewer based bibliometric visualization graphics are used to identify the changes of the research hotspots in the SVM field. This paper provides a relatively broad perspective for the evaluation of SVM scientific researches, and reveals the development trend in this field.

In some information system with order features, when users consider “greater than” or “less than” relations to a certain degree rather than in the full sense, using traditional methods may face great limitations. In light of natural connections among concept lattice, inclusion degree, order relations, and the feasibility of mutual integration among the three (concept lattice is essentially a type of data analysis tool using binary relations as research objects, while inclusion degree is a type of powerful tool for measuring uncertain order relations), the paper attempts to analyze uncertain order relations quantitatively within the framework of integration theory of concept lattice and inclusion degree. By which, the research scope of order relations undergoes an expansion-to-contraction process. Namely, certain order relations are first expanded to fuzzy or uncertain relations, and then the fuzzy or uncertain relations are allowed to contract to a degree of certainty by setting threshold parameters. Clearly, by properly widening the research scope of order relations, the model not only has good robustness and generalization ability, but also can meet actual needs flexibly. On this basis, solutions for algebraic structure, reduction, core, dependency, et al. are further studied deeply in ordered information systems. In short, the paper, as a meaningful try and exploration, is conducive to the integration of theories, and may offer some new and feasible ways for the study of order relations and ordered information systems.

Imbalanced data classification is a challenging problem in the field of machine learning. The problem occurs when data samples have an uneven distribution amongst the classes and classical classifiers are not suitable for classifying such datasets. To overcome this problem, in this paper, the best training samples are selected from data samples with the goal of improving the performance of classifier when dealing with imbalanced data. To do so, some heuristic methods are presented which use local information to give a proper view about whether removing or retaining each sample of training set. Subsequently, the methods are considered as local search algorithms and combined with a global search algorithm in a framework to form memetic algorithms. The global search used in this paper is binary quantum inspired gravitational search algorithm (BQIGSA) which is a new metaheuristic search for optimization of binary encoded problems. BQIGSA is employed since we seek for a highly stochastic and random search algorithm to solve our problem. We propose to use six different local search algorithms, three of which are application oriented that we designed based on the problem and the rest are general, and the best local search is then determined. Experiments are performed on 45 standard datasets, and G-mean and AUC criteria are considered as evaluation tools. Then, the data sets are employed to compare the best memetic approaches with some popular state of the art algorithms as well as a recently proposed memetic algorithm and the results show their superiority. At the last step, the performance of the proposed algorithm for four different classifiers is evaluated and the best classifier is determined to be utilized for this method.

It is difficult for the bat algorithm (BA) to retain good performance with increasing problem complexity and problem. In this paper, an ensemble BA is proposed to solve large-scale optimization problems (LSOPs) by introducing the integration ideas. The characteristics of six improved BA strategies are taken into account for the ensemble strategies. To fuse these strategies perfectly, the probability selection mechanisms, including the constant probability and dynamic probability, are designed by adjusting the odds of different strategies. To verify the performance of the algorithm in this paper, the proposed algorithm is applied to solve numerical optimization problems on benchmark functions with different dimensions. Then, the best ensemble BA is selected by comparing the constant probabilities and dynamic probabilities. The selected algorithm is compared with other excellent swarm intelligence optimization algorithms. Additionally, the superiority of the proposed algorithm is confirmed for solving LSOPs.

Selecting excellent supplier is the foundation of establishing efficient supply chain. This paper presents a novel hybrid model for supplier selection decision making problem by combining double quantitative fuzzy rough set and analytic hierarchy process (AHP), namely $$Fuzzy-AHP$$. we first transform the supplier decision-making into a rough-approximation problem in the double quantization decision approximation space with fuzzy decision objects, and then construct a new supplier selection decision model and method based on double-quantitative fuzzy rough sets. Then, the double quantitative fuzzy rough set is utilized to calculate the upper and lower approximations of the fuzzy decision object in the quantitative approximation space. Furthermore, the lower and upper approximations are applied to establish the pairwise comparison matrix and analytic hierarchy process is employed to rank these suppliers comprehensively. Finally, an experiment study with six ERP bidder in an Indian mining behemoth is carried out in this paper. The results reveal that the proposed technique can select effective suppliers, but also realize a comprehensive ranking. This research has enriched the methodology of supplier evaluation and selection, as well as owns theoretical value in exploring the coordinated development of supply chain to some extent.

Fuzzy Petri nets (FPNs), which have been extensively used in many fields, are a promising method for knowledge representation and reasoning (KRR). However, there are still some shortcomings in these FPNs. Although there are many improved FPNs, the existing FPNs still have difficulties to ‘embrace’ the inconsistent cognition of experts and individualized features of different systems. In this paper, a new type FPNs are proposed, called self-learning interval type-2 fuzzy Petri nets (SLIT2FPNs). The extended TOPSIS is proposed to determine the optimal alternative, and to collect cognition of experts. An interval numbers ordered weighted averaging operator is introduced to improve the knowledge reasoning capabilities of SLIT2FPNs. Moreover, because of the introduction of state transition algorithm, the model has the ability of self-learning and adjustment according to information dynamics. Finally, two comparison tests are presented to demonstrate the proposed methods. In addition, the applications in aluminum reduction process show that SLIT2FPNs are feasible to ‘embrace’ inconsistent cognition, and to attract individualized features of each reduction cell, and acquire a KRR model with good performance. These facts demonstrate that the methods and ideas proposed in this paper provide a solution for KRR of knowledge automation in the industrial processes.

Over the last 2 decades, face identification has been an active field of research in computer vision. As an important class of image representation methods for face identification, fused descriptor-based methods are known to lack sufficient discriminant information, especially when compared with deep learning-based methods. This paper presents a new face representation method, multi-descriptor fusion (MDF), which represents face images through a combination of multiple descriptors, resulting in hyper-high dimensional fused descriptor features. MDF enables excellent performance in face identification, exceeding the state-of-the-art, but it comes with high memory and computational costs. As a solution to the high cost problem, this paper also presents an optimisation method, discriminant ability-based multi-descriptor selection (DAMS), to select a subset of descriptors from the set of 65 initial descriptors whilst maximising the discriminant ability. The MDF face representation, after being refined by DAMS, is named selective multi-descriptor fusion (SMDF). Compared with MDF, SMDF has much smaller feature dimension and is thus usable on an ordinary PC, but still has similar performance. Various experiments are conducted on the CAS-PEAL-R1 and LFW datasets to demonstrate the performance of the proposed methods.

Multi-verse optimizer is one of the recently proposed nature-inspired algorithms that has proven its efficiency in solving challenging optimization problems. The original version of Multi-verse optimizer is able to solve problems with continuous variables. This paper proposes a binary version of this algorithm to solve problems with discrete variables such as feature selection. The proposed Binary Multi-verse optimizer is equipped with a V-shaped transfer function to covert continuous values to binary, and update the solutions over the course of optimization. A comparative study is conducted to compare Binary Multi-verse optimizer with other binary optimization algorithms such as Binary Bat Algorithm, Binary Particle Swarm Optimization, Binary Dragon Algorithm, and Binary Grey Wolf Optimizer. As case studies, a set of 13 benchmark functions including unimodal and multimodal is employed. In addition, the number of variables of these test functions are changed (5, 10, and 20) to test the proposed algorithm on problems with different number of parameters. The quantitative results show that the proposed algorithm significantly outperforms others on the majority of benchmark functions. Convergence curves qualitatively show that for some functions, proposed algorithm finds the best result at early iterations. To demonstrate the applicability of proposed algorithm, the paper considers solving feature selection and knapsack problems as challenging real-world problems in data mining. Experimental results using seven datasets for feature selection problem show that proposed algorithm tends to provide better accuracy and requires less number of features compared to other algorithms on most of the datasets. For knapsack problem 17 benchmark datasets were used, and the results show that the proposed algorithm achieved higher profit and lower error compared to other algorithms.

Hashing learning is efficient for large-scale image retrieval by using the nearest neighbor search with binary codes instead of continuous representations. With the success of deep neural networks in related tasks such as data representation, recent hashing methods based on deep learning can further improve image retrieval quality and classification accuracy. However, most existing methods are primarily designed to maximize the performance of retrieval based on linear scan of hash codes which is still time-consuming on large-scale datasets. Fortunately, Hamming space retrieval is an alternative as it is less time-consuming by retrieving data points that are within a Hamming ball with a given Hamming radius, but few works focus on that. In this paper, we propose a concentrated hashing method with neighborhood embedding (CHNE) for efficient and effective image retrieval and classification. By integrating Cauchy cross-entropy and pair-wise weighted similarity loss, CHNE can enable similar data pairs with smaller Hamming distance and dissimilar data pairs with larger Hamming distance. In addition, existing hashing methods are usually designed for retrieval, thus the performance of classification using the binary codes is not guaranteed. To tackle this problem, we jointly minimize the regression quantization and neighborhood structure reconstruction errors in the loss function to improve the classification accuracy. The proposed end-to-end deep hashing method can be optimized by back-propagation in a standard manner. Experimental results on several datasets demonstrate that the proposed method can improve the performance of retrieval and classification. Due to its generality, the proposed method is expected to be useful for image retrieval and classification in broader areas.

Detecting changes in data streams, with the data flowing continuously, is an important problem which Industry 4.0 has to deal with. In industrial monitoring, the data distribution may vary after a change in the machine’s operating point; this situation is known as concept drift, and it is key to detecting this change. One drawback of conventional machine learning algorithms is that they are usually static, trained offline, and require monitoring at the input level. A change in the distribution of data, in the relationship between the input and the output data, would result in the deterioration of the predictive performance of the models due to the lack of an ability to generalize the model to new concepts. Drift detecting methods emerge as a solution to identify the concept drift in the data. This paper proposes a new approach for concept drift detection—a novel approach to deal with sudden or abrupt drift, the most common drift found in industrial processes-, called CatSight. Briefly, this method is composed of two steps: (i) Use of Common Spatial Patterns (a statistical approach to deal with data streaming, closely related to Principal Component Analysis) to maximize the difference between two different distributions of a multivariate temporal data, and (ii) Machine Learning conventional algorithms to detect whether a change in the data flow has been occurred or not. The performance of the CatSight method, has been evaluated on a real use case, training six state of the art Machine Learning (ML) classifiers; obtained results indicate how adequate the new approach is.

In the blast furnace production site, the disposable thermocouple is used to measure the hot metal temperature. However, this method is not only inconvenient for continuous data acquisition but also costly for the use of one-time thermocouple. Hence, this paper establishes a prediction model to predict the hot metal temperature. Before the prediction model is established, the corresponding factors of influencing the hot metal temperature are selected, and the noises of production data are removed. In this paper, multi-layer extreme learning machine (ML-ELM) is used as the prediction algorithm of the prediction model. However, the input weights, hidden layer weights and hidden biases of ML-ELM are randomly selected, and the solution of the output weights is based on them, which makes ML-ELM inevitably have a set of non-optimal or unnecessary weights and biases. In addition, ML-ELM may suffer from over-fitting problem. Hence, this paper uses the adaptive particle swarm optimization (APSO) and the ensemble model to improve ML-ELM, and the improved algorithm is named as EAPSO-ML-ELM. APSO can optimize the selections of the input weights, hidden layer weights and hidden biases, the ensemble model can alleviate the over-fitting problem, i.e., this paper combines several of the optimized ML-ELMs which have different input weights, hidden layer weights and hidden biases. Finally, this paper also uses other algorithms to establish the prediction model, and simulation results demonstrate that the prediction model based on EAPSO-ML-ELM has better prediction accuracy and generalization performance.

Automated detection and tracking of a person’s actions plays a vital role in surveillance systems. Human activity detection has been carried out by using a variety of features; including flow-based, spatio-temporal and interest points based. We have created a fusion of features by incorporating those which give better results. LBP, HOG, Haar wavelets, SIFT, velocity and displacement being the major ones. By employing the time efficiency and optimality of SMO to train SVM, we have trained our system for both single person and multi-human action classification with improved accuracy. A generalized hierarchy of actions has been presented in this paper to demonstrate the extension of our methodology. We have achieved an accuracy of 91.99% on combination of KTH and Weizmann dataset and 86.48% on multi-human dataset. We have introduced our self-generated multi-human activity dataset in the following paper.

International Journal of Machine Learning and Cybernetics (IJMLC) is one of the influential journals in the area of computer science, and it published its first issue in 2010. On the one hand, taking the 544 IJMLC publications between 2010 and 2017 as the research object, this paper uses bibliometric methods to study the citation characteristics, international cooperation and institutional cooperation, the author’s cooperation rate and cooperation degree, geographical distribution of the IJMLC publications. On the other hand, CiteSpace and Vosviewer, two data visualization software tools, are used to make the comprehensive analysis of the co-occurrence of the author keywords of the IJMLC publications. The document co-citation clusters visualization and burst detection of keywords are also presented to explore the development of the research trends. The research results in this paper provide a basis for further improving the academic level and quality of the IJMLC.

High-dimensional problems result in bad classification results because some combinations of features have an adverse effect on classification; while class-imbalanced problems make the classifier to concern the majority class more but the minority less, because the number of samples of majority class is more than minority class. The problem of both high-dimensional and class-imbalanced classification is found in many fields such as bioinformatics, healthcare and so on. Many researchers study either the high-dimensional problem or class-imbalanced problem and come up with a series of algorithms, but they ignore the above new problem, which indicates high-dimensional problems affect sampling process while class-imbalanced problems interfere feature selection. Firstly, this paper analyses the new problem arising from the mutual influence of the two problems, and then introduces SVM and analyses its advantages in dealing high-dimensional problem and class-imbalanced problem. Next, this paper proposes a new algorithm named BRFE-PBKS-SVM aimed at high-dimensional class-imbalanced datasets, which improves SVM-RFE by considering the class-imbalanced problem in the process of feature selection, and it also improves SMOTE so that the procedure of over-sampling could work in the Hilbert space with an adaptive over-sampling rate by PSO. Finally, the experimental results show the performance of this algorithm.

Internet of Things (IoT) has become an important network paradigm and there are lots of smart devices connected by IoT. IoT systems are producing massive data and thus more and more IoT applications and services are emerging. Machine learning, as an another important area, has obtained a great success in several research fields such as computer vision, computer graphics, natural language processing, speech recognition, decision-making, and intelligent control. It has also been introduced in networking research. Many researches study how to utilize machine learning to solve networking problems, including routing, traffic engineering, resource allocation, and security. Recently, there has been a rising trend of employing machine learning to improve IoT applications and provide IoT services such as traffic engineering, network management, security, Internet traffic classification, and quality of service optimization. This survey paper focuses on providing an overview of the application of machine learning in the domain of IoT. We provide a comprehensive survey highlighting the recent progresses in machine learning techniques for IoT and describe various IoT applications. The application of machine learning for IoT enables users to obtain deep analytics and develop efficient intelligent IoT applications. This paper is different from the previously published survey papers in terms of focus, scope, and breadth; specifically, we have written this paper to emphasize the application of machine learning for IoT and the coverage of most recent advances. This paper has made an attempt to cover the major applications of machine learning for IoT and the relevant techniques, including traffic profiling, IoT device identification, security, edge computing infrastructure, network management and typical IoT applications. We also make a discussion on research challenges and open issues.

Extreme learning machine (ELM) is a novel and recent machine learning algorithm which was first proposed by Huang et al. (Proceedings of 2004 IEEE international joint conference on, pp 985–990, 2004). Over the last decade, ELM has gained a remarkable research interest with tremendous audiences from different domains in a short period of time due to its impressive characteristics over other single hidden-layer feedforward neural networks. Although ELM enjoys powerful advantages, it still has some potential weaknesses like performance sensitivity to the initial condition of the input weights, number of hidden neurons, and the selection of activation functions. In order to overcome the limitations of classical ELM, many metaheuristic algorithms including the evolutionary algorithms, swarm intelligence, memetic and trajectory algorithms have been proposed for optimizing the different components of ELM by researchers aiming to improve the generalization performance of ELM networks for different types of complex problems and applications. Therefore our review paper intent to conduct a deep study of the important aspects of applying metaheuristic algorithms for optimizing ELM networks. Three main streams of research lines are identified: the optimization of input weights and hidden biases, selection of hidden neurons, and optimization of activation functions. Furthermore, this paper will discuss a wide spectrum of applications of metaheuristic-based ELM models. We will highlight the strengths of these models and the improvements that are suggested in the literature to overcome their weaknesses. We touch upon several interesting and challenging open issues in optimizing ELM using metaheuristics.