Search

Automatic identity recognition from ear images is an active research topic in the biometric community. The ability to secretly acquire images of the ear remotely and the stability of the ear shape over time make this technology a promising alternative for surveillance, authentication, and forensic applications. In recent years, significant research has been conducted in this area. Nevertheless, challenges remain that limit the commercial use of this technology. Several phases of the ear recognition system have been studied in the literature, from ear detection, normalization, and feature extraction to classification. This paper reviews the most recent methods used to describe and classify biometric features of the ear. We propose a first taxonomy to group existing approaches to ear recognition, including 2D, 3D, and combined 2D and 3D methods, as well as an overview of historical advances in this field. It is well known that data and algorithms are the essential components in biometrics, particularly in-ear recognition. However, early ear recognition datasets were very limited and collected in laboratory with controlled environments. With the wider use of deep neural networks, a considerable amount of training data has become necessary if acceptable ear recognition performance is to be achieved. As a consequence, current ear recognition datasets have increased significantly in size. This paper gives an overview of the chronological evolution of ear recognition datasets and compares the performance of conventional vs. deep learning methods on several datasets. We proposed a second taxonomy to classify the existing databases, including 2D, 3D, and video ear datasets. Finally, some open challenges and trends are debated for future research. [ABSTRACT FROM AUTHOR]

• A linearly combined form of the objective is reformulated to control the trade-off between the return and the divergence more effectively. • An improved proximal policy optimization method (i.e., alphaPPO) is proposed, with a more elaborative alpha divergence for two adjacent policies. • The effectiveness of our alphaPPO is validated using detailed experimental comparison and analysis for six benchmark environments. Proximal policy optimization (PPO) is a recent advancement in reinforcement learning, which is formulated as an unconstrained optimization problem including two terms: accumulative discount return and Kullback–Leibler (KL) divergence. Currently, there are three PPO versions: primary, adaptive, and clipping. The most widely used PPO algorithm is the clipping version, in which the KL divergence is replaced by a clipping function to measure the difference between two policies indirectly. In this paper, we revisit this primary PPO and improve it in two aspects. One is to reformulate it as a linearly combined form to control the trade-off between two terms. The other is to substitute a parametric alpha divergence for KL divergence to measure the difference of two policies more effectively. This novel PPO variant is referred to as alphaPPO in this paper. Experiments on six benchmark environments verify the effectiveness of our alphaPPO, compared with clipping and combined PPOs. [ABSTRACT FROM AUTHOR]

Conventional convergence analysis on mini-batch learning is usually based on the stochastic gradient concept, in which we assume that the training data are presented in a random order. Also, some convergence results require that the learning rate should decrease with the number of training cycles, and that the objective function is a smooth function. Practically speaking, a deterministic presentation scheme with a fixed learning rate is more preferable. Hence, there is a gap between theoretical results and actual implementation. This paper aims at filling the gap. We use the radial basis function (RBF) model for nonlinear regression problems as an example to analyze the convergence properties of mini-batch learning. This paper considers a nonsmooth objective function, which consists of three terms. The coexistence of these three terms is able to handle a number of situations. The first term is a conventional training set error. The second term is a quadratic term which is used to suppress the effect of imperfections in the implementation. The last term is an ℓ 1 -norm term which is used to select important RBF nodes for the resultant network. Note that the ℓ 1 -norm term is a nonsmooth function. Although a nonsmooth ℓ 1 -norm is included and the mini-batch algorithm is deterministic, we are still able to derive the convergence properties, including the sufficient conditions for convergence and range of learning rate. With our results, we have a better theoretical understanding on the behaviour of mini-batch learning and obtain some guidelines on choosing the learning rate. The analysis results can be extended to other flat structural neural network models and other objective functions, which are with quadratic terms and ℓ 1 -norm. [ABSTRACT FROM AUTHOR]

Generative Adversarial Networks (GANs) are well-known tools for data generation and semi-supervised classification. GANs, with less labeled data, outperform Deep Neural Networks (DNNs) and Convolutional Neural Networks (CNNs) in classification tasks. The success of GANs in classification tasks motivates the development of GAN-based techniques for semi-supervised regression tasks. However, developing GANs for regression introduces two major challenges: (1) inherent instability in the GAN formulation and (2) performing regression and achieving stability simultaneously. This paper introduces techniques that show improvement in the GANs' regression capability. We bake a differentiable fuzzy logic system at multiple locations in a GAN. The fuzzy logic takes the output of either the generator or the discriminator to predict the output, y , and evaluate the generator's performance. We outline the results of applying the fuzzy logic system across multiple GANs and summarize each approach's efficacy. This paper shows that adding a fuzzy logic layer can enhance GAN's ability to perform regression; the most desirable injection location is problem-specific, and we show this through experiments over various datasets. Besides, we demonstrate empirically that the fuzzy-infused GANs are competitive with the DNNs. [ABSTRACT FROM AUTHOR]

The training method for deep neural networks mainly adopts the gradient descent (GD) method. These methods, however, are very sensitive to initialization and hyperparameters. In this paper, an enhanced gradient descent method guided by the trajectory-based method for training deep neural networks, termed the Trajectory Unified Framework (TJU) method, is presented. From a theoretical viewpoint, the robustness of the TJU-based method is supported by an analytical basis presented in the paper. From a computational viewpoint, a TJU methodology consisting of a Block-Diagonal-Pseudo-Transient-Continuation method and a gradient descent method, termed the TJU-GD method, for training deep neural networks is added to obtain high-quality results. Furthermore, to resolve the issue of imbalanced classification, a TJU-Focal-GD method is developed and evaluated. Experimental numerical evaluation of the proposed TJU-GD on various public datasets reveals that the proposed method can achieve great improvements over baseline methods. Specifically, the proposed TJU-Focal-GD also possesses several advantages over other methods for a class of imbalanced datasets from the homemade power line inspection dataset (PLID). [ABSTRACT FROM AUTHOR]

In this paper, a three-step model based on the integration of Deep Neural Networks (DNN) and Decision Models is introduced for image classification which is inspired by the human visual system. To make a decision about an object, many actions should be done in a hierarchical process in the brain. First, the retina receives visual stimuli and transfers them to the visual cortex in the brain. The information extracted in the visual cortex, is accumulated over time to select an appropriate response. Many of the current decision-making models do not show how each image is converted into useful information for the decision model. Some models have used neural networks to convert each image into the information needed in the decision-making model; however, the role of the retina is ignored among these models. In this paper, a combination of retina inspired filters, CNN-based description and accumulator-based decision model is used to classify images. This model's structure resembles the human brain due to the usage of the DoG filter bank as retina inspired filter in the first stage of it. This model shows a significant improvement in accuracy in comparison to other models; furthermore, its performance is acceptable even with the small sample training set. [ABSTRACT FROM AUTHOR]

Stochastic optimization acceleration is extremely significant and challenging for deep neural networks (DNNs). In recent years, several novel proportional-integral–differential-based (PID-based) optimizers have been proposed to speed up the optimization by alleviating the oscillation behavior of stochastic gradient descent with momentum (SGD-M), yet lacked theoretical analysis. Along this line of research, this paper adopts dynamical system theory to design a new hybrid optimizer and present theoretical analysis. Firstly, it is found that DNN optimization is equivalent to a discrete time dynamical system. Building upon the equivalence, high order augmented dynamical system viewpoint is utilized to design a PI-like optimizer for ensuring high accuracy, which is more stable than SGD-M. Then, hybrid dynamical system viewpoint is employed to improve the PI-like optimizer as a new hybrid form for suppressing oscillation and accelerating optimization. Lyapunov method, Taylor series, matrix theory and equilibrium are combined to theoretically investigate the convergence and the oscillation of loss function, showing that the proposed hybrid optimizer can alleviate oscillation, boost optimization speed, and maintain high accuracy. In theoretical analyses, explicit conditions of hyper-parameters that guarantee training stability are calculated and presented, practically guiding the adjustment of hyper-parameters and promoting the application of hybrid optimizer. Experiments are presented on three commonly used benchmark datasets, i.e., MNIST, CIFAR10 and CIFAR100, demonstrating that the hybrid optimizer obtains up to 42% acceleration with competitive accuracy relative to state-of-the-art optimizers. In short, this paper not only presents a new hybrid optimizer for accelerating optimization, but also provides a novel, theoretical and systematic perspective to find and analyze new optimizer for DNNs. [ABSTRACT FROM AUTHOR]

In this paper, an approach that makes use of knuckle creases and fingernails for person identification is presented. It introduces a framework for automatic person identification that includes localisation of the region of interest (ROI) of many components within hand images, recognition and segmentation of the detected components using bounding boxes, and similarity matching between two different sets of segmented images. The following hand components are considered: i) the metacarpophalangeal (MCP) joint, commonly known as the base knuckle; ii) the proximal interphalangeal (PIP) joint, commonly known as the major knuckle; iii) the distal interphalangeal (DIP) joint, commonly known as the minor knuckle; iv) the interphalangeal (IP) joint, commonly known as the thumb knuckle, and v) the fingernails. Crucial elements of the proposed framework are the feature extraction and similarity matching. This paper exploits different deep learning neural networks (DLNNs), which are essential in extracting discriminative high-level abstract features. We further use various similarity measures for the matching process. We validate the proposed approach on well-known benchmarks, including the 11k Hands dataset and the Hong Kong Polytechnic University Contactless Hand Dorsal Images known as PolyU. The results indicate that knuckle patterns and fingernails play a significant role in the person identification framework. The 11K Hands dataset results indicate that the left-hand results are better than the right-hand results and the fingernails produce consistently higher identification results than other hand components, with a rank-1 score of 100 %. In addition, the PolyU dataset attains 100 % in the fingernail of the thumb finger. [ABSTRACT FROM AUTHOR]

External attack can affect the normal functioning of electronic equipment including neuromorphic hardware, which leads to failure. Research on the brain-like model with robustness is beneficial to obtain its stable performance under external attack. Synaptic time delay (STD) is highly correlated with bio-brain function. However, the synaptic plasticity of brain-like models still lacks bio-rationality. Inspired by the bio-synaptic time delay, the purpose of this paper is to investigate a bio-rational brain-like model with bio-consistent STD evaluated by the anti-damage capabilities. In this paper, we propose a spiking neural network (SNN) with the topology of a complex network called the ComSNN, in which the topology has both the SWP and SFP conforming to biological functional brain networks, the nodes are Izhikevich neuron models, and the edges are synaptic plasticity models with random time delay conforming to the dynamic range of bio-synaptic time delay. Then, the anti-damage capabilities of the ComSNNs with different types of STDs under random attacks are evaluated based on the two anti-damage indicators. Further, taking a speech recognition task as the case study, the anti-damage capabilities of these ComSNNs are verified in application. Finally, the anti-damage mechanism of the ComSNN with STD is discussed. Our results indicate the following: (i) In terms of two anti-damage indicators, the ComSNN with random STD is superior to the ComSNN with fixed STD; in turn, the ComSNN with fixed STD is superior to the ComSNN without STD. (ii) Compared with the ComSNN without random attacks, the speech recognition accuracy of the ComSNN with random STD under random attacks still remains almost the same, which indicates the ComSNN has anti-damage capabilities in application; the recognition accuracies of ComSNNs with different types of STDs present the consistent order with the results based on the two anti-damage indicators. (iii) A correlation between the mean synaptic weight and the anti-damage capabilities implies that the intrinsic factor of the anti-damage capabilities is the synaptic plasticity; synaptic plasticity can change dynamic topological characteristics of SNNs, the analysis results of dynamic topological characteristics imply that the STD impacts the anti-damage capabilities of the network. • We propose a complex spiking neural network (ComSNN) with random synaptic time delay. • Our ComSNN with random synaptic time delay has better anti-damage capabilities. • Taking speech recognition task, its anti-damage capabilities are further verified. • Our discussion implies that synaptic time delay impacts the anti-damage capabilities. [ABSTRACT FROM AUTHOR]

• An overview of deep learning algorithms used in medical image segmentation is presented. • More than 150 papers applying deep learning to different medical applications are summarised. • Challenges and future directions in medical image segmentation are discussed. Deep learning (DL) algorithms have rapidly become a robust tool for analyzing medical images. They have been used extensively for medical image segmentation as the first and significant components of the diagnosis and treatment pipeline. Medical image segmentation is efficiently addressed by many types of deep neural networks, such as convolutional neural networks, fully convolutional network recurrent networks, adversarial networks, and U-shaped networks. This paper reviews the major DL models and applications pertinent to medical image segmentation and summarizes over 150 contributions to the field. Brief overviews of articles are provided by application area: anatomical structures such as organs, bones, and vessels, and abnormalities such as lesions and calcification. Moreover, we discuss current challenges and suggest directions for future research. [ABSTRACT FROM AUTHOR]

Computational research on facial micro-expressions has long focused on videos captured under constrained laboratory conditions due to the challenging elicitation process and limited samples that are publicly available. Moreover, processing micro-expressions is extremely challenging under unconstrained scenarios. This paper introduces, for the first time, a completely automatic micro-expression "spot-and-recognize" framework that is performed on in-the-wild videos, such as in poker games and political interviews. The proposed method first spots the apex frame from a video by handling head movements and unconscious actions which are typically larger in motion intensity, with alignment employed to enforce a canonical face pose. Optical flow guided features play a central role in our method: they can robustly identify the location of the apex frame, and are used to learn a shallow neural network model for emotion classification. Experimental results demonstrate the feasibility of the proposed methodology, establishing good baselines for both spotting and recognition tasks – ASR of 0.33 and F1-score of 0.6758 respectively on the MEVIEW micro-expression database. In addition, we present comprehensive qualitative and quantitative analyses to further show the effectiveness of the proposed framework, with new suggestion for an appropriate evaluation protocol. In a nutshell, this paper provides a new benchmark for apex spotting and emotion recognition in an in-the-wild setting. [ABSTRACT FROM AUTHOR]

The problem of global warming is getting more serious, and vehicle emission is the main cause. In recent years, the number of locomotives in China has been increasing at a rate of more than 20% per year, and the problem of automobile pollution is becoming more serious. The transportation industry is the main source of fossil fuel combustion and environmental pollution. Therefore, in this paper, we propose a multi-view deep neural network (MVDNN) to analyze the key factors affecting the fuel consumption of automobiles. The experiments show that the introduction of human input improves the prediction accuracy and the root mean square error (RMSE) achieves 0.993. In addition, this paper also finds that for drivers, driving habits, driving frequency, and safety awareness are the most important factors affecting the fuel consumption of vehicles by combining Lasso regression with MVDNN. Finally, by comparing the prediction accuracy of different experiments, relevant policy suggestions are made. [ABSTRACT FROM AUTHOR]

Skeleton-based action recognition has gained significant attention in computer vision. Most state-of-the-art (SOTA) approaches view the skeleton as a homogeneous graph. Unlike those approaches, this paper shows that methods in the heterogeneous graph manner can also achieve competitive performance. In this paper, a logical heterogeneous skeleton graph is built under the assumption of the heterogeneity of joints and bones at different positions, and the action recognition task is formulated as message aggregation and prediction on this heterogeneous graph. Specifically, a novel semantic concept named pseudo-metapath is introduced to represent dependencies between joints, based on which a hierarchical graph attention network with the joint-level attention and the semantic-level attention modules is proposed to capture richer skeleton features. The joint-level attention module intends to get the local difference among the joints within each pseudo-metapath, while the semantic-level attention module is capable of learning the global importance of different pseudo-metapaths. Extensive experiments on the NTU-RGB + D 60, NTU-RGB + D 120 and the SYSU datasets, validate that our model can attain comparable performance to the SOTA methods with 15x fewer input frames, 26.3x less FLOPs and 2.8x less parameters. [ABSTRACT FROM AUTHOR]

This paper introduces Elegans-AI models, a class of neural networks that leverage the connectome topology of the Caenorhabditis elegans to design deep and reservoir architectures. Utilizing deep learning models inspired by the connectome, this paper leverages the evolutionary selection process to consolidate the functional arrangement of biological neurons within their networks. The initial goal involves the conversion of natural connectomes into artificial representations. The second objective centers on embedding the complex circuitry topology of artificial connectomes into both deep learning and deep reservoir networks, highlighting their neural-dynamic short-term and long-term memory and learning capabilities. Lastly, our third objective aims to establish structural explainability by examining the heterophilic/homophilic properties within the connectome and their impact on learning capabilities. In our study, the Elegans-AI models demonstrate superior performance compared to similar models that utilize either randomly rewired artificial connectomes or simulated bio-plausible ones. Notably, these Elegans-AI models achieve a top-1 accuracy of 99.99% on both Cifar10 and Cifar100, and 99.84% on MNIST Unsup. They do this with significantly fewer learning parameters, particularly when reservoir configurations of the connectome are used. Our findings indicate a clear connection between bio-plausible network patterns, the small-world characteristic, and learning outcomes, emphasizing the significant role of evolutionary optimization in shaping the topology of artificial neural networks for improved learning performance. [ABSTRACT FROM AUTHOR]

In personal identity recognition systems, detecting a person's age, gender, and language using voice signal characteristics is a crucial issue, especially because of the importance of security considerations. Age, gender, and language classification problems are important in signal processing because they are used to analyze and understand human behavior, interactions, and preferences. This can be especially useful in the fields of human-computer interaction, psychology, and social science research. In this paper, a new system for detecting a speaker's age, gender, and language based on deep learning models is presented. Deep learning models have shown great efficacy in various fields of signal processing. For this paper, a range of deep models were tested, including convolutional neural networks (CNNs), recurrent neural network (RNN), and a fine-tuning ResNet34 architecture. Additionally, techniques such as transfer learning were applied to improve the efficiency of the proposed system. The input voice signals are preprocessed by applying the spectro-temporal transform to obtain additional features that can be fed to the ResNet34 model, which is designed specifically for the task of voice signal processing. The dataset used in this paper was sourced from the Mozilla common voice initiative, which is dedicated to advancing speech recognition and language identification technologies. The performance of the proposed algorithm was evaluated in the presence of Gaussian noise to determine its robustness. The experimental results demonstrated that the proposed algorithm significantly outperformed basic algorithms and other deep neural networks in terms of age and gender recognition from voice signals. [ABSTRACT FROM AUTHOR]

In this work, we examine the advantages of using multiple types of behaviours in recommendation systems. Intuitively, each user often takes some implicit actions (e.g., click) before making an explicit decision (e.g., purchase). Previous studies show that implicit and explicit feedback has different roles for a useful recommendation. However, these studies either exploit implicit and explicit behaviours separately or ignore the semantics of sequential interactions between users and items. In addition, we go from the hypothesis that a user's preferences at a time are combinations of long-term and short-term interests. In this paper, we propose some Deep Learning architectures. The first one is Implicit to Explicit (ITE) , to exploit users' interests through the sequence of their actions. The second and third ones are two versions of ITE with Bidirectional Encoder Representations from Transformers based (BERT-based) architecture called BERT-ITE and BERT-ITE-Si , which combine users' long- and short-term preferences without and with side information to enhance users' representations. The experimental results show that our models outperform previous state-of-the-art ones and also demonstrate our views on the effectiveness of exploiting the implicit to explicit order as well as combining long- and short-term preferences in three large-scale datasets. The source code of our paper is available at: https://github.com/tranquyenbk173/BERT_ITE. [ABSTRACT FROM AUTHOR]

In this paper, we consider the finite-time anti-synchronization for the master–slave neural networks with time delays. By means of combining integral inequality skills with other inequality skills, two novel criteria to assure the finite-time anti-synchronization for the discussed master–slave neural networks are presented under two classes of different controllers. Our results and method are different from those in existing papers. [ABSTRACT FROM AUTHOR]

Intrusion Detection, the ability to detect malware and other attacks, is a crucial aspect to ensure cybersecurity. So is the ability to identify this myriad of attacks. Artificial Neural Networks (as well as other machine learning bio-inspired approaches) are an established and proven method of accurate classification. ANNs are extremely versatile – a wide range of setups can achieve significantly different classification results. The main objective and contribution of this paper is the evaluation of the way the hyperparameters can influence the final classification result. In this paper, a wide range of ANN setups is put to comparison. We have performed our experiments on two benchmark datasets, namely NSL-KDD and CICIDS2017. The most effective arrangement achieves the multi-class classification accuracy of 99.909% on an established benchmark dataset. [ABSTRACT FROM AUTHOR]

• This paper proposed a novel two-stage spatiotemporal features learning method based on a Siamese 3D CNN for MEs recognition. • The focal loss was introduced to address the problem of inefficient model training caused by the class imbalance in the MEs datasets. • An adaptive construction method based on adaptive CNN was proposed to construct the key-frames sequence to summarize the original MEs. Micro-expressions (MEs) are spontaneous and involuntary facial subtle reactions which often reveal the genuine emotions within human beings. Recognizing MEs automatically is becoming increasingly crucial for many areas such as diagnosis and security. However, the short time duration and low spatial intensity of MEs pose great challenges for accurately recognizing them. Additionally, the lack of sufficient and balanced spontaneous MEs data also makes this problem even harder to solve, and some adaptive modeling strategies have been quite urgent recently. To this end, this paper draws inspirations from few-shot learning to propose a novel two-stage learning (i.e., prior learning and target learning) method based on a siamese 3D convolutional neural network for MEs recognition (MERSiamC3D). Specifically, in the prior learning stage, the proposed MERSiamC3D is used to extract the generic features of MEs. In the target learning stage, the structure and parameters of the MERSiamC3D will be carefully adjusted and the Focal Loss is adopted for high-level features learning. Afterwards, in order to effectively retain the spatiotemporal information of the original MEs video, an adaptive construction method based on adaptive convolutional neural network is proposed to construct the key-frames sequence to summarize the original MEs video, which is able to help drop the redundant frames and relatively highlight the movement of the apex frame. Then, the new key-frames are taken as the input of the two-stage learning method. Finally, through extensive evaluations and experiments on three publically available MEs datasets, the proposed method in this work could outperform traditional methods and other deep learning baselines, which provides a novel insight on how to leverage scarce data for MEs recognition. [ABSTRACT FROM AUTHOR]

This paper presents the neural dynamical network to compute the generalized and restricted singular value decompositions (GSVD/RSVD) in the regularization methods for ill-posed problems. The neural network model is defined by ordinary differential equations (ODE) which can be solved by many state-of-the-art techniques. The main purpose of the paper is to develop two neural network models for finding approximations of the GSVD and the RSVD. The globally asymptotic stability analyses are provided and numerical experiments illustrate our theory and methods. For small scale problems, the estimation can be as accurate as O (10 - 15). For ill-posed problems, the truncation regularization method implementing the GSVD/RSVD algorithms also produces accurate results. [ABSTRACT FROM AUTHOR]

• ANN, CART, and RF regression trees have shown the potential in modeling energy cost. • Three different strategies regarding variable selection were tested and compared. • Machine learning and Boruta method have produced the highest accuracy of prediction. • The model has extracted heating and occupational data as the most important. • The created model could be used to assess the concept of smart buildings and cities. The paper deals with modeling the cost of energy consumed in public buildings by leveraging three machine learning methods: artificial neural networks, CART, and random forest regression trees. Energy consumption is one of the major issues in global and national policies, therefore scientific efforts in creating prediction models of energy consumption and cost are highly important. One of the largest energy consumers in every state is its public sector, consisting of educational, health, public administration, military, and other types of public buildings. Recent technologies based on sensor networks and Big data platforms enable collection of large amounts of data that could be used to analyze energy consumption and cost. A real data from Croatian public sector is used in this paper including a large number of constructional, energetic, occupational, climate and other attributes. The algorithms for data pre-processing and modeling by optimizing parameters are suggested. Three strategies were tested: (1) with all available variables, (2) with a filter-based variable selection, and (3) with a wrapper-based variable selection which integrates Boruta algorithm and random forest. Prediction models of energy cost are created using two approaches: (a) comparative usage of artificial neural networks and two types of regression trees, CART and random forest, and (b) integration of RF-Boruta variable selection and machine learning methods for prediction. A cross-validation procedure was used to optimize the artificial neural network and regression tree topology, as well to select the most appropriate activation function. Along with creating a prediction model, the aim of the paper was also to extract the relevant predictors of energy cost in public buildings which are important in planning the construction or renovation of buildings. The results have shown that the second approach which integrates machine learning with Boruta method, where the random forest algorithm is used for both variable reduction and prediction modeling, has produced a higher accuracy of prediction than the individual usage of three machine learning methods. Such findings confirm the potential of hybrid machine learning methods which are suggested in previous research, but in favor of random forest method over CART and artificial neural networks. Regarding the variable selection, the model has extracted heating and occupational data as the most important, followed by constructional, cooling, electricity, and lighting attributes. The model could be implemented in public buildings information systems and their IoT networks within the concept of smart buildings and smart cities. [ABSTRACT FROM AUTHOR]

Deep neural networks (DNNs) have been shown to be vulnerable to backdoor attacks during training. Most of the existing backdoor defense methods are designed for specific types of backdoor attacks, and the work of detecting backdoors and mitigating backdoors is mostly separate. Currently, few general and complete defense frameworks have been developed. In this paper, we propose a lightweight, general, and complete defense framework against three main types of backdoor attacks. It can efficiently detect poisoned images and remove trigger patterns on poisoned images without costly retraining of the backdoor model. First, we use the feature difference between clean samples and poisoned samples in the middle layer of the model to distinguish them. Then, we remove the backdoor using image inpainting algorithm to remove the backdoor triggering pattern on the poisoned samples. We deploy three of the most popular backdoor attacks on three datasets to test the effectiveness of our defenses. Extensive experimental results show that our method can effectively defend against various backdoor attacks with a relatively small cost. In particular, we reduce the attack success rate of the more stealthy clean-label poisoning attack from 94.9% to 0.02% with little impact on the classification accuracy of the inpainted images. [ABSTRACT FROM AUTHOR]

Although Deep Neural Networks (DNNs)-based object detectors are widely used in various fields, especially on aerial imagery object detections, it has been observed that a small elaborately designed patch attached to the images can mislead the DNNs-based detectors into producing erroneous output. However, the target detectors being attacked are quite simple, and the attack efficiency is relatively low in previous works, making it not practicable in real scenarios. To address these limitations, a new adversarial patch attack algorithm is proposed in this paper. Firstly, we designed a novel loss function using the intermediate outputs of the models rather than the model's final outputs interpreted by the detection head to optimize adversarial patches. The experiments conducted on the DOTA, RSOD, and NWPU VHR-10 datasets demonstrate that our method can significantly degrade the performance of the detectors. Secondly, we conducted intensive experiments to investigate the impact of different outputs of the detection model on generating adversarial patches, demonstrating the class score is not as effective as the objectness score. Thirdly, we comprehensively analyzed the attack transferability across different aerial imagery datasets, verifying that the patches generated on one dataset are also effective in attacking another. Moreover, we proposed ensemble training to boost the attack's transferability across models. Our work alarms the application of DNNs-based object detectors in aerial imagery. [ABSTRACT FROM AUTHOR]

[Display omitted] • Connecting Hessian trace with a generalization error bound. • Flat minima of loss landscape and stability analysis in dynamical systems. • Efficient Hessian trace regularization algorithm with Dropout. • Performance comparison on vision and language tasks. In this paper, we develop a novel regularization method for deep neural networks by penalizing the trace of Hessian. This regularizer is motivated by a recent guarantee bound of the generalization error. We explain its benefits in finding flat minima and avoiding Lyapunov stability in dynamical systems. We adopt the Hutchinson method as a classical unbiased estimator for the trace of a matrix and further accelerate its calculation using a Dropout scheme. Experiments demonstrate that our method outperforms existing regularizers and data augmentation methods, such as Jacobian, Confidence Penalty, Label Smoothing, Cutout, and Mixup. The code is available at https://github.com/Dean-lyc/Hessian-Regularization. [ABSTRACT FROM AUTHOR]

Recent studies show that gradient descent can train a deep neural network (DNN) to achieve small training and test errors when the DNN is sufficiently wide. This result applies to various over-parameterized neural network models including fully-connected neural networks and convolutional neural networks. However, existing theory does not apply to graph convolutional networks (GCNs), as GCNs is built according to the topological structures of the data. It has been empirically observed that GCNs can outperform vanilla neural networks when the underlying graph captures geometric information of the data. However, there is few theoretical justification of such observation. In this paper, we establish theoretical guarantees of the high-probability convergence of gradient descent for training over-parameterized GCNs. Specifically, we introduce a novel measurement of the relation between the graph and the data, called the "graph disparity coefficient", and show that the convergence of GCN is faster when the graph disparity coefficient is smaller. Our analysis provides novel insights into how the graph convolution operation in a GCN helps training, and provides useful guidance for GCN training in practice. [ABSTRACT FROM AUTHOR]

Human intention prediction is of great significance in many applications, such as human-robot interaction, intelligent rehabilitation robots. This paper studies the problem of short-term next-active-object prediction in egocentric images. The short-term next-active-object refers to the object that a human is going to interact with in the short-term future, which is an embodiment of human intention. Most current methods usually use object-centered cues, such as the deviation of object appearance change and the unique shape of the egocentric object trajectory, to predict the next-active-object. In this paper, inspired by the fact that human intention is also revealed by human-centered cues, we propose a deep neural network model that integrates the cues from visual attention and hand positions to predict the next-active-object. Firstly, the probability maps of visual attention and hand positions are constructed, and then the probability distribution of next-active-object is generated. We experimentally compare our method with several baseline methods using two datasets and confirm its effectiveness. In addition, ablation experiments are conducted, and crucial points concerning the next-active-object are discussed. [ABSTRACT FROM AUTHOR]

This paper is concerned with the event-triggered state estimation (ETSE) problem for a class of discrete-time Markovian jumping neural networks with mode-dependent time-delays and uncertain transition probabilities. The parameters of the neural networks experience switches that are characterized by a Markovian chain whose transition probabilities are allowed to be uncertain. The event-triggered mechanism is introduced in the sensor-to-estimator channel to reduce the frequency of signal communication. The aim of this paper is to develop an ETSE scheme such that the estimation error dynamics is exponentially ultimately bounded in the mean square. To achieve the aim, two sufficient conditions are proposed with the first one guaranteeing the existence of the required state estimator, and the second one giving the algorithm for designing the corresponding estimator gain by solving some matrix inequalities. In the end, the validity of the proposed estimation scheme is illustrated by a numerical example. [ABSTRACT FROM AUTHOR]

In this paper, a novel control scheme is developed to solve an optimal containment control problem of unknown continuous-time multi-agent systems. Different from traditional adaptive dynamic programming (ADP) algorithms, this paper proposes an internal reinforcement ADP algorithm (IR-ADP), in which the internal reinforcement signals are added in order to facilitate the learning process. Then a distributed containment control law is designed for each agent with the internal reinforcement signal. The convergence of this IR-ADP algorithm and the stability of the closed-loop multi-agent system are analyzed theoretically. For the implementation of the optimal controllers, three neural networks (NNs), namely internal reinforcement NNs, critic NNs and actor NNs, are utilized to approximate the internal reinforcement signals, the performance indices and optimal control laws, respectively. Finally, some simulation results are provided to demonstrate the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

The chemi-thermo-mechanical pulping (CTMP) is an important process that produces fibers for paper-making, whose output fiber length distribution (FLD) directly determines the energy consumption and product quality of the subsequent papermaking production. Therefore, study on modeling and control of the output FLD is essential for improving pulp quality and saving energy in refining process. However, the output FLD of refining process has non-Gaussian stochastic distribution property, making it difficult to use conventional methods to establish the output FLD model effectively. Under the framework of stochastic distribution control theory, this paper presents a novel modeling approach for output probability density function (PDF) of fiber length in CTMP by combining with the improved wavelet neural network (WNN). In this context, the square root B-spline approximation principle is firstly adopted to extract the B-spline weights of fiber length PDF shape as the target outputs for the WNN based B-spline weights model. Secondly, a novel two-stage hybrid learning algorithm is proposed to establish the parameters system for the WNN based weighs model by defining a multi-objective evaluation index for modeling accuracy in advance. This learning algorithm integrates multi-objective NSGA II algorithm in the first stage for better initial solutions at a global scope, and gradient descent method is employed in the second stage for accurate solutions of WNN model parameters inside a local range. As a result, the final output PDF of fiber length is reconstructed by the estimated B-spline weights using the B-spline approximation principle again. Experiments using actual industrial data have demonstrated the superiority and practicability of the proposed method. [ABSTRACT FROM AUTHOR]

This paper presents a federated learning approach based on utilizing computational resources of the IoT edge devices for training deep neural networks. In this approach, the edge devices and the cloud server collaborate in the training phase while preserving the privacy of the edge device data. Owing to the limited computational power and resources available to the edge devices, instead of the original neural network (NN), we suggest to use a smaller NN generated using a proposed heuristic method. In the proposed approach, the smaller model, which is trained on the edge device, is generated from the main NN model. By the exploiting Knowledge Distillation (K D) approach, the learned knowledge in the server and the edge devices can be exchanged, leading to lower required computation on the server and preserving data privacy of the edge devices. Also, to reduce the knowledge transfer overhead on the communication links between the server and the edge devices, a method for selecting the most valuable data to transfer the knowledge is introduced. The effectiveness of this method is assessed by comparing it to state-of-the-art methods. The results show that the proposed method lowers the communication traffic by up to 250 × and increases the learning accuracy by an average of 8.9 % in the cloud compared to the prior K D -based distributed training approaches in CIFAR-10 dataset. [ABSTRACT FROM AUTHOR]

[Display omitted] With the widespread applications of Deep Neural Networks (DNNs), the safety of DNNs has become a significant issue. The vulnerability of the neural networks against adversarial examples deepens concerns about the safety of DNNs applications. This paper proposed a novel defence method to improve the adversarial robustness of DNN classifiers without using adversarial training. This method introduces two new loss functions. First, a zero-cross-entropy loss is used to punish overconfidence and find the appropriate confidence for different instances. Second, a logit balancing loss is proposed to protect DNNs from non-targeted attacks by regularising incorrect classes' logits distribution. This method achieved competitive adversarial robustness compared to advanced adversarial training methods. Meanwhile, a novel robustness diagram is proposed to analyse, interpret and visualise the robustness of DNN classifiers against adversarial attacks. Furthermore, a Log-Softmax-pattern-based adversarial attack detection method is proposed. This detection method can distinguish clean inputs and multiple adversarial attacks via one multi-classification MLP. In particular, it is state-of-the-art in identifying white-box gradient-based attacks; it achieved at least 95.5% accuracy for classifying four white-box gradient-based attacks with maximum 0.1% false positive ratio. [ABSTRACT FROM AUTHOR]

Spiking neural networks (SNNs) are a promising energy-efficient alternative to artificial neural networks (ANNs) due to their rich dynamics, capability to process spatiotemporal patterns, and low-power consumption. The complex intrinsic properties of SNNs give rise to a diversity of their learning rules which are essential to functional SNNs. This paper is aimed at presenting a comprehensive overview of learning rules in SNNs. Firstly, we introduce the basic concepts of SNNs and commonly used neuromorphic datasets. Then, guided by a hierarchical classification of SNN learning rules, we present a comprehensive survey of these rules with discussions on their characteristics, advantages, limitations, and performance on several datasets. Moreover, we review practical applications of SNNs, including event-based vision and audio signal processing. Finally, we conclude this survey with a discussion on challenges and promising future research directions in this area. [ABSTRACT FROM AUTHOR]

Neural responding variability to the same stimulus typically decreases after a stimulus presented. During neural variability quenching, the pre-stimulus neural activities interact with the post-stimulus neural responses. However, whether these interactions have influences on information coding remains unclear. In this paper, we construct a two-layer k-winner-take-all (k-WTA) spiking network which simulates primary visual cortical neural responses through probabilistic inference. Generating the phenomenon of neural variability quenching, the network could reflect interactions between pre- and post-stimulus neural responses consistent with experimental observations. During neural variability quenching, pre-stimulus neural responding variability and complexity are considered as factors for the post-stimulus neural responses. Simulations to given stimuli are classified with each varying factor, respectively. Neural responding dimensionality measures the capacity of information coding to given stimuli. Over classified simulations, both of two factors could modify interactions between pre- and post-stimulus neural responses, leading to different neural responding dimensionalities. During neural variability quenching, the temporal structure of stimuli performs as another factor which also could modify neural interactions and induce the varying neural responding dimension. Our model provides the possible interpretation to how the pre-stimulus neural responses participate in neural variability quenching and affect the information coding. [ABSTRACT FROM AUTHOR]

Electroencephalogram (EEG) data are produced in quantity for measuring brain activity in response to external stimuli. With the rapid development of brain-inspired intelligence, spiking neural network (SNN) possesses the potential to handle EEG data by using spiking activity transmitted among spatially located synapses and neurons. As an original and unifying SNN architecture, NeuCube, is developed to model, recognize and understand EEG data. However, the NeuCube still faces some challenges for EEG-based pattern recognition, such as few labeled data and changes of data probability distribution. Hence, this paper proposes a novel method to improve the performance of the NeuCube for EEG-based pattern recognition by transfer learning. In the first place, the covariance matrix alignment of EEG data is implemented for every subject in the Euclidean space, which reduces the probability distribution discrepancy of EEG data between different subjects. Different estimation methods for reference covariance matrix are tested and the optimal one is selected for different subjects. Secondly, spatio-temporal features of EEG data are extracted based on the NeuCube reservoir. Since hyper-parameters of the NeuCube reservoir have a great impact on its spatio-temporal representation, an improved cuckoo search algorithm is proposed to discover the optimal hyper-parameters for obtaining the optimal spatio-temporal features. Last but not least, a weighted transfer support vector machine is proposed to improve the original output classifier of the NeuCube in order to make the model adaptive to the cross-domain variability of EEG data. The proposed method is tested on open dataset 2a from BCI competition IV 2008 and achieves good spatio-temporal pattern recognition results. Furthermore, the neuron connectivity and activation level associated with the process of mental tasks are illustrated. [ABSTRACT FROM AUTHOR]

As a powerful data modeling tool, Heterogeneous Information Network (HIN) has been successfully used in auxiliary information exploitation to boost recommendation performance. For HIN based recommendation, it is challenging to extract and fuse useful features of user preferences and item attributes under different semantic paths in HINs. Existing methods leverage a pre-defined fusion function to integrate different semantics for recommendation, which cannot characterize the complex nonlinear interactions between users and items. In this paper, we present a general framework named MNRec, short for Meta-path fusion based Neural Recommendation, to extract and fuse user and item embeddings under different meta-paths for recommendation. Under the framework, we propose an instantiation of MNRec with Multi-Layer Perceptron (MLP) structure. It consists of two major steps, i.e., meta-path based heterogeneous network embedding and deep learning based rating prediction. Concretely, appropriate meta-paths are first designed according to domain knowledge. Then the embeddings of users and items are obtained through a meta-path and commuting matrix based heterogeneous network embedding method. Finally, in light of the powerful nonlinear modeling capabilities of deep neural networks, the learned embeddings under different meta-paths are integrated into a two-pathway MLP structure for rating prediction. Experimental results on three real-world datasets demonstrate the superiority and effectiveness of MNRec compared with state-of-the-art baselines in rating prediction. [ABSTRACT FROM AUTHOR]

Referring image segmentation has sprung up benefiting from the outstanding performance of deep neural networks. However, most existing methods explore either local details or the global context of the scene without sufficiently modelling the coordination between them, leading to sub-optimal results. In this paper, we propose a transformer-based method to enforce the in-depth coordination between short- and long-range dependencies in both explicit and implicit fusion processes. Specifically, we design a Cross Modality Transformer (CMT) module with two successive blocks for explicitly integrating linguistic and visual features, which first locates the related visual region in a global view before concentrating on local patterns. Besides, a Hybrid Transformer Architecture (HTA) is utilized as a feature extractor in the encoding stage to capture global relationships and retain local cues. It can further aggregate the multi-modal features in an implicit manner. In the decoding stage, a Cross-level Information Integration module (CI2) is developed to gather information from adjacent levels by dual top-down paths, including a guided filtration path and a residual reservation path. Experimental results show that the proposed method outperforms the state-of-the-art methods on four RIS benchmarks. [ABSTRACT FROM AUTHOR]

This paper is concerned with the stability issue of quaternion-valued neural networks with neutral delay, proportional delay and leakage delay. Taking use of the principle of homeomorphism, techniques of matrix inequality and Lyapunov stability theory, a main stability criterion is derived in the form of quaternion-valued linear matrix inequality for ensuring the unique existence and global stability of the equilibrium point for the considered quaternion-valued neural networks. An illustrative example and its simulations are given to show the effectiveness of the theoretical result. [ABSTRACT FROM AUTHOR]

Image quality assessment (IQA) is a process of visuo-cognitive, which is an essential stage in human interaction with the environment. The study of the context effect (Brown and Daniel, 1987) also shows that the evaluation results made by the human vision system (HVS) is related to the contrast between the distorted image and the background environment. However, the existing IQA methods carry out the quality evaluation that only depends on the distorted image itself and ignores the impact of environment to human perception. In this paper, we propose a novel blind image quality assessment(BIQA) based on the context effect. At first, we use a graphical model to describe how the context effect influences human perception of image quality. Based on the established graph, we construct the context relation between the distorted image and the background environment by the X. Han et al. (2015). Then the context features are extracted from the constructed relation, and the quality-related features are extracted by the fine-tuned neural network from the distorted image in pixel-wise. Finally, these features are concatenated to quantify image quality degradations and then regress to quality scores. In addition, the proposed method is adaptive to various deep neural networks. Experimental results show that the proposed method not only has the state-of-art performance on the synthetic distorted images, but also has a great improvement on the authentic distorted images. [ABSTRACT FROM AUTHOR]

The state-of-the-art performance for several real-world problems is currently reached by deep and, in particular, convolutional neural networks (CNN). Such learning models exploit recent results in the field of deep learning, leading to highly performing, yet very large neural networks with typically millions to billions of parameters. As a result, such models are often redundant and excessively oversized, with a detrimental effect on the environment in terms of unnecessary energy consumption and a limitation to their deployment on low-resource devices. The necessity for compression techniques able to reduce the number of model parameters and their resource demand is thereby increasingly felt by the research community. In this paper we propose the first extensive comparison, to the best of our knowledge, of the main lossy and structure-preserving approaches to compress pre-trained CNNs, applicable in principle to any existing model. Our study is intended to provide a first and preliminary guidance to choose the most suitable compression technique when there is the need to reduce the occupancy of pre-trained models. Both convolutional and fully-connected layers are included in the analysis. Our experiments involved two pre-trained state-of-the-art CNNs (proposed to solve classification or regression problems) and five benchmarks, and gave rise to important insights about the applicability and performance of such techniques w.r.t. the type of layer to be compressed and the category of problem tackled. [ABSTRACT FROM AUTHOR]

In many public security applications such as anomaly detection, it is important to re-identify a group of pedestrians by other surveillance cameras, which ascribes to the group retrieval problem. Most previous studies focus on single-person re-identification (re-id) and ignore the correlations among group members, and they lack a large and comprehensive group retrieval benchmark to associate these two tasks. To address this issue, this paper focuses on solving the group retrieval problem and uses it to improve re-id. First, the paper build a comprehensive benchmark for both group retrieval and the group-aided re-id task by proposing a novel pedestrian group retrieval dataset named "SYSU-Group" and a corresponding group-associated re-id dataset named "Group-reID", which introduces realistic challenges such as variations of pose, viewpoint, illumination, and intra-group layout. The paper then proposes the Siamese Verification-Identification-based Group Retrieval (SVIGR) method, which combines verification and identification modules in a Siamese network to extract robust person features and follows the principle of minimum distance matching to realize group retrieval. Finally, a group-guided re-id method named group retrieval correlation (GRC) is proposed to improve re-id with additional group information. Experimental results on three various group retrieval benchmarks demonstrate the superiority and effectiveness of our method. [ABSTRACT FROM AUTHOR]

This paper focuses on state estimation problem for discrete-time high-order neural networks with time-varying delays. First, the delay-dependent global exponential stability criterion of the error system is derived. Then, the state observer is designed by using the generalized inverse theory of matrices. Last, two numerical examples are given to illustrate the validity of the theoretical results. The method proposed in this paper has two advantages: (i) it is directly based on the definitions of global exponential stability and Moore–Penrose inverse of matrix, which avoids the construction of Lyapunov–Krasovskii functional; (ii) the obtained stability criteria contain only several simple matrix inequalities, which are easier to solve. More valuable, this paper fills in the gaps in designing state observers for discrete-time high-order neural network models. [ABSTRACT FROM AUTHOR]

Artificial intelligence and all its supporting tools, e.g. machine and deep learning in computational intelligence-based systems, are rebuilding our society (economy, education, life-style, etc.) and promising a new era for the social welfare state. In this paper we summarize recent advances in data science and artificial intelligence within the interplay between natural and artificial computation. A review of recent works published in the latter field and the state the art are summarized in a comprehensive and self-contained way to provide a baseline framework for the international community in artificial intelligence. Moreover, this paper aims to provide a complete analysis and some relevant discussions of the current trends and insights within several theoretical and application fields covered in the essay, from theoretical models in artificial intelligence and machine learning to the most prospective applications in robotics, neuroscience, brain computer interfaces, medicine and society, in general. [ABSTRACT FROM AUTHOR]

Implementing associative memory experiments with nanoscale memristors is an interesting subject, which can allow robots to mimic human thinking. This paper is concerned with a new emotion model of memristor-based neural network and its circuit implementation. The model has three inputs and two outputs, which can feel happy when it receives good news or feel sad when it receives bad news. Unknown news can be recognized through associative memory, which simulates human emotions. Associative learning and three kinds of forgetting process together make up the full-function emotion model. In addition, the Ag/AgInSbTe/Ta-based model closely related to the actual physical properties of memristors is used to design synaptic structures. The circuits of memristor-based neural networks are also simplified. Furthermore, the new emotion model is able to adjust the changing rate of emotion. The process reflects the fact that humans learn the same thing faster at the second time, compared with the first time. Finally, PSPICE is used to simulate all the circuits of the emotion model. The presented emotion model in this paper offers more possibilities for designing intelligent machines. [ABSTRACT FROM AUTHOR]

Cross-modal person re-identification(Re-id) under infrared light and visible light (RGB-IR) is of great significance for modern video surveillance, especially nighttime surveillance. The existing research results in the single-mode person re-identification field have reached a high level. Cross-model person re-identification, however, is rather challenging for the tremendous cross-modality and intra-modality difference in addition to common issues such as lighting conditions, human posture, camera angle, and etc..The Cross-modal Multi-granularity Attention Network (CMGN) proposed by this paper enables network to learn the common features of different modalities and map them to the same feature space. Major contributions made by this paper includes:1) A new "butterfly" attention module that can be used for cross-modal tasks is designed to constrain the network attention to common areas of different modes. And in the RegDB and SYSU-MM01 dataset reached the effect of State-of-the-art(SOA). 2) An end-to-end multi-granularity feature fusion network dedicated to processing cross-modal problems is proposed. [ABSTRACT FROM AUTHOR]

It is well known that weighted sum of node states plays an essential role in function implementation of neural networks. Therefore, this paper proposes a new weighted sum synchronization model for memristive neural networks. Unlike the existing synchronization models of memristive neural networks which control each network node to reach synchronization, the proposed model treats the networks as dynamic entireties by weighted sum of node states and makes the entireties instead of each node reach expected synchronization. In this paper, weighted sum complete synchronization and quasi-synchronization are both investigated by designing feedback controller and aperiodically intermittent controller, respectively. Meanwhile, a flexible control scheme is designed for the proposed model by utilizing some switching parameters and can improve anti-interference ability of control system. By applying Lyapunov method and some differential inequalities, some effective criteria are derived to ensure the synchronizations of memristive neural networks. Moreover, the error level of the quasi-synchronization is given. Finally, numerical simulation examples are used to certify the effectiveness of the derived results. [ABSTRACT FROM AUTHOR]

• A novel AS layer is proposed to improve the expression power of Siamese network. • Auxiliary relocation branch to refine tracking results and introduce prior knowledge. • Switch function to monitor tracking process and evaluate the status of AS-SiamFC. • Impact of pooling layer on Siamese network is discussed in the experiment part. Siamese network is highly regarded in the visual object tracking filed because of its unique advantages of pairwise input and pairwise training. It can measure the similarity between two image patches, which coincides with the principle of the matching-based tracking algorithm. In this paper, a variant Siamese network based tracker is proposed to introduce attention module into traditional Siamese network, and relocate the object with some auxiliary relocation methods, when the proposed tracker runs under an untrusted state. Firstly, a novel attention shake layer is proposed to replace the max pooling layer in Siamese network. This layer could introduce and train two different kinds of attention modules at the same time, which means the proposed attention shake layer could also help to improve the expression power of Siamese network without increasing the depth of the network. Secondly, an auxiliary relocation branch is proposed to assist in object relocation and tracking. According to the prior assumptions of visual object tracking, some weights are involved in the auxiliary relocation branch, such as structure similarity weight, motion similarity weight, motion smoothness weight and object saliency weight. Thirdly, a novel response map based switch function is proposed to monitor the tracking process and control the effect of auxiliary relocation branch. Furthermore, in order to discuss the effect of pooling layer in Siamese network, 9 pooling and attention architectures are proposed and discussed in this paper. Some empirical results are shown in the experiment part. Comparing with the state-of-the-art trackers, the proposed tracker could achieve comparable performance in multiple benchmarks. [ABSTRACT FROM AUTHOR]

• We propose an effective method to generate class-specific attention map. By introducing the concept of spatial class score, our attention map retains more accurate object location clues. • We propose a superpixel-CRF model to refine the segmentation masks for training images. We apply superpixels to recover object boundaries and design a new energy function to obtain high-quality segmentation masks. • We propose an iterative training framework which iteratively refines pixel-level annotations (segmentation masks) and trains the segmentation network to achieve better image segmentation performance. • Our method shows impressive performance, and achieves the state-of-the-art results on PASCAL VOC 2012 dataset and MS COCO dataset compared with existing approaches. In the recent years, there have been remarkable improvements in the semantic segmentation based on deep convolutional neural networks. However, DCNN-based weakly supervised segmentation approaches are still inferior to the fully supervised manner. We observe that the performance gap mainly comes from the limitation of producing high-quality dense object localization clues from image-level labels. To mitigate this gap, in this paper, we are committed to finding more precise and complete pixel-level annotations from image-level tags. So this paper proposes a new iterative training framework for progressively refining pixel-wise labels and training the segmentation network. We first propose a new attention map generating method to locate more discriminative object regions. To find out less-discriminative regions and rectify wrong object location clues, the method fuses saliency map into the attention map to generate the initial pseudo pixel-level annotations. In the iteration training process, we train the segmentation network by treating pseudo pixel-level annotations as supervision. In order to correct the inaccurate labels of segmentation masks produced by current segmentation network, a superpixel-CRF refinement model is exploited to produce more accurate pixel-level annotations and use these annotations as supervision to train the segmentation network again. Our framework iterates between refining pixel-level annotations and optimizing the segmentation network. Experimental results demonstrate that our method significantly outperforms other previous weakly supervised semantic segmentation methods, and obtains the state-of-the-art performance, which are 64.7% mIoU score on PASCAL VOC 2012 test set and 26.3% mIoU score on MS COCO validation set. [ABSTRACT FROM AUTHOR]