Search

This paper addresses the questions and concerns raised in the comment paper entitled “Comments on ‘Traffic Sign Recognition Using Kernel Extreme Leaning Machines With Deep Perceptual Features”’. It is analyzed that the major questions and concerns in the comment paper are due to some misunderstanding on the statements and the main contributions of the original paper by Zeng et al.. The main clarifications are as follows: (1) The weight update rule used by Zeng et al. is a standard form of kernel extreme learning machine (KELM) but not the rule of reduced KELM mentioned by the comment paper. (2) The aim and main contribution of the original paper by Zeng et al. are to improve the classification precision of traffic sign recognition (TSR) based on convolutional neural networks without dramatically increasing the scale or complexity of the deep neural network model, which leads to a relatively less training cost. The authors of the comment paper focused on the reduction of KELM’s computational costs but it is beyond the topic of the original paper. In addition, the reduction of KELM’s computational costs has been well studied in the literature. (3) The supplementary experiment is provided to show that the TSR method proposed by Zeng et al. is valid whether or not the training samples are much more than the needed kernels. [ABSTRACT FROM AUTHOR]

Surface defect is a serious problem in real-world manufacturing system and it is important to use vision-based recognition to ensure the surface quality of products. Currently, due to the ability of automatic feature extraction, deep learning models, such as convolutional neural network (CNN), have been widely used in this area. However, these CNN-based models may not solve a problem well - inter-class similarities and intra-class variations (ISIV), which affect their ability of feature extraction and thus influence their recognition performance. To address this problem, this paper introduces a graph guidance mechanism into CNN to improve the ability of feature extraction, called Graph guided Convolutional Neural Network (GCNN). Firstly, GCNN defines a graph by computing the similarities between training samples. Secondly, the graph is introduced into VGG11, a popular CNN structure, to increase the inter-class distances and decrease the intra-class distances between defect samples. Meanwhile, a learnable coefficient is introduced into the training process to balance the effect of graph guidance automatically. The experimental results on four famous surface defect datasets demonstrate that the graph guidance helps CNN models have better ability of feature extraction and thus achieve better performance. Compared with state-of-the-art models, the proposed method can achieve the best performance. Furthermore, the final discussion shows that the learnable coefficient can help the proposed model to gain better performance, and that the proposed model increases a little computation cost compared to its original CNN model. Note to Practitioners—This paper is motivated by the problem in real-world manufacturing process – inter-class similarities and intra-class variations. Most of current CNN-based models may not solve this problem well, which limits their applications in real-world system. This paper proposes a graph guidance mechanism and introduces it to the training of CNN. The proposal can improve the ability of feature extraction, and thus have better performance than those without the graph guidance. By applying the proposal to four examples, the results show that the proposal is more feasible and effective than the state-of-the-art models in surface defect recognition. Furthermore, the mechanism assists the training of CNN and is removed in the process of recognition, so it will not increase time and occupied memory in recognition, and meanwhile, only limited computation cost is increased in training process. [ABSTRACT FROM AUTHOR]

This article presents a paradigm that adapts general large-scale pretrained models (PTMs) to speech emotion recognition task. Although PTMs shed new light on artificial general intelligence, they are constructed with general tasks in mind, and thus, their efficacy for specific tasks can be further improved. Additionally, employing PTMs in practical applications can be challenging due to their considerable size. Above limitations spawn another research direction, namely, optimizing large-scale PTMs for specific tasks to generate task-specific PTMs that are both compact and effective. In this paper, we focus on the speech emotion recognition task and propose an improVed emotion-specific pretrained encoder called Vesper. Vesper is pretrained on a speech dataset based on WavLM and takes into account emotional characteristics. To enhance sensitivity to emotional information, Vesper employs an emotion-guided masking strategy to identify the regions that need masking. Subsequently, Vesper employs hierarchical and cross-layer self-supervision to improve its ability to capture acoustic and semantic representations, both of which are crucial for emotion recognition. Experimental results on the IEMOCAP, MELD, and CREMA-D datasets demonstrate that Vesper with 4 layers outperforms WavLM Base with 12 layers, and the performance of Vesper with 12 layers surpasses that of WavLM Large with 24 layers. [ABSTRACT FROM AUTHOR]

The success of supervised models for emotion recognition on images heavily depends on the availability of images properly annotated. Although millions of images are presently available, only a few are annotated with reliable emotional information. Current emotion recognition solutions either use large amounts of weakly-labeled web images, which often contain noise that is unrelated to the emotions of the image, or transfer learning, which usually results in performance losses. Thus, it would be desirable to know which images would be useful to be annotated to avoid an extensive annotation effort. In this paper, we propose a novel approach based on active learning to choose which images are more relevant to be annotated. Our approach dynamically combines multiple active learning strategies and learns the best ones (without prior knowledge of the best ones). Experiments using nine benchmark datasets revealed that: (i) active learning allows to reduce the annotation effort, while reaching or surpassing the performance of a supervised baseline with as little as 3% to 18% of the baseline's training set, in classification tasks; (ii) our online combination of multiple strategies converges to the performance of the best individual strategies, while avoiding the experimentation overhead needed to identify them. [ABSTRACT FROM AUTHOR]

Multi-Task Learning (MTL) is a learning paradigm in machine learning and its aim is to leverage useful information contained in multiple related tasks to help improve the generalization performance of all the tasks. In this paper, we give a survey for MTL from the perspective of algorithmic modeling, applications and theoretical analyses. For algorithmic modeling, we give a definition of MTL and then classify different MTL algorithms into five categories, including feature learning approach, low-rank approach, task clustering approach, task relation learning approach and decomposition approach as well as discussing the characteristics of each approach. In order to improve the performance of learning tasks further, MTL can be combined with other learning paradigms including semi-supervised learning, active learning, unsupervised learning, reinforcement learning, multi-view learning and graphical models. When the number of tasks is large or the data dimensionality is high, we review online, parallel and distributed MTL models as well as dimensionality reduction and feature hashing to reveal their computational and storage advantages. Many real-world applications use MTL to boost their performance and we review representative works in this paper. Finally, we present theoretical analyses and discuss several future directions for MTL. [ABSTRACT FROM AUTHOR]

This paper develops an intelligent grid-interactive building controller, which optimizes building operation during both normal hours and demand response (DR) events. To avoid costly on-demand computation and to adapt to non-linear building models, the controller utilizes reinforcement learning (RL) and makes real-time decisions based on a near-optimal control policy. Learning such a policy typically amounts to solving a hard non-convex optimization problem. We propose to address this problem with a novel global-local policy search method. In the first stage, an RL algorithm based on zero-order gradient estimation is leveraged to search for the optimal policy globally, due to its scalability and the potential to escape some poor performing local optima. The obtained policy is then fine-tuned locally to bring the first-stage solution closer to that of the original unsmoothed problem. Experiments on a simulated five-zone commercial building demonstrate the advantages of the proposed method over existing learning approaches. They also show that the learned control policy outperforms a pragmatic linear model predictive controller (MPC) and approaches the performance of an oracle MPC in testing scenarios. Using a state-of-the-art advanced computing system, we demonstrate that the controller can be learned and deployed within hours of training. [ABSTRACT FROM AUTHOR]

This paper investigates the feasibility of novel data-driven deep learning (DL) models to predict the solution of Maxwell’s equations for low-frequency electromagnetic (EM) devices. With ground truth (empirical evidence) data being generated from a finite-element analysis solver, a deep convolutional neural network is trained in a supervised manner to learn a mapping for magnetic field distribution for topologies of different complexities of geometry, material, and excitation, including a simple coil, a transformer, and a permanent magnet motor. Preliminary experiments show DL model predictions in close agreement with the ground truth. A probabilistic model is introduced to improve the accuracy and to quantify the uncertainty in the prediction, based on Monte Carlo dropout. This paper establishes a basis for a fast and generalizable data-driven model used in the analysis, design, and optimization of EM devices. [ABSTRACT FROM AUTHOR]

Generative adversarial networks (GANs) have been effective for learning generative models for real-world data. However, accompanied with the generative tasks becoming more and more challenging, existing GANs (GAN and its variants) tend to suffer from different training problems such as instability and mode collapse. In this paper, we propose a novel GAN framework called evolutionary GANs (E-GANs) for stable GAN training and improved generative performance. Unlike existing GANs, which employ a predefined adversarial objective function alternately training a generator and a discriminator, we evolve a population of generators to play the adversarial game with the discriminator. Different adversarial training objectives are employed as mutation operations and each individual (i.e., generator candidature) are updated based on these mutations. Then, we devise an evaluation mechanism to measure the quality and diversity of generated samples, such that only well-performing generator(s) are preserved and used for further training. In this way, E-GAN overcomes the limitations of an individual adversarial training objective and always preserves the well-performing offspring, contributing to progress in, and the success of GANs. Experiments on several datasets demonstrate that E-GAN achieves convincing generative performance and reduces the training problems inherent in existing GANs. [ABSTRACT FROM AUTHOR]

With the proliferation of user-generated online videos, Multimodal Sentiment Analysis (MSA) has attracted increasing attention recently. Despite significant progress, there are still two major challenges on the way towards robust MSA: 1) inefficiency when modeling cross-modal interactions in unaligned multimodal data; and 2) vulnerability to random modality feature missing which typically occurs in realistic settings. In this paper, we propose a generic and unified framework to address them, named Efficient Multimodal Transformer with Dual-Level Feature Restoration (EMT-DLFR). Concretely, EMT employs utterance-level representations from each modality as the global multimodal context to interact with local unimodal features and mutually promote each other. It not only avoids the quadratic scaling cost of previous local-local cross-modal interaction methods but also leads to better performance. To improve model robustness in the incomplete modality setting, on the one hand, DLFR performs low-level feature reconstruction to implicitly encourage the model to learn semantic information from incomplete data. On the other hand, it innovatively regards complete and incomplete data as two different views of one sample and utilizes siamese representation learning to explicitly attract their high-level representations. Comprehensive experiments on three popular datasets demonstrate that our method achieves superior performance in both complete and incomplete modality settings. [ABSTRACT FROM AUTHOR]

The prior domain knowledge, i.e., the regularity property of continuous multiobjective optimization problems (MOPs), could be learned to guide the search for evolutionary multiobjective optimization. This paper proposes a learning-to-guide strategy (LGS) for assisting the search for multiobjective optimization algorithms in dealing with MOPs. The main idea behind LGS is to capture the regularity via learning techniques to guide the evolutionary search to generate promising offspring solutions. To achieve this, a generative model called the generative topographic mapping (GTM) is adopted to capture the manifold distribution of a population. A set of regular grid points in the latent space are mapped into the decision space within some manifold structures to guide the search for mating with some parents for offspring generation. Following this idea, three alternative LGS-based generation operators are developed and investigated, which combine the local and global information in the offspring generation. To learn the regularity more efficiently in an algorithm, the proposed LGS is embedded in an efficient evolutionary algorithm (called LGSEA). The LGSEA includes an incremental training procedure aimed at reducing the computational cost of GTM training by reusing the built GTM model. The developed algorithm is compared with some newly developed or classical learning-based algorithms on several benchmark problems. The results demonstrate the advantages of LGSEA over other approaches, showcasing its potential for solving complex MOPs. [ABSTRACT FROM AUTHOR]

Time series classification models have been garnering significant importance in the research community. However, not much research has been done on generating adversarial samples for these models. These adversarial samples can become a security concern. In this paper, we propose utilizing an adversarial transformation network (ATN) on a distilled model to attack various time series classification models. The proposed attack on the classification model utilizes a distilled model as a surrogate that mimics the behavior of the attacked classical time series classification models. Our proposed methodology is applied onto 1-nearest neighbor dynamic time warping (1-NN DTW) and a fully convolutional network (FCN), all of which are trained on 42 University of California Riverside (UCR) datasets. In this paper, we show both models were susceptible to attacks on all 42 datasets. When compared to Fast Gradient Sign Method, the proposed attack generates a larger faction of successful adversarial black-box attacks. A simple defense mechanism is successfully devised to reduce the fraction of successful adversarial samples. Finally, we recommend future researchers that develop time series classification models to incorporating adversarial data samples into their training data sets to improve resilience on adversarial samples. [ABSTRACT FROM AUTHOR]

Short text clustering has far-reaching effects on semantic analysis, showing its importance for multiple applications such as corpus summarization and information retrieval. However, it inevitably encounters the severe sparsity of short text representations, making the previous clustering approaches still far from satisfactory. In this paper, we present a novel attentive representation learning model for shot text clustering, wherein cluster-level attention is proposed to capture the correlations between text representations and cluster representations. Relying on this, the representation learning and clustering for short texts are seamlessly integrated into a unified model. To further ensure robust model training for short texts, we apply adversarial training to the unsupervised clustering setting, by injecting perturbations into the cluster representations. The model parameters and perturbations are optimized alternately through a minimax game. Extensive experiments on four real-world short text datasets demonstrate the superiority of the proposed model over several strong competitors, verifying that robust adversarial training yields substantial performance gains. [ABSTRACT FROM AUTHOR]

Pretrained deep models hold their learnt knowledge in the form of model parameters. These parameters act as “memory” for the trained models and help them generalize well on unseen data. However, in absence of training data, the utility of a trained model is merely limited to either inference or better initialization towards a target task. In this paper, we go further and extract synthetic data by leveraging the learnt model parameters. We dub them Data Impressions, which act as proxy to the training data and can be used to realize a variety of tasks. These are useful in scenarios where only the pretrained models are available and the training data is not shared (e.g., due to privacy or sensitivity concerns). We show the applicability of data impressions in solving several computer vision tasks such as unsupervised domain adaptation, continual learning as well as knowledge distillation. We also study the adversarial robustness of lightweight models trained via knowledge distillation using these data impressions. Further, we demonstrate the efficacy of data impressions in generating data-free Universal Adversarial Perturbations (UAPs) with better fooling rates. Extensive experiments performed on benchmark datasets demonstrate competitive performance achieved using data impressions in absence of original training data. [ABSTRACT FROM AUTHOR]

As deep learning models are usually massive and complex, distributed learning is essential for increasing training efficiency. Moreover, in many real-world application scenarios like healthcare, distributed learning can also keep the data local and protect privacy. Recently, the asynchronous decentralized parallel stochastic gradient descent (ADPSGD) algorithm has been proposed and demonstrated to be an efficient and practical strategy where there is no central server, so that each computing node only communicates with its neighbors. Although no raw data will be transmitted across different local nodes, there is still a risk of information leak during the communication process for malicious participants to make attacks. In this paper, we present a differentially private version of asynchronous decentralized parallel SGD framework, or A(DP) $^2$ 2 SGD for short, which maintains communication efficiency of ADPSGD and prevents the inference from malicious participants. Specifically, Rényi differential privacy is used to provide tighter privacy analysis for our composite Gaussian mechanisms while the convergence rate is consistent with the non-private version. Theoretical analysis shows A(DP) $^2$ 2 SGD also converges at the optimal $\mathcal {O}(1/\sqrt{T})$ O (1 / T) rate as SGD. Empirically, A(DP) $^2$ 2 SGD achieves comparable model accuracy as the differentially private version of Synchronous SGD (SSGD) but runs much faster than SSGD in heterogeneous computing environments. [ABSTRACT FROM AUTHOR]

Plant species identification represents an extraordinary challenge for machine learning due to visual interspecies similarities and large intraspecies variations. Furthermore, research literature reports that plant species identification usually lacks sufficiently large datasets for training classification models. In this paper, we address this problem with a model that simplifies the VGG-16 architecture, the N-VGG model. The idea behind N-VGG is to reduce experimentally observed overfitting on VGG-16 by using as few trainable parameters as possible. To do this, we substitute the flattening layer on the VGG architecture with a global average pooling layer. This reduces the size of the feature vector. In addition, we eliminate one of the two fully-connected layers and use a new hyper-parameter, N, to indicate the number of nodes on the remaining layer. To show the robustness of the N-VGG model, we conducted extensive experimentation. We trained N-VGG on five datasets for plant species identification. Four of these datasets are publicly available and have been widely used as benchmarks for plant identification models. For all datasets, we compare the accuracy of N-VGG to that of the VGG-16, Inception-v4, and EfficienNet-B3 models. The experimental results show that the N-VGG model achieved the best classification performance for all but one datasets, whereas all the models showed a remarkable performance for the remaining dataset. This evidence supports our initial idea that, for plant species classification, some accuracy might be lost due to overfitting and that having fewer trainable parameters helps in producing a more robust model. [ABSTRACT FROM AUTHOR]

Federated learning (FL), as a distributed machine learning setting, has received considerable attention in recent years. To alleviate privacy concerns, FL essentially promises that multiple parties jointly train the model by exchanging gradients rather than raw data. However, intrinsic privacy issue still exists in FL, e.g., user’s training samples could be revealed by solely inferring gradients. Moreover, the emerging poisoning attack also poses a crucial security threat to FL. In particular, due to the distributed nature of FL, malicious users may submit crafted gradients during the training process to undermine the integrity and availability of the model. Furthermore, there exists a contradiction in simultaneously addressing two issues, that is, privacy-preserving FL solutions are dedicated to ensuring gradients indistinguishability, whereas the defenses against poisoning attacks tend to remove outliers based on their similarity. To solve such a dilemma, in this paper, we aim to build a bridge between the two issues. Specifically, we present a privacy-enhanced FL (PEFL) framework that adopts homomorphic encryption as the underlying technology and provides the server with a channel to punish poisoners via the effective gradient data extraction of the logarithmic function. To the best of our knowledge, the PEFL is the first effort to efficiently detect the poisoning behaviors in FL under ciphertext. Detailed theoretical analyses illustrate the security and convergence properties of the scheme. Moreover, the experiments conducted on real-world datasets show that the PEFL can effectively defend against label-flipping and backdoor attacks, two representative poisoning attacks in FL. [ABSTRACT FROM AUTHOR]

The problem of single-group multicast beamforming (SMBF) is well-known NP-hard. It motivates the pursuit of computationally efficient near-optimal solutions. Due to multicasting, the multicast group is bottlenecked by the user(s) with the minimum received signal-to-noise ratio (SNR). This paper provides an in-depth interpretation of the SMBF problem from the multicasting point of view and proposes to solve it in two steps: i) select the bottlenecking users by a machine learning approach based on determinantal point process (DPP), and ii) design the beamformer for the selected users. The DPP model jointly considers the magnitudes and directions of users’ channel vectors, and thus enables an efficient selection of the bottlenecking users. Moreover, for a specific channel model, the DPP model is only associated with network size and each takes a one-off training cost, thus can be used as a codebook. The proposed DPP-based subset selection is incorporated adaptively into two fast beamforming algorithms, i.e., the QR decomposition algorithm and the successive beamforming (SB) algorithm. They specifically design the beamformers for the selected users by leveraging channel orthogonalization therein. Numerical results demonstrate the superiority of the proposed QR-DPP and SB-DPP algorithms in terms of the performance-complexity compromise and their robustness to different scenarios. [ABSTRACT FROM AUTHOR]

Histopathological tissue classification is a simpler way to achieve semantic segmentation for the whole slide images, which can alleviate the requirement of pixel-level dense annotations. Existing works mostly leverage the popular CNN classification backbones in computer vision to achieve histopathological tissue classification. In this paper, we propose a super lightweight plug-and-play module, named Pyramidal Deep-Broad Learning (PDBL), for any well-trained classification backbone to improve the classification performance without a re-training burden. For each patch, we construct a multi-resolution image pyramid to obtain the pyramidal contextual information. For each level in the pyramid, we extract the multi-scale deep-broad features by our proposed Deep-Broad block (DB-block). We equip PDBL in three popular classification backbones, ShuffLeNetV2, EfficientNetb0, and ResNet50 to evaluate the effectiveness and efficiency of our proposed module on two datasets (Kather Multiclass Dataset and the LC25000 Dataset). Experimental results demonstrate the proposed PDBL can steadily improve the tissue-level classification performance for any CNN backbones, especially for the lightweight models when given a small among of training samples (less than 10%). It greatly saves the computational resources and annotation efforts. The source code is available at: https://github.com/linjiatai/PDBL. [ABSTRACT FROM AUTHOR]

Deep learning (DL) techniques have shown promising performance for designing data-driven power system transient stability assessment (TSA) models. However, due to the deep structure of the DL, the resulting model is always a black-box and hard to explain, which hinders its practical adoption by the industry. This paper proposes an interpretable DL-based TSA model to balance the TSA accuracy and transparency. The proposed method combines the strong nonlinear modelling capability of a deep neural network and the interpretability of a decision tree (DT). Through regularizing DL-based model with the average DT path length in the training process, the proposed interpretable DL-based TSA method can visually explain the TSA decision-making process. Simulation results have shown that the proposed method can deliver highly accurate TSA results and interpretable TSA decision-making rules, which can be used for designing preventive control actions. [ABSTRACT FROM AUTHOR]

Few-shot learning is an extremely challenging task in computer vision that has attracted increased research attention in recent years. However, most recent methods do not fully use the task’s information, and few of the seen samples result in large intraclass differences among the same classes. In this paper, we propose a novel task encoding with distribution calibration (TEDC) model for few-shot learning, which uses the relationships among the feature distributions to reduce intraclass differences. In the TEDC model, an integrated feature extraction module (IFEM) is proposed, which extracts the multiangle visual features of an image and fuses them to obtain more representative features. To effectively utilize the task information, a novel task encoding module (TEM) is proposed, which obtains the task features by fusing all the seen samples’ information and uses them to adjust all the samples’ features for more generalizable task-specific representations. We also propose a distribution calibration module (DCM) to reduce the bias between the distribution of the support features and the query features in the same class. Extensive experiments show that our proposed TEDC model achieves an excellent performance and outperforms the state-of-the-art methods on three widely used few-shot classification benchmarks, specifically miniImageNet, tieredImageNet and CUB-200-2011. [ABSTRACT FROM AUTHOR]

A deep belief network (DBN) is effective to create a powerful generative model by using training data. However, it is difficult to fast determine its optimal structure given specific applications. In this paper, a growing DBN with transfer learning (TL-GDBN) is proposed to automatically decide its structure size, which can accelerate its learning process and improve model accuracy. First, a basic DBN structure with single hidden layer is initialized and then pretrained, and the learned weight parameters are frozen. Second, TL-GDBN uses TL to transfer the knowledge from the learned weight parameters to newly added neurons and hidden layers, which can achieve a growing structure until the stopping criterion for pretraining is satisfied. Third, the weight parameters derived from pretraining of TL-GDBN are further fine-tuned by using layer-by-layer partial least square regression from top to bottom, which can avoid many problems of traditional backpropagation algorithm-based fine-tuning. Moreover, the convergence analysis of the TL-GDBN is presented. Finally, TL-GDBN is tested on two benchmark data sets and a practical wastewater treatment system. The simulation results show that it has better modeling performance, faster learning speed, and more robust structure than existing models. Note to Practitioners—Transfer learning (TL) aims to improve training effectiveness by transferring knowledge from a source domain to target domain. This paper presents a growing deep belief network (DBN) with TL to improve the training effectiveness and determine the optimal model size. Facing a complex process and real-world workflow, DBN tends to require long time for its successful training. The proposed growing DBN with TL (TL-GDBN) accelerates the learning process by instantaneously transferring the knowledge from a source domain to each new deeper or wider substructure. The experimental results show that the proposed TL-GDBN model has a great potential to deal with complex system, especially the systems with high nonlinearity. As a result, it can be readily applicable to some industrial nonlinear systems. [ABSTRACT FROM AUTHOR]

This paper presents a method to improve the traditional mixture of deformable part models (MDPM) method from the learning perspective. First, an object part configuration learning algorithm based on group sparsity constraint is introduced to automatically discover the object part number, size, and location. The algorithm imposes two additional regularization terms in addition to the standard hinge loss function. The first term focuses on automatic part selection and the second term focuses on automatic part placement. Second, this paper introduces an improved MDPM training framework. The framework applies a learned transformation to normalize the prediction score from each individual deformable part model (DPM) into a pseudo probability such that the partition of the entire object appearance feature space becomes less sensitive to the prior distributions of different DPMs. Finally, the two proposed improvements are combined and formulated under the expectation-maximization framework. We evaluate our method mainly using the PASCAL VOC2007 and VOC2010 detection benchmarks and show that the proposed learning algorithms could increase the detection mean AP score by 2.4% and 0.9%, respectively, on these two data sets when using the proposed part selection method and the training algorithm. We also present further in-depth analysis of the proposed algorithm in the experiments. [ABSTRACT FROM PUBLISHER]

Recent studies have shown that robustness to adversarial attacks can be transferred across deep neural networks. In other words, we can make a weak model more robust with the help of a strong teacher model. In this paper, we ask if models can “learn together” and “teach each other” to achieve better robustness instead of learning from a static teacher. We study how interactions among models enhance robustness via knowledge distillation. We propose mutual adversarial training (MAT), in which multiple models are trained together and share the knowledge of adversarial examples to achieve improved robustness. MAT allows robust models to explore a larger space of adversarial samples and find more robust feature spaces and decision boundaries. Through extensive experiments on the CIFAR-10, CIFAR-100, and mini-ImageNet datasets, we demonstrate that MAT can effectively improve model robustness and outperform state-of-the-art methods under white-box attacks. In addition, we show that MAT can also mitigate the robustness trade-off among different perturbation types. Specially, we train specialist models that learn to defend a specific perturbation type and a generalist model that learns to defend multiple perturbation types by learning from the specialists, which brings as much as 13.4% accuracy gain to AT baselines against the union of $l_{\infty} $ , $l_{2}$ , and $l_{1}$ attacks. Our results show the effectiveness of the proposed method and demonstrate that collaborative learning is an effective strategy for designing robust models. [ABSTRACT FROM AUTHOR]

Text encoding is one of the most important steps in Natural Language Processing (NLP). It has been done well by the self-attention mechanism in the current state-of-the-art Transformer encoder, which has brought about significant improvements in the performance of many NLP tasks. Though the Transformer encoder may effectively capture general information in its resulting representations, the backbone information, meaning the gist of the input text, is not specifically focused on. In this paper, we propose explicit and implicit text compression approaches to enhance the Transformer encoding and evaluate models using this approach on several typical downstream tasks that rely on the encoding heavily. Our explicit text compression approaches use dedicated models to compress text, while our implicit text compression approach simply adds an additional module to the main model to handle text compression. We propose three ways of integration, namely backbone source-side fusion, target-side fusion, and both-side fusion, to integrate the backbone information into Transformer-based models for various downstream tasks. Our evaluation on benchmark datasets shows that the proposed explicit and implicit text compression approaches improve results in comparison to strong baselines. We therefore conclude, when comparing the encodings to the baseline models, text compression helps the encoders to learn better language representations. [ABSTRACT FROM AUTHOR]

In power system analysis, probabilistic load flow (PLF) accounts for uncertainties stemming from power entities such as renewable generation systems and consumer demands. In this paper, we present a Gaussian Process (GP) emulator for enabling data-driven PLF on practical distribution networks (DNs) without requiring knowledge of underlying system parameters. The main novelty of our proposal lies in the kernel design process. An ideal kernel allows for greater efficiency during training and inferencing stages without being subject to common issues that include overfitting to the training dataset and poor accuracies. In this proposal, the kernel selection process is formulated as a bi-level optimization problem. This is a very difficult problem to solve as it is composed of discrete variables and non-convex constraints. We overcome these issues by proposing a best-response strategy refinement process to identify an efficient kernel configuration in an iterative manner. The convergence properties of this iterative algorithm and the approximating capabilities of the resulting GP emulator are established using potential game theoretic constructs, universal approximation theorem and representer theorem. The performance of the proposed emulator is then showcased via comprehensive simulations conducted on practical DNs and comparisons with the state-of-the-art. [ABSTRACT FROM AUTHOR]

Signal classification problems arise in a wide variety of applications, and their demand is only expected to grow. In this paper, we focus on the wireless sensor network signal classification setting, where each sensor forwards quantized signals to a fusion center to be classified. Our primary goal is to train a decision function and quantizers across the sensors to maximize the classification performance in an online manner. Moreover, we are interested in sparse sensor selection using a marginalized weighted kernel approach to improve network resource efficiency by disabling less reliable sensors with minimal effect on classification performance. To achieve our goals, we develop a multi-sensor online kernel scalar quantization (MSOKSQ) learning strategy that operates on the sensor outputs at the fusion center. Our theoretical analysis reveals how the proposed algorithm affects the quantizers across the sensors. Additionally, we provide a convergence analysis of our online learning approach by studying its relationship to batch learning. We conduct numerical studies under different classification and sensor network settings which demonstrate the accuracy gains from optimizing different components of MSOKSQ and robustness to reduction in the number of sensors selected. [ABSTRACT FROM AUTHOR]

We give an overview of recent developments in the modeling of radiowave propagation, based on machine learning (ML) algorithms. We identify the input and output specification and the architecture of the model as the main challenges associated with ML-driven propagation models. Relevant papers are discussed and categorized based on their approach to each of these challenges. Emphasis is given on presenting the prospects and open problems in this promising and rapidly evolving area. [ABSTRACT FROM AUTHOR]

Effective non-parametric density estimation is a key challenge in high-dimensional multivariate data analysis. In this paper, we propose a novel approach that builds upon tensor factorization tools. Any multivariate density can be represented by its characteristic function, via the Fourier transform. If the sought density is compactly supported, then its characteristic function can be approximated, within controllable error, by a finite tensor of leading Fourier coefficients, whose size depends on the smoothness of the underlying density. This tensor can be naturally estimated from observed and possibly incomplete realizations of the random vector of interest, via sample averaging. In order to circumvent the curse of dimensionality, we introduce a low-rank model of this characteristic tensor, which significantly improves the density estimate especially for high-dimensional data and/or in the sample-starved regime. By virtue of uniqueness of low-rank tensor decomposition, under certain conditions, our method enables learning the true data-generating distribution. We demonstrate the very promising performance of the proposed method using several toy, measured, and image datasets. [ABSTRACT FROM AUTHOR]

The explosive growth in video streaming requires video understanding at high accuracy and low computation cost. Conventional 2D CNNs are computationally cheap but cannot capture temporal relationships; 3D CNN based methods can achieve good performance but are computationally intensive. In this paper, we propose a generic and effective Temporal Shift Module (TSM) that enjoys both high efficiency and high performance. The key idea of TSM is to shift part of the channels along the temporal dimension, thus facilitate information exchanged among neighboring frames. It can be inserted into 2D CNNs to achieve temporal modeling at zero computation and zero parameters. TSM offers several unique advantages. First, TSM has high performance; it ranks the first on the Something-Something leaderboard upon submission. Second, TSM has high efficiency; it achieves a high frame rate of 74fps and 29fps for online video recognition on Jetson Nano and Galaxy Note8. Third, TSM has higher scalability compared to 3D networks, enabling large-scale Kinetics training on 1,536 GPUs in 15 minutes. Lastly, TSM enables action concepts learning, which 2D networks cannot model; we visualize the category attention map and find that spatial-temporal action detector emerges during the training of classification tasks. The code is publicly available at https://github.com/mit-han-lab/temporal-shift-module. [ABSTRACT FROM AUTHOR]

Over-the-air computation (AirComp) has emerged as a new analog power-domain non-orthogonal multiple access (NOMA) technique for low-latency model/gradient-updates aggregation in federated edge learning (FEEL). By integrating communication and computation into a joint design, AirComp can significantly enhance the communication efficiency, but at the cost of aggregation errors caused by channel fading and noise. This paper studies a particular type of FEEL with federated averaging (FedAvg) and AirComp-based model-update aggregation, namely over-the-air FedAvg (Air-FedAvg). We investigate the transmission power control in Air-FedAvg to combat against the AirComp aggregation errors for enhancing the training accuracy and accelerating the training speed. Towards this end, we first analyze the convergence behavior (in terms of the optimality gap) of Air-FedAvg with aggregation errors at different outer iterations. Then, to enhance the training accuracy, we minimize the optimality gap by jointly optimizing the transmission power control at edge devices and the denoising factors at edge server, subject to a series of power constraints at individual edge devices. Furthermore, to accelerate the training speed, we also minimize the training latency of Air-FedAvg with a given targeted optimality gap, in which learning hyper-parameters including the numbers of outer iterations and local training epochs are optimized jointly with the power control. Finally, numerical results show that the proposed transmission power control policy achieves significantly faster convergence speed for Air-FedAvg, as compared with benchmark policies with fixed power transmission or per-iteration mean squared error (MSE) minimization. It is also shown that the Air-FedAvg achieves an order-of-magnitude shorter training latency than the conventional FedAvg with digital orthogonal multiple access (OMA-FedAvg). [ABSTRACT FROM AUTHOR]

How should affect be appropriately annotated and how should machine learning best be employed to map manifestations of affect to affect annotations? What is the use of ratings of affect for the study of affective computing and how should we treat them? These are the key questions this paper attempts to address by investigating the impact of dissimilar representations of annotated affect on the efficacy of affect modelling. In particular, we compare several different binary-class and pairwise preference representations for automatically learning from ratings of affect. The representations are compared and tested on three datasets: one synthetic dataset (testing “in vitro ”) and two affective datasets (testing “in vivo”). The synthetic dataset couples a number of attributes with generated rating values. The two affective datasets contain physiological and contextual user attributes, and speech attributes, respectively; these attributes are coupled with ratings of various affective and cognitive states. The main results of the paper suggest that ratings (when used) should be naturally transformed to ordinal (ranked) representations for obtaining more reliable and generalisable models of affect. The findings of this paper have a direct impact on affect annotation and modelling research but, most importantly, challenge the traditional state-of-practice in affective computing and psychometrics at large. [ABSTRACT FROM PUBLISHER]

Image retargeting techniques adjust images into different sizes and have attracted much attention recently. Objective quality assessment (OQA) of image retargeting results is often desired to automatically select the best results. Existing OQA methods train a model using some benchmarks (e.g., RetargetMe), in which subjective scores evaluated by users are provided. Observing that it is challenging even for human subjects to give consistent scores for retargeting results of different source images (diff-source-results), in this paper we propose a learning-based OQA method that trains a General Regression Neural Network (GRNN) model based on relative scores—which preserve the ranking—of retargeting results of the same source image (same-source-results). In particular, we develop a novel training scheme with provable convergence that learns a common base scalar for same-source-results. With this source specific offset, our computed scores not only preserve the ranking of subjective scores for same-source-results, but also provide a reference to compare the diff-source-results. We train and evaluate our GRNN model using human preference data collected in RetargetMe. We further introduce a subjective benchmark to evaluate the generalizability of different OQA methods. Experimental results demonstrate that our method outperforms ten representative OQA methods in ranking prediction and has better generalizability to different datasets. [ABSTRACT FROM AUTHOR]

Cross-validation (CV) is a widely adopted approach for selecting the optimal model. However, the computation of empirical cross-validation error (CVE) has high complexity due to multiple times of learner training. In this paper, we develop a novel approximation theory of CVE and present an approximate approach to CV based on the Bouligand influence function (BIF) for kernel-based algorithms. We first represent the BIF and higher order BIFs in Taylor expansions, and approximate CV via the Taylor expansions. We then derive an upper bound of the discrepancy between the original and approximate CV. Furthermore, we provide a novel computing method to calculate the BIF for general distribution, and evaluate BIF criterion for sample distribution to approximate CV. The proposed approximate CV requires training on the full data set only once and is suitable for a wide variety of kernel-based algorithms. Experimental results demonstrate that the proposed approximate CV is sound and effective. [ABSTRACT FROM AUTHOR]

The inherent hysteresis property of piezoelectric actuator (PEA) brings challenges to its modeling and control. This paper proposes a model learning method that is suitable for both forward and inverse PEA models. The hysteresis property is learned based on least squares support vector machines (LS-SVMs). A larger dataset is used for training LS-SVM to guarantee a good generalization performance. Support vectors pruning is utilized to reduce the model complexity. The rate-dependent property of PEA is identified as a linear dynamic submodel. Moreover, a pointing control system with two dual-PEA-axis steering mirrors is developed, which can regulate the 4-degree-of-freedom pose of a laser beam. The coordinated control of four PEAs is realized based on the Jacobian matrix. The learned inverse PEA models are used for the feedforward compensation of each PEA’s nonlinearity. A series of experiments were conducted to evaluate the proposed method’s effectiveness. [ABSTRACT FROM AUTHOR]

Automatic target recognition (ATR) is an important part for many computer vision applications. Despite the extensive research which has been carried out in this area for many years, there is no ATR system which performs well on all applications. Recently, different object recognition frameworks have been proposed which yield a high performance in baseline databases. However, our experiments showed that they can fail in real-world scenarios, when dealing with a limited number of data samples. In this paper, we propose a new ATR system, based on deep convolutional neural network (DCNN), to detect the targets in forward looking infrared (FLIR) scenes and recognize their classes. In our proposed ATR framework, a fully convolutional network is trained to map the input FLIR imagery data to a fixed stride correspondingly-sized target score map. The potential targets are identified by applying a threshold on the target score map. Finally, the corresponding regions centered at these target points are fed to a DCNN to classify them into different target types while at the same time rejecting the false alarms. The proposed architecture achieves a significantly better performance in comparison with that of the state-of-the-art methods on two large FLIR image databases. [ABSTRACT FROM AUTHOR]

We present a novel approach to deformable object manipulation that does not rely on highly accurate modeling. The key contribution of this paper is to formulate the task as a multiarmed bandit problem, with each arm representing a model of the deformable object. To “pull” an arm and evaluate its utility, we use the arm’s model to generate a velocity command for the gripper(s) holding the object and execute it. As the task proceeds and the object deforms, the utility of each model can change. Our framework estimates these changes and balances the exploration of the model set with exploitation of high-utility models. We also propose an approach based on Kalman filtering for nonstationary multiarmed normal bandits to leverage the coupling between the models to learn more from each arm pull. We demonstrate that our method outperforms previous methods on synthetic trials and performs competitively on several manipulation tasks in simulation. Note to Practitioners—This paper is motivated by the problem of how to choose appropriate parameters for models used in deformable object manipulation. Existing approaches use time-consuming data collection and analysis methods to generate models for a given object and/or task. In contrast, our approach can be applied to multiple deformable objects or tasks without this time-consuming model generation step, learning what models are useful during the execution of the task itself. This enables the faster development of systems for a wide range of tasks, including those where a priori data collection is not possible. In this paper, we are only considering the modeling and control aspects of deformable object manipulation. These ideas can then be integrated with sensing and planning systems for commercial/industrial applications. We briefly discuss some methods and considerations for such an integration at the end of this paper. [ABSTRACT FROM AUTHOR]

The peak-to-average power ratio (PAPR) reduction and linearization techniques are both effective methods to improve the efficiency of the transmitter in digital video broadcasting (DVB) systems. Traditional methods deploy the PAPR reduction model and the linearization model, respectively, without considering their mutual influence. Therefore, the joint optimizations of PAPR reduction and linearization techniques are proposed. However, these methods train the PAPR reduction model and the linearization model based on the time-division training method. It is difficult to meet the requirements of multiple objectives. To address this issue, this paper proposes a joint PAPR reduction and digital predistortion (DPD) method using the real-valued neural network (RVNN) for Orthogonal Frequency Division Multiplexing (OFDM) systems. The proposed method jointly trains the PAPR reduction function and the DPD function with multi-objective optimization, to achieve PAPR reduction and linearization simultaneously. Especially, this method unifies the PAPR reduction function and the DPD function into one model based on RVNN, and no extra processing is required at the receiver. Compared with the traditional methods, the experimental results show that the proposed method has superior performance in PAPR, adjacent channel power ratio (ACPR) and bit error rate (BER), while having lower computational complexity. [ABSTRACT FROM AUTHOR]

Learning algorithms based on synchronization of oscillators are currently being pursued as an alternative to standard neural-network algorithms for data classification. For such oscillator-based algorithms, while learning can be implemented off-line on a computer, on-chip inference/ testing can be implemented on an array of uniform-mode spin Hall nano-oscillators (SHNOs), leading to a scalable and energy-efficient technology. Here, we propose a modification to an existing oscillator-based off-line learning algorithm for binary classification: unlike in the previous version of the algorithm, here the difference between natural frequencies of the output oscillators is kept constant throughout the learning process. This helps in preservation of the shape of the synchronization region and leads to higher classification accuracy, as we show for binary-classification tasks using two popular data sets: Fisher's Iris and MNIST. Next, in this paper, we show how a synchronization pattern obtained after such training, using our proposed algorithm, can be implemented on an array of dipole-coupled uniform-mode SHNOs for on-chip inference. We model the SHNO system through the macro-spin model, which is computationally much less resource-intensive to simulate compared to the micromagnetic model used previously for this kind of study. We also extend our algorithm to multi-class classification and thereby discuss scaling of our system. [ABSTRACT FROM AUTHOR]

Mobile crowdsensing (MCS) combined with federated learning, as an emerging data collection and intelligent process paradigm, has received lots of attention in social networks and mobile Internet-of-Things, etc. However, as the openness and transparent of mobile crowdsensing tasks, federated learning model and training samples for crowdsensing data still face enormous privacy revealing risks, and it will reduce the willingness of people or nodes to actively participate and provide data in MCS. In this paper, we present a Privacy-Enhanced Aggregation for Federated Learning in MCS, namely PrivacyEAFL, to implement the training of federated learning under mobile crowdsensing system in terms of privacy protection of all participants. Firstly, considering that the crowdsensing server might share information with some participants to obtain and leak some local models, we design a collusion-resistant data aggregation approach by combining homomorphic cryptosystem and hashed Diffie-Hellman key exchange protocol. Secondly, we design a data encoding and aggregating method with data packing which can reduce the computation cost and communication overhead for the system. Thirdly, as the number of participants’ samples are dynamically changeable in MCS, we design a sample number protection method that can implement the security and privacy of the number of training samples owned by participants. Finally, we provide the experimental results on real-world datasets (i.e, MNIST and Car Evaluation) with crowdsensing devices under Raspberry-Pi 4B and Redmi-K30 Pro, respectively, and the results demonstrate that our scheme is more efficient and practical in secure and privacy-enhanced model aggregation for federated learning in mobile crowdsensing. [ABSTRACT FROM AUTHOR]

Federated learning (FL) is a privacy-preserving machine learning paradigm that enables multiple clients to train a unified model without disclosing their private data. However, susceptibility to membership inference attacks (MIAs) arises due to the natural inclination of FL models to overfit on the training data during the training process, thereby enabling MIAs to exploit the subtle differences in the FL model’s parameters, activations, or predictions between the training and testing data to infer membership information. It is worth noting that most if not all existing MIAs against FL require access to the model’s internal information or modification of the training process, yielding them unlikely to be performed in practice. In this paper, we present with TEAR the first evidence that it is possible for an honest-but-curious federated client to perform MIA against an FL system, by exploring the Temporal Evolution of the Adversarial Robustness between the training and non-training data. We design a novel adversarial example generation method to quantify the target sample’s adversarial robustness, which can be utilized to obtain the membership features to train the inference model in a supervised manner. Extensive experiment results on five realistic datasets demonstrate that TEAR can achieve a strong inference performance compared with two existing MIAs, and is able to escape from the protection of two representative defenses. [ABSTRACT FROM AUTHOR]

Prediction of human actions in social interactions has important applications in the design of social robots or artificial avatars. In this paper, we focus on a unimodal representation of interactions and propose to tackle interaction generation in a data-driven fashion. In particular, we model human interaction generation as a discrete multi-sequence generation problem and present SocialInteractionGAN, a novel adversarial architecture for conditional interaction generation. Our model builds on a recurrent encoder-decoder generator network and a dual-stream discriminator, that jointly evaluates the realism of interactions and individual action sequences and operates at different time scales. Crucially, contextual information on interacting participants is shared among agents and reinjected in both the generation and the discriminator evaluation processes. Experiments show that albeit dealing with low dimensional data, SocialInteractionGAN succeeds in producing high realism action sequences of interacting people, comparing favorably to a diversity of recurrent and convolutional discriminator baselines, and we argue that this work will constitute a first stone towards higher dimensional and multimodal interaction generation. Evaluations are conducted using classical GAN metrics, that we specifically adapt for discrete sequential data. Our model is shown to properly learn the dynamics of interaction sequences, while exploiting the full range of available actions. [ABSTRACT FROM AUTHOR]

There is growing interest in affective computing for the representation and prediction of emotions along ordinal scales. However, the term ordinal emotion label has been used to refer to both absolute notions such as low or high arousal, as well as relation notions such as arousal is higher at one instance compared to another. In this paper, we introduce the terminology absolute and relative ordinal labels to make this distinction clear and investigate both with a view to integrate them and exploit their complementary nature. We propose a Markovian framework referred to as Dynamic Ordinal Markov Model (DOMM) that makes use of both absolute and relative ordinal information, to improve speech based ordinal emotion prediction. Finally, the proposed framework is validated on two speech corpora commonly used in affective computing, the RECOLA and the IEMOCAP databases, across a range of system configurations. The results consistently indicate that integrating relative ordinal information improves absolute ordinal emotion prediction. [ABSTRACT FROM AUTHOR]

Acquisition of labeled training samples for affective computing is usually costly and time-consuming, as affects are intrinsically subjective, subtle and uncertain, and hence multiple human assessors are needed to evaluate each affective sample. Particularly, for affect estimation in the 3D space of valence, arousal and dominance, each assessor has to perform the evaluations in three dimensions, which makes the labeling problem even more challenging. Many sophisticated machine learning approaches have been proposed to reduce the data labeling requirement in various other domains, but so far few have considered affective computing. This paper proposes two multi-task active learning for regression approaches, which select the most beneficial samples to label, by considering the three affect primitives simultaneously. Experimental results on the VAM corpus demonstrated that our optimal sample selection approaches can result in better estimation performance than random selection and several traditional single-task active learning approaches. Thus, they can help alleviate the data labeling problem in affective computing, i.e., better estimation performance can be obtained from fewer labeling queries. [ABSTRACT FROM AUTHOR]