Search

In recent years, distributed machine learning in WANs (DML-WANs), i.e., collaboratively training a high-quality ML model cross geo-distributed micro-clouds or edge devices, has attracted attention and been widely applied. Compared with cloud-centric training, DML-WANs avoids the high cost of transferring large amounts of raw data to a central cloud and privacy concerns. However, performing DML-WANs still faces challenges. Model synchronization, an essential step of DML-WANs, is accompanied by a lot of model communication cross limited-bandwidth WANs, which generates high communication overhead. Moreover, the parameter server system, which has been widely used, performs model synchronization in a centralized manner, resulting in serious communication in-cast problem. Such communication in-cast further raises the communication overhead, leading to the low efficiency of DML-WANs. To alleviate the communication in-cast, existing researches attempt to build tree-based communication overlays over the parameter server and workers. However, we identify that these approaches can not adapt to the dynamic and heterogeneous network of DML-WANs, resulting in insufficient improvements. This paper proposes TSEngine, an adaptive communication scheduler for efficient communication overlay of the parameter server system in DML-WANs. Its core idea is to dynamically schedule the communication logic over the parameter server and workers based on the active network perception. Specifically, we propose novel communication scheduling protocols for model distribution and model aggregation, respectively. We have implemented TSEngine in a mainstream parameter server system and verified its effectiveness in DML-WANs testbeds. [ABSTRACT FROM AUTHOR]

Since heavy flows account for a significant fraction of network traffic, being able to predict heavy flows has benefited many network management applications for mitigating link congestion, scheduling of network capacity, exposing network attacks and so on. Existing machine learning based predictors are largely implemented on the control plane of Software Defined Networking (SDN) paradigm. As a result, frequent communication between the control and data planes can cause unnecessary overhead and additional delay in decision making. In this paper, we present pHeavy, a machine learning based scheme for predicting heavy flows directly on the programmable data plane, thus eliminating network overhead and latency to SDN controller. Considering the scarce memory and limited computation capability in the programmable data plane, pHeavy includes a packet processing pipeline which deploys pre-trained decision tree models for in-network prediction. We have implemented pHeavy in both bmv2 software switch and P4 hardware switch (i.e., Barefoot Tofino). Evaluation results demonstrate that pHeavy has achieved 85% and 98% accuracy after receiving the first 5 and 20 packets of a flow respectively, while being able to reduce the size of decision tree by 5.4x on average. More importantly, pHeavy can predict heavy flows at line rate on the P4 hardware switch. [ABSTRACT FROM AUTHOR]

In this paper, we propose MiT: a novel multi-view transformer model for 3D/4D facial affect recognition. MiT incorporates patch and position embeddings from various patches of multi-views and uses them for learning various facial muscle movements to showcase an effective recognition performance. We also propose a multi-view loss function that is not only gradient-friendly, and hence speeds up the gradient computation during back-propagation, but it also leverages the correlation associated with the underlying facial patterns among multi-views. Additionally, we offer multi-view weights that are trainable and learnable, and help substantially in training. Finally, we equip our model with distributed performance for faster learning and computational convenience. With the help of extensive experiments, we show that our model outperform the existing methods on widely-used datasets for 3D/4D FER. [ABSTRACT FROM AUTHOR]

Membership inference attacks seek to infer membership of individual training instances of a model to which an adversary has black-box access through a machine learning-as-a-service API. In providing an in-depth characterization of membership privacy risks against machine learning models, this paper presents a comprehensive study towards demystifying membership inference attacks from two complimentary perspectives. First, we provide a generalized formulation of the development of a black-box membership inference attack model. Second, we characterize the importance of model choice on model vulnerability through a systematic evaluation of a variety of machine learning models and model combinations using multiple datasets. Through formal analysis and empirical evidence from extensive experimentation, we characterize under what conditions a model may be vulnerable to such black-box membership inference attacks. We show that membership inference vulnerability is data-driven and corresponding attack models are largely transferable. Though different model types display different vulnerabilities to membership inference, so do different datasets. Our empirical results additionally show that (1) using the type of target model under attack within the attack model may not increase attack effectiveness and (2) collaborative learning exposes vulnerabilities to membership inference risks when the adversary is a participant. We also discuss countermeasure and mitigation strategies. [ABSTRACT FROM AUTHOR]

Predicting the future trajectories of surrounding agents has become an crucial problem to be solved for the safety of autonomous vehicles. Recent studies based on Long Short Term Memory (LSTM) networks have shown powerful abilities to model social interactions. However, many of these approaches focus on spatial interactions of the neighborhood agents but ignore temporal interactions that accompany spatial interactions. In this paper, we propose a Hierarchical Spatio-Temporal Attention architecture (HSTA), which activates the utilization of spatial interactions with different weights, and jointly considers the temporal interactions across time steps of all agents. More specially, the graph attention mechanism (GAT) is presented to capture spatial interactions, the multi-head attention mechanism (MHA) is conducted to encode temporal correlations of interactions and a state gated fusion (SGF) layer is used to integrate spatial and temporal interactions. We evaluate our proposed method against baselines on both pedestrian and vehicle datasets. The results show that our model is effective and achieves state-of-the-art achievements. [ABSTRACT FROM AUTHOR]

Radar radio source (RRS) recognition plays an important role in the fields of military electronic support systems (ESM) and civilian autonomous driving. The rapid development of machine learning technology, especially deep learning, has effectively and efficiently improved RRS intelligent recognition performances when operating in the increasingly complex electromagnetic environment. However, the data sampling limitation and computation cost are still severe challenges in real RRS recognition scenarios. In this paper, we propose a novel network based on meta-transfer learning, called RRSARNet, to achieve effective adaptive RRS recognition in the context of low signal-to-noise ratio (SNR). First, by using the short-time Fourier transform, a six-type small samples RRS simulation dataset with different SNR levels is constructed. Then, a novel RRSARNet, based on metric learning, is proposed, which consists of a four-layer embedding module and a four-layer relational module. Finally, the RRS dataset is divided into training, supporting and testing subsets, which are used to train and test the RRSARNet in a meta-transfer learning method. Experiments on the RRS dataset show that the proposed RRSARNet can achieve an overall accuracy (OA) above 96% and 99% when the SNR is above −15 dB and −10 dB, respectively. Even when the SNR is -30 dB, OA can reach more than 70%. For 5-way 1-shot and 5-way 5-shot experiments, the inference time of an image is about 0.043 and 0.140 milliseconds, respectively. Besides, experiments on the RRS simulation dataset and the two benchmark datasets, the RRSARNet performs better or more competitive than many existing state-of-the-art technologies in terms of recognition accuracy. [ABSTRACT FROM AUTHOR]

In this paper, we provide a variational Bayesian (VB) treatment of multistream fused hidden Markov models (MFHMMs), and apply it in the context of active learning-based visual workflow recognition (WR). Contrary to training methods yielding point estimates, such as maximum likelihood or maximum a posteriori training, the VB approach provides an estimate of the posterior distribution over the MFHMM parameters. As a result, our approach provides an elegant solution toward the amelioration of the overfitting issues of point estimate-based methods. Additionally, it provides a measure of confidence in the accuracy of the learned model, thus allowing for the easy and cost-effective utilization of active learning in the context of MFHMMs. Two alternative active learning algorithms are considered in this paper: query by committee, which selects unlabeled data that minimize the classification variance, and a maximum information gain method that aims to maximize the alteration in model variance by proper data labeling. We demonstrate the efficacy of the proposed treatment of MFHMMs by examining two challenging WR scenarios, and show that the application of active learning, which is facilitated by our VB approach, allows for a significant reduction of the MFHMM training costs. [ABSTRACT FROM AUTHOR]

A number of visual tracking methods achieve the state-of-the-art performance based on deep learning recently. However, most of these trackers utilize the deep neural network in regression task or classification task separately. In this paper, we propose an adversarial deep tracking framework. The framework is composed of a fully convolutional Siamese neural network (regression network) and a discriminative classification network. Then, we jointly optimize the regression network and the classification network by adversarial learning. In the uniform framework, the regression network and classification network can be trained end-to-end as a whole using large amounts of video training data sets. During the testing phase, the regression network generates a response map which reflects the location and the size of the target within each candidate search patch, and the classification network discriminates which response map is the best in terms of the corresponding template patch and candidate search patch. In addition, we propose an attention visualization algorithm for our tracker, and it reflects the area that attracts the attention of our tracker during tracking. The experimental results on three large-scale visual tracking benchmarks (OTB-100, TC-128, and VOT2016) demonstrate the effectiveness of the proposed tracking algorithm and show that our tracker performs comparably against the state-of-the-art trackers. [ABSTRACT FROM AUTHOR]

Image representation based on sparse coding generalizes the bag of words model. Although it reduces the reconstruction error for local features to achieve the state-of-the-art image classification performance, the large computational cost hinders the application of sparse coding-based image features. In this paper, we propose approximating a sparse code using the output of a simple neural network. The resulting parameter learning model for the neural network automatically incorporates non-negative and shift-invariant constraints, leading to an efficient normalized non-negative sparse coding (N3SC) sparse encoder. Without the use of the traditional iterative process to solve the sparse coding objective, the sparse encoder directly “converts” each local feature into a sparse code. We also introduce a method for training the encoder based on the auto-encoder method. In addition, we formally propose the corresponding sparse coding scheme called N3SC, which enforces both the non-negative constraint and the shift-invariant constraint in addition to the traditional sparse coding criteria. As demonstrated by several experiments, the obtained N3SC encoder requires only 3%–10% of the processing time for image feature extraction compared with the standard sparse coding scheme. At the same time, the features extracted using the exact solutions of the N3SC coding scheme and the N3SC encoder offer superior image classification accuracy compared to the accuracy of many existing sparse coding-based representations. [ABSTRACT FROM AUTHOR]

This paper presents an ultra-low-power voice activity detector (VAD). It uses analog signal processing for acoustic feature extraction (AFE) directly on the microphone output, approximate event-driven analog-to-digital conversion (ED-ADC), and digital deep neural network (DNN) for speech/non-speech classification. New circuits, including the low-noise amplifier, bandpass filter, and full-wave rectifier contribute to the more than 9 $\times $ normalized power/channel reduction in the feature extraction front-end compared to the best prior art. The digital DNN is a three-hidden-layer binarized multilayer perceptron (MLP) with a 2-neuron output layer and a 48-neuron input layer that receives parallel event streams from the ED-ADCs. To obtain the DNN weights via off-line training, a customized front-end model written in python is constructed to accelerate feature generation in software emulation, and the model parameters are extracted from Spectre simulations. The chip, fabricated in 0.18- $\mu \text{m}$ CMOS, has a core area of 1.66 $\times $ 1.52 mm2 and consumes 1 $\mu \text{W}$. The classification measurements using the 1-hour 10-dB signal-to-noise ratio audio with restaurant background noise show a mean speech/non-speech hit rate of 84.4%/85.4% with a 1.88%/4.65% 1- $\sigma $ variation across ten dies that are all loaded with the same weights. [ABSTRACT FROM AUTHOR]

Deep computation, as an advanced machine learning model, has achieved the state-of-the-art performance for feature learning on big data in industrial Internet of Things (IoT). However, the current deep computation model usually suffers from overfitting due to the lack of public available labeled training samples, limiting its performance for big data feature learning. Motivated by the idea of active learning, an adaptive dropout deep computation model (ADDCM) with crowdsourcing to cloud is proposed for industrial IoT big data feature learning in this paper. First, a distribution function is designed to set the dropout rate for each hidden layer to prevent overfitting for the deep computation model. Furthermore, the outsourcing selection algorithm based on the maximum entropy is employed to choose appropriate samples from the training set to crowdsource on the cloud platform. Finally, an improved supervised learning from multiple experts scheme is presented to aggregate answers given by human workers and to update the parameters of the ADDCM simultaneously. Extensive experiments are conducted to evaluate the performance of the presented model by comparing with the dropout deep computation model and other state-of-the-art crowdsourcing algorithms. The results demonstrate that the proposed model can prevent overfitting effectively and aggregate the labeled samples to train the parameters of the deep computation model with crowdsouring for industrial IoT big data feature learning. [ABSTRACT FROM AUTHOR]

In this paper, a novel feature extraction method is proposed for hyperspectral image classification using a Gaussian–Bernoulli restricted Boltzmann machine (GBRBM) in parallel. The proposed approach employs several GBRBMs with different hidden layers to extract deep features from hyperspectral images, which are nonlinear and local invariant. Based on the learned deep features, a logistic regression layer is trained for classification. The proposed approaches are carried out on two public hyperspectral datasets: Pavia University dataset and Salinas dataset, and a new dataset obtained by HySpex imaging spectrometer in the mining area in Xuzhou. The obtained results reveal that the proposed approach offers superior performance compared to traditional classifiers. The advantage of the proposed GBRBM is that it can extract deep features in an unsupervised way and reduce the prediction time by using GPU. In particular, the classification results of the mining area provide valuable suggestions to improve environmental protection. [ABSTRACT FROM AUTHOR]

Representation-based classification (RC) methods such as sparse RC (SRC) have attracted great interest in pattern recognition recently. Despite their empirical success, few theoretical results are reported to justify their effectiveness. In this paper, we establish the theoretical guarantees for a general unified framework termed as atomic representation-based classification (ARC), which includes most RC methods as special cases. We introduce a new condition called atomic classification condition (ACC), which reveals important geometric insights for the theory of ARC. We show that under such condition ARC is provably effective in correctly recognizing any new test sample, even corrupted with noise. Our theoretical analysis significantly broadens the range of conditions under which RC methods succeed for classification in the following two aspects: (1) prior theoretical advances of RC are mainly concerned with the single SRC method while our theory can apply to the general unified ARC framework, including SRC and many other RC methods; and (2) previous works are confined to the analysis of noiseless test data while we provide theoretical guarantees for ARC using both noiseless and noisy test data. Numerical results are provided to validate and complement our theoretical analysis of ARC and its important special cases for both noiseless and noisy test data. [ABSTRACT FROM AUTHOR]

The increasing uncertainty that surrounds electricity system operation renders security assessment a highly challenging task; the range of possible operating states expands, rendering traditional approaches based on heuristic practices and ad hoc analysis obsolete. In turn, machine learning can be used to construct surrogate models approximating the system's security boundary in the region of operation. To this end, past system history can be useful for generating anticipated system states suitable for training. However, inferring the underlying data model, to allow high-density sampling, is problematic due to the large number of variables, their complex marginal probability distributions and the nonlinear dependence structure they exhibit. In this paper, we adopt the C-Vine pair-copula decomposition scheme; clustering and principal component transformation stages are introduced, followed by a truncation to the pairwise dependency modeling, enabling efficient fitting and sampling of large datasets. Using measurements from the French grid, we show that a machine learning training database sampled from the proposed method can produce binary security classifiers with superior predictive capability compared to other approaches. [ABSTRACT FROM AUTHOR]

When implementing unfamiliar programming tasks, developers commonly search code examples and learn usage patterns of APIs from the code examples or reuse them by copy-pasting and modifying. For providing high-quality code examples, previous studies present several methods to recommend code snippets mainly based on information retrieval. In this paper, to provide better recommendation results, we propose ROSF, Recommending code Snippets with multi-aspect Features, a novel method combining both information retrieval and supervised learning. In our method, we recommend Top-K code snippets for a given free-form query based on two stages, i.e., coarse-grained searching and fine-grained re-ranking. First, we generate a code snippet candidate set by searching a code snippet corpus using an information retrieval method. Second, we predict probability values of the code snippets for different relevance scores in the candidate set by the learned prediction model from a training set, re-rank these candidate code snippets according to the probability values, and recommend the final results to developers. We conduct several experiments to evaluate our method in a large-scale corpus containing 921,713 real-world code snippets. The results show that ROSF is an effective method for code snippets recommendation and outperforms the-state-of-the-art methods by 20-41percent in Precision and 13-33 percent in NDCG. [ABSTRACT FROM AUTHOR]

This paper presents a multi-resolution modeling approach using wavelet neural networks. A lithium battery model, which has three resolutions, is developed to depict the modeling approach. By combining the advantages of dyadic activation functions and the orthonormal property of wavelet functions, the developed battery model possesses two salient features. First, the model is built from a coarser approximation to a finer representation by adding more details incrementally. Second, the model at a low resolution is compatible with the model at a high resolution, which means that the parameters used in a low resolution can be directly incorporated into a high resolution without any modification. This paper's results show that this battery modeling provides great flexibility for users to choose a suitable resolution to meet their requirements for model accuracy and model execution speed. [ABSTRACT FROM PUBLISHER]

Gradient descent (GD) is a widely used optimization algorithm in machine learning. In this paper, a novel stochastic computing GD circuit (SC-GDC) is proposed by encoding the gradient information in stochastic sequences. Inspired by the structure of a neuron, a stochastic integrator is used to optimize the weights in a learning machine by its “inhibitory” and “excitatory” inputs. Specifically, two AND (or XNOR) gates for the unipolar representation (or the bipolar representation) and one stochastic integrator are, respectively, used to implement the multiplications and accumulations in a GD algorithm. Thus, the SC-GDC is very area- and power-efficient. As per the formulation of the proposed SC-GDC, it provides unbiased estimate of the optimized weights in a learning algorithm. The proposed SC-GDC is then used to implement a least-mean-square algorithm and a softmax regression. With a similar accuracy, the proposed design achieves more than $30 \times $ improvement in throughput per area (TPA) and consumes less than 13% of the energy per training sample, compared with a fixed-point implementation. Moreover, a signed SC-GDC is proposed for training complex neural networks (NNs). It is shown that for a 784-128-128-10 fully connected NN, the signed SC-GDC produces a similar training result with its fixed-point counterpart, while achieving more than 90% energy saving and 82% reduction in training time with more than $50 \times $ improvement in TPA. [ABSTRACT FROM AUTHOR]

Deep computation models achieve super performance for big data feature learning. However, training a deep computation model poses a significant challenge since a deep computation model typically involves a large number of parameters. Specially, it needs a high-performance computing server with a large-scale memory and a powerful computing unit to train a deep computation model, making it difficult to increase the size of a deep computation model further for big data feature learning on low-end devices such as conventional desktops and portable CPUs. In this paper, we propose an improved deep computation model based on the canonical polyadic decomposition scheme to compress the parameters and to improve the training efficiency. Furthermore, we devise a learning algorithm based on the back-propagation strategy to train the parameters of the proposed model. The learning algorithm can be directly performed on the compressed parameters to improve the training efficiency. Finally, we carry on the experiments on three representative datasets, i.e., CUAVE, SNAE2, and STL-10, to evaluate the performance of the proposed model by comparing with the conventional deep computation model and other two improved deep computation models based on the Tucker decomposition and the tensor-train network. Results demonstrate that the proposed model can compress parameters greatly and improve the training efficiency significantly with a low classification accuracy drop. [ABSTRACT FROM AUTHOR]

Nonlinear complex system modeling has drawn attention from diverse fields and many approaches have been developed. Among those approaches, the advantages of the belief rule base (BRB) expert system have been shown for managing multiple types of information under uncertainty and modeling the nonlinearity present in many theoretical and practical complex systems. However, two challenges still need to be addressed. First, BRB needs to be downsized to conserve modeling and computational effort. For this challenge, a new disjunctive assumption is applied, which can significantly downsize BRB while maintaining its completeness. Second, the structure and parameters of BRB need to be jointly optimized. For this challenge, a new Akaike information criterion (AIC)-based optimization objective is derived to represent both modeling accuracy and modeling complexity. Moreover, a joint bi-level optimization model with an AIC-based objective is constructed for the BRB structure and parameters, and a bi-level optimization algorithm is proposed. Three evolutionary algorithms, namely, the genetic algorithm, particle swarm optimization algorithm, and differential evolutionary algorithm, are tested in a comparative fashion to determine the best fit for the optimization engine. The results of two practical case studies show that the joint optimization approach can identify an optimal configuration for both its structure and parameters, which is referred to as the best decision structure in this paper. [ABSTRACT FROM AUTHOR]

Unmanned aerial vehicles (UAVs) are capable of serving as flying base stations (BSs) for supporting data collection, machine learning (ML) model training, and wireless communications. However, due to the privacy concerns of devices and limited computation or communication resource of UAVs, it is impractical to send raw data of devices to UAV servers for model training. Moreover, due to the dynamic channel condition and heterogeneous computing capacity of devices in UAV-enabled networks, the reliability and efficiency of data sharing require to be further improved. In this paper, we develop an asynchronous federated learning (AFL) framework for multi-UAV-enabled networks, which can provide asynchronous distributed computing by enabling model training locally without transmitting raw sensitive data to UAV servers. The device selection strategy is also introduced into the AFL framework to keep the low-quality devices from affecting the learning efficiency and accuracy. Moreover, we propose an asynchronous advantage actor-critic (A3C) based joint device selection, UAVs placement, and resource management algorithm to enhance the federated convergence speed and accuracy. Simulation results demonstrate that our proposed framework and algorithm achieve higher learning accuracy and faster federated execution time compared to other existing solutions. [ABSTRACT FROM AUTHOR]

Recent workat the U.S. Army CCDC Ground Vehicle Systems Center has shown that machine learning classifiers can quickly construct high-fidelity mobility maps. Training these classifiers, on the other hand, is still a challenge, since each data instance is labeled by performing a computationally intensive, physics-based simulation. In this paper we introduce an active learning framework, based on the query-by-bagging algorithm, that substantially reduces the number of simulations needed to train a classifier. Experimental results suggest that our sampling algorithm can train a neural network, with higher accuracy, using less than half the number of simulations when compared to random sampling. [ABSTRACT FROM AUTHOR]

Multi-access edge computing (MEC) has already shown great potential in enabling mobile devices to bear the computation-intensive applications by offloading some computing jobs to a nearby access point (AP) integrated with a MEC server (MES). However, due to the varying network conditions and limited computational resources of the MES, the offloading decisions taken by a mobile device and the computational resources allocated by the MES can be formulated as a mixed-integer nonlinear programming (MINLP) problem, which may not be optimized with the lowest cost. In this paper, we propose a novel offloading framework for the multi-server MEC network where each AP is equipped with an MES assisting mobile users (MUs) in executing computation-intensive jobs via offloading. Specifically, we formulate the offloading decision problem as a multiclass classification problem and formulate the MES computational resource allocation problem as a regression problem. Then a multi-task learning based feedforward neural network (MTFNN) model is designed and trained to jointly optimize the offloading decision and computational resource allocation. Numerical results show that the proposed MTFNN outperforms the conventional optimization method in terms of inference accuracy and computational complexity. [ABSTRACT FROM AUTHOR]

Distant supervision is widely used to extract relational facts with automatically labeled datasets to reduce high cost of human annotation. However, current distantly supervised methods suffer from the common problems of word-level and sentence-level noises, which come from a large proportion of irrelevant words in a sentence and inaccurate relation labels for numerous sentences. The problems lead to unacceptable precision in relation extraction and are critical for the success of using distant supervision. In this paper, we propose a novel and robust neural approach to deal with both problems by reducing influences of the multi-granularity noises. Three levels of noises from word, sentence until knowledge type are carefully considered in this work. We first initiate a question-answering based relation extractor (QARE) to remove noisy words in a sentence. Then we use multi-focus multi-instance learning (MMIL) to alleviate the effects of sentence-level noise by utilizing wrongly labeled sentences properly. Finally, to enhance our method against all the noises, we initialize parameters in our method with a priori knowledge learned from the relevant task of entity type classification by transfer learning. Extensive experiments on both existing benchmark and an improved larger dataset demonstrate that our proposed approach remarkably achieves new state-of-the-art performance. [ABSTRACT FROM AUTHOR]

Deep reinforcement learning (DRL) enables autonomic optical networking by allowing the network control and management systems to self-learn successful networking policies from operational experiences. This paper proposes a transfer learning approach for effective and scalable DRL in optical networks. We first present a modular DRL agent design to facilitate retrieving and transferring relevant knowledge between tasks requiring different dimensions of network state data. In particular, we partition network state data into common states, which contain generic information critical to multiple tasks (e.g., the spectrum utilization on fiber links), and task-specific states that are only used by a specific task (e.g., the utilization of virtual network functions). Separate neural network blocks are employed to process different state data. Based on the modular agent design, a multi-task learning (MTL) aided knowledge transferring scheme is proposed. The proposed scheme trains an MTL agent that can master multiple tasks simultaneously and thus enables to learn and transfer better-generalized knowledge across tasks. We perform case studies on the proposed transfer DRL approach considering two scenarios, namely, knowledge transferring between routing, modulation and spectrum assignment (RMSA) tasks for different networks and knowledge transferring from RMSA tasks to anycast service provisioning tasks. The DRL designs for RMSA and anycast service provisioning, including the state, action, and reward formulations and the training mechanisms, are also elaborated. Performance evaluations under both scenarios show that the proposed approach can effectively expedite the training processes of the target tasks and improve the ultimate service throughput. [ABSTRACT FROM AUTHOR]

When applying machine learning techniques to the Internet of things, aggregating massive amount of data seriously reduce the system efficiency. To tackle this challenge, a distributed learning framework called federated learning has been proposed. Due to the parallel training structure, the performance of federated learning suffers from the straggler effect. In this paper, to mitigate the straggler effect, we propose a novel learning scheme, edge-assisted federated learning (EAFL), which utilizes edge computing to reduce the computational burdens for stragglers in federated learning. It enables stragglers to offload partial computation to the edge server, and leverages the server's idle computing power to assist clients in model training. The offloading data size is optimized to minimize the learning delay of the system. Based on the optimized data size, a threshold-based offloading strategy for EAFL is proposed. Moreover, we extend EAFL to a dynamic scenario where clients may be offline after several update rounds. By grouping clients into different sets, we formulate the new EAFL delay optimization problem and derive the corresponding offloading strategy for the dynamic scenario. Simulation results are presented to show that EAFL has lower system delay than the original federated learning scheme. [ABSTRACT FROM AUTHOR]

The time and monetary costs of training sophisticated deep neural networks are exorbitant, which motivates resource-limited users to outsource the training process to the cloud. Concerning that an untrustworthy cloud service provider may inject backdoors to the returned model, the user can leverage state-of-the-art defense strategies to examine the model. In this paper, we aim to develop robust backdoor attacks (named RobNet) that can evade existing defense strategies from the standpoint of malicious cloud providers. The key rationale is to diversify the triggers and strengthen the model structure so that the backdoor is hard to be detected or removed. To attain this objective, we refine the trigger generation algorithm by selecting the neuron(s) with large weights and activations and then computing the triggers via gradient descent to maximize the value of the selected neuron(s). In stark contrast to existing works that fix the trigger location, we design a multi-location patching method to make the model less sensitive to mild displacement of triggers in real attacks. Furthermore, we extend the attack space by proposing multi-trigger backdoor attacks that can misclassify inputs with different triggers into the same or different target label(s). We evaluate the performance of RobNet on MNIST, GTSRB, and CIFAR-10 datasets, against four representative defense strategies Pruning, NeuralCleanse, Strip, and ABS. The comparison with two state-of-the-art baselines BadNets and Hidden Backdoors demonstrates that RobNet achieves higher attack success rate and is more resistant to potential defenses. [ABSTRACT FROM AUTHOR]

When running in Parameter Server (PS), the Distributed Stochastic Gradient Descent (D-SGD) incurs significant communication delays and huge communication overhead due to the model synchronization. Moreover, considering the heterogeneity of computational capability among workers, traditional synchronization modes incur under-utilization of computational resources because fast workers have to wait for slow ones finishing the computation. Although our previous work OSP can effectively solve these problems by overlapping the computation and communication procedures and allowing adaptive multiple local updates in distributed training, it causes the staleness problem brought by the overlap, yielding a performance degradation. In this paper, we propose a new method named LOSP by introducing local compensation to our previous synchronization mechanism, which mitigates adverse effects caused by the overlapping synchronization. We theoretically prove that LOSP (1) preserves the same convergence rate as the sequential SGD for non-convex problems, and (2) exhibits good scalability due to the linear speedup property with respect to both the number of workers and the average number of local updates. Evaluations show that LOSP significantly improves performance over the state-of-the-art ones in terms of both convergence accuracy and communication cost. [ABSTRACT FROM AUTHOR]

Federated learning (FL) is a promising technique that enables many edge devices to train a machine learning model collaboratively in wireless networks. By exploiting the superposition nature of wireless waveforms, over-the-air computation (AirComp) can accelerate model aggregation and hence facilitate communication-efficient FL. Due to channel fading, power control is crucial in AirComp. Prior works assume that the signals to be aggregated from each device, i.e., local gradients have identical statistics. In FL, however, gradient statistics vary over both training iterations and feature dimensions, and are unknown in advance. This paper studies the power control problem for over-the-air FL by taking gradient statistics into account. The goal is to minimize the aggregation error by optimizing the transmit power at each device subject to average power constraints. We obtain the optimal policy in closed form when gradient statistics are given. Notably, we show that the optimal transmit power is continuous and monotonically decreases with the squared multivariate coefficient of variation (SMCV) of gradient vectors. We then propose a method to estimate gradient statistics with negligible communication cost. Experimental results demonstrate that the proposed gradient-statistics-aware power control achieves higher test accuracy than the existing schemes for a wide range of scenarios. [ABSTRACT FROM AUTHOR]

Federated Learning (FL) is a distributed learning framework that can deal with the distributed issue in machine learning and still guarantee high learning performance. However, it is impractical that all users will sacrifice their resources to join the FL algorithm. This motivates us to study the incentive mechanism design for FL. In this paper, we consider a FL system that involves one base station (BS) and multiple mobile users. The mobile users use their own data to train the local machine learning model, and then send the trained models to the BS, which generates the initial model, collects local models and constructs the global model. Then, we formulate the incentive mechanism between the BS and mobile users as an auction game where the BS is an auctioneer and the mobile users are the sellers. In the proposed game, each mobile user submits its bids according to the minimal energy cost that the mobile users experiences in participating in FL. To decide winners in the auction and maximize social welfare, we propose the primal-dual greedy auction mechanism. The proposed mechanism can guarantee three economic properties, namely, truthfulness, individual rationality and efficiency. Finally, numerical results are shown to demonstrate the performance effectiveness of our proposed mechanism. [ABSTRACT FROM AUTHOR]

Performing deep neural network (DNN) inference in real time requires excessive network resources, which poses a big challenge to the resource-limited industrial Internet of things (IIoT) networks. To address the challenge, in this paper, we introduce an end-edge-cloud orchestration architecture, in which the inference task assignment and DNN model placement are flexibly coordinated. Specifically, the DNN models, trained and pre-stored in the cloud, are properly placed at the end and edge to perform DNN inference. To achieve efficient DNN inference, a multi-dimensional resource management problem is formulated to maximize the average inference accuracy while satisfying the strict delay requirements of inference tasks. Due to the mix-integer decision variables, it is difficult to solve the formulated problem directly. Thus, we transform the formulated problem into a Markov decision process which can be solved efficiently. Furthermore, a deep reinforcement learning based resource management scheme is proposed to make real-time optimal resource allocation decisions. Simulation results are provided to demonstrate that the proposed scheme can efficiently allocate the available spectrum, caching, and computing resources, and improve average inference accuracy by 31.4 % compared with the deep deterministic policy gradient benchmark. [ABSTRACT FROM AUTHOR]

Federated Learning (FL) has recently received a lot of attention for large-scale privacy-preserving machine learning. However, high communication overheads due to frequent gradient transmissions decelerate FL. To mitigate the communication overheads, two main techniques have been studied: (i) local update of weights characterizing the trade-off between communication and computation and (ii) gradient compression characterizing the trade-off between communication and precision. To the best of our knowledge, studying and balancing those two trade-offs jointly and dynamically while considering their impacts on convergence has remained unresolved even though it promises significantly faster FL. In this paper, we first formulate our problem to minimize learning error with respect to two variables: local update coefficients and sparsity budgets of gradient compression who characterize trade-offs between communication and computation/precision, respectively. We then derive an upper bound of the learning error in a given wall-clock time considering the interdependency between the two variables. Based on this theoretical analysis, we propose an enhanced FL scheme, namely Fast FL (FFL), that jointly and dynamically adjusts the two variables to minimize the learning error. We demonstrate that FFL consistently achieves higher accuracies faster than similar schemes existing in the literature. [ABSTRACT FROM AUTHOR]

This paper proposes a radially uniform (RU) design to sample representative datasets from a large volume of wind turbine data to build accurate data-driven models. The sampling capability and computational complexity are theoretically analyzed. It is shown that the RU design is representative of the original dataset and has computational complexity that is of the same order as sorting algorithms. Five algorithms, the neural networks (NN), multivariate adaptive regression splines (MARS), support vector machines (SVM), $k$ nearest neighbors (kNN), and linear regression (LR) are applied to model the wind turbine power output, drive-train vibratory acceleration, and tower vibratory acceleration based on the training dataset and sampled datasets. Extensive computational experiments are conducted to demonstrate advantages of the RU sampler over the random and maximin samplers. Results show that RU sampler outperforms the random sampler for building all five types of models and is more effective than the maximin sampler for building nonlinear models. [ABSTRACT FROM PUBLISHER]

Individual who have lost their lower limb because of amputation can use the prosthesis to restore daily living activities. The amputee intent recognition during locomotion modes can be used as source to control lower limb prosthesis. Due to continuous data recording from multiple sensors, the timely recognition of activities of daily living have become a challenging issue for traditional technology and conventional machine learning algorithms. This work hypothesize that parallel discriminant features can be learned from large amount of data generated by aggregating the neuromechanical signals from multiple subjects with parallel and distributed computing platform. Consequently, this paper apply three classifiers including support vector machine, decision tree and random forest on large data sets. The model performance is extensively evaluated in terms of different performance measurement parameters such as accuracy, efficiency, scalability and speedup in sequential and distributed environment. The experimental results show that the parallel approach achieved 3.9x computation speedup as compared to the sequential approach without affecting accuracy level. The parallel support vector machine algorithm demonstrated high speedup and scalability in comparison with random forest and decision tree algorithms. The outcome of this study could promote parallel based model for the unobtrusive recognition of lower limb locomotion modes and could promote the future design for the intelligent control of prostheses and exoskeleton. [ABSTRACT FROM AUTHOR]

In practice, deep neural networks have been found to be vulnerable to various types of noise, such as adversarial examples and corruption. Various adversarial defense methods have accordingly been developed to improve adversarial robustness for deep models. However, simply training on data mixed with adversarial examples, most of these models still fail to defend against the generalized types of noise. Motivated by the fact that hidden layers play a highly important role in maintaining a robust model, this paper proposes a simple yet powerful training algorithm, named Adversarial Noise Propagation (ANP), which injects noise into the hidden layers in a layer-wise manner. ANP can be implemented efficiently by exploiting the nature of the backward-forward training style. Through thorough investigations, we determine that different hidden layers make different contributions to model robustness and clean accuracy, while shallow layers are comparatively more critical than deep layers. Moreover, our framework can be easily combined with other adversarial training methods to further improve model robustness by exploiting the potential of hidden layers. Extensive experiments on MNIST, CIFAR-10, CIFAR-10-C, CIFAR-10-P, and ImageNet demonstrate that ANP enables the strong robustness for deep models against both adversarial and corrupted ones, and also significantly outperforms various adversarial defense methods. [ABSTRACT FROM AUTHOR]

In this paper, we propose a structure-coherent deep feature learning method for face alignment. Unlike most existing face alignment methods which overlook the facial structure cues, we explicitly exploit the relation among facial landmarks to make the detector robust to hard cases such as occlusion and large pose. Specifically, we leverage a landmark-graph relational network to enforce the structural relationships among landmarks. We consider the facial landmarks as structural graph nodes and carefully design the neighborhood to passing features among the most related nodes. Our method dynamically adapts the weights of node neighborhood to eliminate distracted information from noisy nodes, such as occluded landmark point. Moreover, different from most previous works which only tend to penalize the landmarks absolute position during the training, we propose a relative location loss to enhance the information of relative location of landmarks. This relative location supervision further regularizes the facial structure. Our approach considers the interactions among facial landmarks and can be easily implemented on top of any convolutional backbone to boost the performance. Extensive experiments on three popular benchmarks, including WFLW, COFW and 300W, demonstrate the effectiveness of the proposed method. In particular, due to explicit structure modeling, our approach is especially robust to challenging cases resulting in impressive low failure rate on COFW and WFLW datasets. The model and code are publicly available at https://github.com/BeierZhu/Sturcture-Coherency-Face-Alignment [ABSTRACT FROM AUTHOR]

Inspired by the perceived saturation of human visual system, this paper proposes a two-stream hybrid networks to simulate binocular vision for salient object detection (SOD). Each stream in our system consists of unsupervised and supervised methods to form a two-branch module, so as to model the interaction between human intuition and memory. The two-branch module parallel processes visual information with bottom-up and top-down SODs, and output two initial saliency maps. Then a polyharmonic neural network with random-weight (PNNRW) is utilized to fuse two-branch’s perception and refine the salient objects by learning online via multi-source cues. Depend on visual perceptual saturation, we can select optimal parameter of superpixel for unsupervised branch, locate sampling regions for PNNRW, and construct a positive feedback loop to facilitate perception saturated after the perception fusion. By comparing the binary outputs of the two-stream, the pixel annotation of predicted object with high saturation degree could be taken as new training samples. The presented method constitutes a semi-supervised learning framework actually. Supervised branches only need to be pre-trained initial, the system can collect the training samples with high confidence level and then train new models by itself. Extensive experiments show that the new framework can improve performance of the existing SOD methods, that exceeds the state-of-the-art methods in six popular benchmarks. [ABSTRACT FROM AUTHOR]

Human attention is an interactive activity between our visual system and our brain, using both low-level visual stimulus and high-level semantic information. Previous image salient object detection (SOD) studies conduct their saliency predictions via a multitask methodology in which pixelwise saliency regression and segmentation-like saliency refinement are conducted simultaneously. However, this multitask methodology has one critical limitation: the semantic information embedded in feature backbones might be degenerated during the training process. Our visual attention is determined mainly by semantic information, which is evidenced by our tendency to pay more attention to semantically salient regions even if these regions are not the most perceptually salient at first glance. This fact clearly contradicts the widely used multitask methodology mentioned above. To address this issue, this paper divides the SOD problem into two sequential steps. First, we devise a lightweight, weakly supervised deep network to coarsely locate the semantically salient regions. Next, as a postprocessing refinement, we selectively fuse multiple off-the-shelf deep models on the semantically salient regions identified by the previous step to formulate a pixelwise saliency map. Compared with the state-of-the-art (SOTA) models that focus on learning the pixelwise saliency in single images using only perceptual clues, our method aims at investigating the object-level semantic ranks between multiple images, of which the methodology is more consistent with the human attention mechanism. Our method is simple yet effective, and it is the first attempt to consider salient object detection as mainly an object-level semantic reranking problem. [ABSTRACT FROM AUTHOR]

Photorealistic style transfer is a challenging task, which demands the stylized image remains real. Existing methods are still suffering from unrealistic artifacts and heavy computational cost. In this paper, we propose a novel Style-Corpus Constrained Learning (SCCL) scheme to address these issues. The style-corpus with the style-specific and style-agnostic characteristics simultaneously is proposed to constrain the stylized image with the style consistency among different samples, which improves photorealism of stylization output. By using adversarial distillation learning strategy, a simple fast-to-execute network is trained to substitute previous complex feature transforms models, which reduces the computational cost significantly. Experiments demonstrate that our method produces rich-detailed photorealistic images, with 13 ~ 50 times faster than the state-of-the-art method (WCT2). [ABSTRACT FROM AUTHOR]

The existing fusion-based RGB-D salient object detection methods usually adopt the bistream structure to strike a balance in the fusion trade-off between RGB and depth (D). While the D quality usually varies among the scenes, the state-of-the-art bistream approaches are depth-quality-unaware, resulting in substantial difficulties in achieving complementary fusion status between RGB and D and leading to poor fusion results for low-quality D. Thus, this paper attempts to integrate a novel depth-quality-aware subnet into the classic bistream structure in order to assess the depth quality prior to conducting the selective RGB-D fusion. Compared to the SOTA bistream methods, the major advantage of our method is its ability to lessen the importance of the low-quality, no-contribution, or even negative-contribution D regions during RGB-D fusion, achieving a much improved complementary status between RGB and D. Our source code and data are available online at https://github.com/qdu1995/DQSD. [ABSTRACT FROM AUTHOR]

In visual tracking, how to effectively model the target appearance using limited prior information remains an open problem. In this paper, we leverage an ensemble of diverse models to learn manifold representations for robust object tracking. The proposed ensemble framework includes a shared backbone network for efficient feature extraction and multiple head networks for independent predictions. Trained by the shared data within an identical structure, the mutually correlated head models heavily hinder the potential of ensemble learning. To shrink the representational overlaps among multiple models while encouraging the diversity of individual predictions, we propose the model diversity and response diversity regularization terms during training. By fusing these distinctive prediction results via a fusion module, the tracking variance caused by the distractor objects can be largely restrained. Our whole framework is end-to-end trained in a data-driven manner, avoiding the heuristic designs of multiple base models and fusion strategies. The proposed method achieves state-of-the-art results on seven challenging benchmarks while operating in real-time. [ABSTRACT FROM AUTHOR]

While convolutional neural network (CNN) has achieved overwhelming success in various vision tasks, its heavy computational cost and storage overhead limit the practical use on mobile or embedded devices. Recently, compressing CNN models has attracted considerable attention, where pruning CNN filters, also known as the channel pruning, has generated great research popularity due to its high compression rate. In this paper, a new channel pruning framework is proposed, which can significantly reduce the computational complexity while maintaining sufficient model accuracy. Unlike most existing approaches that seek to-be-pruned filters layer by layer, we argue that choosing appropriate layers for pruning is more crucial, which can result in more complexity reduction but less performance drop. To this end, we utilize a long short-term memory (LSTM) to learn the hierarchical characteristics of a network and generate a global network pruning scheme. On top of it, we propose a data-dependent soft pruning method, dubbed Squeeze-Excitation-Pruning (SEP), which does not physically prune any filters but selectively excludes some kernels involved in calculating forward and backward propagations depending on the pruning scheme. Compared with the hard pruning, our soft pruning can better retain the capacity and knowledge of the baseline model. Experimental results demonstrate that our approach still achieves comparable accuracy even when reducing 70.1% Floating-point operation per second (FLOPs) for VGG and 47.5% for Resnet-56. [ABSTRACT FROM AUTHOR]

Internet of Vehicles (IoVs) is highly characterized by collaborative environment data sensing, computing and processing. Emerging big data and Artificial Intelligence (AI) technologies show significant advantages and efficiency for knowledge sharing among intelligent vehicles. However, it is challenging to guarantee the security and privacy of knowledge during the sharing process. Moreover, conventional AI-based algorithms cannot work properly in distributed vehicular networks. In this paper, a hierarchical blockchain framework and a hierarchical federated learning algorithm are proposed for knowledge sharing, by which vehicles learn environmental data through machine learning methods and share the learning knowledge with each others. The proposed hierarchical blockchain framework is feasible for the large scale vehicular networks. The hierarchical federated learning algorithm is designed to meet the distributed pattern and privacy requirement of IoVs. Knowledge sharing is then modeled as a trading market process to stimulate sharing behaviours, and the trading process is formulated as a multi-leader and multi-player game. Simulation results show that the proposed hierarchical algorithm can improve the sharing efficiency and learning quality. Furthermore, the blockchain-enabled framework is able to deal with certain malicious attacks effectively. [ABSTRACT FROM AUTHOR]

In this paper, we investigate the use of recurrent neural network (RNN)-based detection of magnetic recording channels with inter-symbol interference (ISI). We refer to the proposed detection method, which is intended for recording channels with partial-response equalization, as Partial-Response Neural Network (PR-NN). We train bi-directional gated recurrent units (bi-GRUs) to recover the ISI channel inputs from noisy channel output sequences and evaluate the network performance when applied to continuous, streaming data. The computational complexity of PR-NN during the evaluation process is comparable to that of a Viterbi detector. The recording system on which the experiments were conducted uses a rate-2/3, (1,7) runlength-limited (RLL) code with an E2PR4 partial-response channel target. Experimental results with ideal PR signals show that the performance of PR-NN detection approaches that of Viterbi detection in additive white gaussian noise (AWGN). Moreover, the PR-NN detector outperforms Viterbi detection and achieves the performance of Noise-Predictive Maximum Likelihood (NPML) detection in additive colored noise (ACN) at different channel densities. A PR-NN detector trained with both AWGN and ACN maintains the performance observed under separate training. Similarly, when trained with ACN corresponding to two different channel densities, PR-NN maintains its performance at both densities. Experiments confirm that this robustness is consistent over a wide range of signal-to-noise ratios (SNRs). Finally, PR-NN displays robust performance when applied to a more realistic magnetic recording channel with MMSE-equalized Lorentzian signals. [ABSTRACT FROM AUTHOR]

Quantization has been proven to be an effective method for reducing the computing and/or storage cost of DNNs. However, the trade-off between the quantization bitwidth and final accuracy is complex and non-convex, which makes it difficult to be optimized directly. Minimizing direct quantization loss (DQL) of the coefficient data is an effective local optimization method, but previous works often neglect the accurate control of the DQL, resulting in a higher loss of the final DNN model accuracy. In this paper, we propose a novel metric, called Vector Loss. Using this new metric, we decompose the minimization of the DQL to two independent optimization processes, which significantly outperform the traditional iterative L2 loss minimization process in terms of effectiveness, quantization loss as well as final DNN accuracy. We also develop a new DNN quantization solution called VecQ, which provides minimal direct quantization loss and achieve higher model accuracy. In order to speed up the proposed quantization process during model training, we accelerate the quantization process with a parameterized probability estimation method and template-based derivation calculation. We evaluate our proposed algorithm on MNIST, CIFAR, ImageNet, IMDB movie review and THUCNews text data sets with numerical DNN models. The results demonstrate that our proposed quantization solution is more accurate and effective than the state-of-the-art approaches yet with more flexible bitwidth support. Moreover, the evaluation of our quantized models on Salient Object Detection (SOD) tasks maintains comparable feature extraction quality with up to 16× weight size reduction. [ABSTRACT FROM AUTHOR]

SRAM cells usually require extremely low failure rate or equivalently extremely high production yield, making it impractical to perform yield analysis using Monte Carlo (MC) method as huge amount of samples are needed. Fast MC methods, e.g., importance sampling methods, are still too expensive as the anticipated failure rate is very low. In this paper, a new SRAM yield analysis method is proposed to tackle this issue. The key idea is to improve traditional importance sampling method with an efficient online surrogate model. Experimental results show that the proposed yield analysis method achieves $5\times $ – $22\times $ speedup over existing state-of-the-art techniques without sacrificing estimation accuracy. Sigma distribution and schmoo plot can be quickly generated by the proposed method, which is very useful for realistic applications. Based on the proposed yield analysis method, an efficient yield optimization method has been developed to further automate the SRAM cell design procedure where process variations can be fully considered. Experimental results show that a fully automatic yield optimization for SRAM cells can be done within only a few hours. [ABSTRACT FROM AUTHOR]

While microtask crowdsourcing provides a new way to solve large volumes of small tasks at a much lower price compared with traditional inhouse solutions, it suffers from quality problems due to the lack of incentives. On the other hand, providing incentives for microtask crowdsourcing is challenging since verifying the quality of submitted solutions is so expensive that it will negate the advantage of microtask crowdsourcing. We study cost-effective incentive mechanisms for microtask crowdsourcing in this paper. In particular, we consider a model with strategic workers, where the primary objective of a worker is to maximize his own utility. Based on this model, we first analyze two basic mechanisms and show their limitations in collecting high-quality solutions with low cost. Then, we propose a cost-effective mechanism that employs quality-aware worker training as a tool to stimulate workers to provide high-quality solutions. We prove theoretically that the proposed mechanism can be designed to obtain high-quality solutions from workers and ensure the budget constraint of the requester at the same time. Beyond its theoretical guarantees, we further demonstrate the effectiveness of our proposed mechanisms through a set of behavioral experiments. [ABSTRACT FROM PUBLISHER]

Real-world tasks in computer vision often touch upon open set recognition: multi-class recognition with incomplete knowledge of the world and many unknown inputs. Recent work on this problem has proposed a model incorporating an open space risk term to account for the space beyond the reasonable support of known classes. This paper extends the general idea of open space risk limiting classification to accommodate non-linear classifiers in a multi-class setting. We introduce a new open set recognition model called compact abating probability (CAP), where the probability of class membership decreases in value (abates) as points move from known data toward open space. We show that CAP models improve open set recognition for multiple algorithms. Leveraging the CAP formulation, we go on to describe the novel Weibull-calibrated SVM (W-SVM) algorithm, which combines the useful properties of statistical extreme value theory for score calibration with one-class and binary support vector machines. Our experiments show that the W-SVM is significantly better for open set object detection and OCR problems when compared to the state-of-the-art for the same tasks. [ABSTRACT FROM AUTHOR]

The deep computation model has been proved to be effective for big data hierarchical feature and representation learning in the tensor space. However, it requires expensively computational resources including high-performance computing units and large memory to train a deep computation model with a large number of parameters, limiting its effectiveness and efficiency for industry informatics big data feature learning. In this paper, a tensor-train deep computation model is presented for industry informatics big data feature learning. Specially, the tensor-train network is used to compress the parameters significantly by converting the dense weight tensors into the tensor-train format. Furthermore, a learning algorithm is implemented based on gradient descent and back-propagation to train the parameters of the presented tensor-train deep computation model. Extensive experiments are carried on STL-10, CUAVE, and SNAE2 to evaluate the presented model in terms of the approximation error, classification accuracy drop, parameters reduction, and speedup. Results demonstrate that the presented model can improve the training efficiency and save the memory space greatly for the deep computation model with small accuracy drops, proving its potential for industry informatics big data feature learning. [ABSTRACT FROM AUTHOR]

This paper presents a method of nonintrusive load monitoring (NILM) for electrical systems consisting of a fixed and known set of devices. This constraint is inherently met by embedded and mobile power systems, and is also commonly satisfied in industrial settings. The proposed NILM method provides time-accurate profiles of device behavior using only probabilistic device models and system-level measurements. The full method consists of model training, construction of a system-level model, and prediction of device-level energy use. Energy use estimations are determined by maximizing the probability of the predicted behavior given the system-level measurements, and are calculated online at each sampling instant. The method is validated using test data from public databases, and its performance is assessed using standard NILM accuracy metrics. The intended application of the proposed method is to support system status assessments and to provide early indications of potential equipment damage through identification of atypical device behavior. [ABSTRACT FROM AUTHOR]