Search

Multi-Task Learning (MTL) is a learning paradigm in machine learning and its aim is to leverage useful information contained in multiple related tasks to help improve the generalization performance of all the tasks. In this paper, we give a survey for MTL from the perspective of algorithmic modeling, applications and theoretical analyses. For algorithmic modeling, we give a definition of MTL and then classify different MTL algorithms into five categories, including feature learning approach, low-rank approach, task clustering approach, task relation learning approach and decomposition approach as well as discussing the characteristics of each approach. In order to improve the performance of learning tasks further, MTL can be combined with other learning paradigms including semi-supervised learning, active learning, unsupervised learning, reinforcement learning, multi-view learning and graphical models. When the number of tasks is large or the data dimensionality is high, we review online, parallel and distributed MTL models as well as dimensionality reduction and feature hashing to reveal their computational and storage advantages. Many real-world applications use MTL to boost their performance and we review representative works in this paper. Finally, we present theoretical analyses and discuss several future directions for MTL. [ABSTRACT FROM AUTHOR]

Federated learning (FL) is an emerging privacy-preserving paradigm that enables multiple participants collaboratively to train a global model without uploading raw data. Considering heterogeneous computing and communication capabilities of different participants, asynchronous FL can avoid the stragglers effect in synchronous FL and adapts to scenarios with vast participants. Both staleness and non-IID data in asynchronous FL would reduce the model utility. However, there exists an inherent contradiction between the solutions to the two problems. That is, mitigating the staleness requires to select less but consistent gradients while coping with non-IID data demands more comprehensive gradients. To address the dilemma, this paper proposes a two-stage weighted $K$ K asynchronous FL with adaptive learning rate (WKAFL). By selecting consistent gradients and adjusting learning rate adaptively, WKAFL utilizes stale gradients and mitigates the impact of non-IID data, which can achieve multifaceted enhancement in training speed, prediction accuracy and training stability. We also present the convergence analysis for WKAFL under the assumption of unbounded staleness to understand the impact of staleness and non-IID data. Experiments implemented on both benchmark and synthetic FL datasets show that WKAFL has better overall performance compared to existing algorithms. [ABSTRACT FROM AUTHOR]

Clinically oriented deep learning algorithms, combined with large-scale medical datasets, have significantly promoted computer-aided diagnosis. To address increasing ethical and privacy issues, Federated Learning (FL) adopts a distributed paradigm to collaboratively train models, rather than collecting samples from multiple institutions for centralized training. Despite intensive research on FL, two major challenges are still existing when applying FL in the real-world medical scenarios, including the performance degradation (i.e., retrogress) after each communication and the intractable class imbalance. Thus, in this paper, we propose a novel personalized FL framework to tackle these two problems. For the retrogress problem, we first devise a Progressive Fourier Aggregation (PFA) at the server side to gradually integrate parameters of client models in the frequency domain. Then, at the client side, we design a Deputy-Enhanced Transfer (DET) to smoothly transfer global knowledge to the personalized local model. For the class imbalance problem, we propose the Conjoint Prototype-Aligned (CPA) loss to facilitate the balanced optimization of the FL framework. Considering the inaccessibility of private local data to other participants in FL, the CPA loss calculates the global conjoint objective based on global imbalance, and then adjusts the client-side local training through the prototype-aligned refinement to eliminate the imbalance gap with such a balanced goal. Extensive experiments are performed on real-world dermoscopic and prostate MRI FL datasets. The experimental results demonstrate the advantages of our FL framework in real-world medical scenarios, by outperforming state-of-the-art FL methods with a large margin. The source code is available at https://github.com/CityU-AIM-Group/PRR-Imbalance https://github.com/CityU-AIM-Group/PRR-Imbalance. [ABSTRACT FROM AUTHOR]

Accurate solar photovoltaic (PV) generation forecast is critical to the reliable and economic operation of a modern power system. In practice, due to various faulty issues in the sensor, communication, or database system, the historical and online measurement data may not be always complete, and the missing data could dramatically degrade the forecasting model's accuracy. To solve this problem, this paper proposes an integrated missing-data tolerant model for probabilistic PV power generation forecasting. Taking historical PV generations as input, this model is based on a recursive long short-term memory network (Rec-LSTM), which can provide multi-step ahead forecasting of the probability distribution of PV generation. The unobserved input data will be imputed recursively based on the model output at the previous time step. During the training process, the imputations and forecasting values are iteratively updated by the negative log-likelihood loss function. As a salient advantage, this method can deal with data missing scenarios at both offline and online stages. Numerical experiments are conducted on two one-year datasets from Australia and Singapore, respectively. Probabilistic forecasting for both large-scale and small-scale building-level PV power generation is tested at the time resolution of 15 mins. Testing results show the proposed method can achieve superior probabilistic prediction accuracy as well as strong robustness under various data missing scenarios, compared to other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Intrinsic image decomposition is the task of mapping image to albedo and shading. Classical approaches derive methods from spatial models. The modern literature stresses evaluation, by comparing predictions to human judgements (“lighter”, “same as”, “darker”). The best modern intrinsic image methods train a map from image to albedo using images rendered from computer graphics models and example human judgements. This approach yields practical methods, but obtaining rendered images can be inconvenient. Furthermore, the approach cannot explain how a one could learn to recover intrinsic images without geometric, surface and illumination models, as people and animals appear to do. This paper describes a method that learns intrinsic image decomposition without seeing human annotations, rendered data, or ground truth data. Instead, the method relies on paradigms – spatial models of albedo and of shading. Rather than finding the “best” albedo and shading for an image via optimization, our approach trains a neural network on synthetic images. The synthetic images are constructed by multiplying albedos and shading fields sampled from our models. The network is subject to a novel smoothing procedure that ensures good behavior at short scales on real images. An averaging procedure ensures that reported albedo and shading are largely equivariant – different crops and scalings of an image will report the same albedo and shading at shared points. This averaging procedure controls long scale error. The standard evaluation for an intrinsic image method is a WHDR score. Our method achieves WHDR scores competitive with those of strong recent methods allowed to see training WHDR annotations, rendered data, and ground truth data. Our method produces albedo and shading maps with attractive qualitative properties – for example, albedo fields do not suppress wood grain and represent narrow grooves in surfaces well. Because our method is unsupervised, we can compute estimates of the test/train variance of WHDR scores; these are quite large, and suggest is unsafe to rely small differences in reported WHDR. [ABSTRACT FROM AUTHOR]

Pretrained deep models hold their learnt knowledge in the form of model parameters. These parameters act as “memory” for the trained models and help them generalize well on unseen data. However, in absence of training data, the utility of a trained model is merely limited to either inference or better initialization towards a target task. In this paper, we go further and extract synthetic data by leveraging the learnt model parameters. We dub them Data Impressions, which act as proxy to the training data and can be used to realize a variety of tasks. These are useful in scenarios where only the pretrained models are available and the training data is not shared (e.g., due to privacy or sensitivity concerns). We show the applicability of data impressions in solving several computer vision tasks such as unsupervised domain adaptation, continual learning as well as knowledge distillation. We also study the adversarial robustness of lightweight models trained via knowledge distillation using these data impressions. Further, we demonstrate the efficacy of data impressions in generating data-free Universal Adversarial Perturbations (UAPs) with better fooling rates. Extensive experiments performed on benchmark datasets demonstrate competitive performance achieved using data impressions in absence of original training data. [ABSTRACT FROM AUTHOR]

Traditional Deep Neural Network (DNN) security is mostly related to the well-known adversarial input example attack. Recently, another dimension of adversarial attack, namely, attack on DNN weight parameters, has been shown to be very powerful. As a representative one, the Bit-Flip-based adversarial weight Attack (BFA) injects an extremely small amount of faults into weight parameters to hijack the executing DNN function. Prior works of BFA focus on un-targeted attack that can hack all inputs into a random output class by flipping a very small number of weight bits stored in computer memory. This paper proposes the first work of targeted BFA based (T-BFA) adversarial weight attack on DNNs, which can intentionally mislead selected inputs to a target output class. The objective is achieved by identifying the weight bits that are highly associated with classification of a targeted output through a class-dependent vulnerable weight bit searching algorithm. Our proposed T-BFA performance is successfully demonstrated on multiple DNN architectures for image classification tasks. For example, by merely flipping 27 out of 88 million weight bits of ResNet-18, our T-BFA can misclassify all the images from ’Hen’ class into ’Goose’ class (i.e., 100% attack success rate) in ImageNet dataset, while maintaining 59.35% validation accuracy. Moreover, we successfully demonstrate our T-BFA attack in a real computer prototype system running DNN computation, with Ivy Bridge-based Intel i7 CPU and 8GB DDR3 memory. [ABSTRACT FROM AUTHOR]

As deep learning models are usually massive and complex, distributed learning is essential for increasing training efficiency. Moreover, in many real-world application scenarios like healthcare, distributed learning can also keep the data local and protect privacy. Recently, the asynchronous decentralized parallel stochastic gradient descent (ADPSGD) algorithm has been proposed and demonstrated to be an efficient and practical strategy where there is no central server, so that each computing node only communicates with its neighbors. Although no raw data will be transmitted across different local nodes, there is still a risk of information leak during the communication process for malicious participants to make attacks. In this paper, we present a differentially private version of asynchronous decentralized parallel SGD framework, or A(DP) $^2$ 2 SGD for short, which maintains communication efficiency of ADPSGD and prevents the inference from malicious participants. Specifically, Rényi differential privacy is used to provide tighter privacy analysis for our composite Gaussian mechanisms while the convergence rate is consistent with the non-private version. Theoretical analysis shows A(DP) $^2$ 2 SGD also converges at the optimal $\mathcal {O}(1/\sqrt{T})$ O (1 / T) rate as SGD. Empirically, A(DP) $^2$ 2 SGD achieves comparable model accuracy as the differentially private version of Synchronous SGD (SSGD) but runs much faster than SSGD in heterogeneous computing environments. [ABSTRACT FROM AUTHOR]

The accuracy of artificial intelligence (AI) models is crucial for connected and autonomous vehicles (CAVs). However, in reality, model training under less frequent weather faces the problem of insufficient sampling. Also, in the real world, weather, sunlight, etc., can only change with the speed of the real-time clock, so the traditional sampling process is very slow. Moreover, currently, collective learning, which can make up the limited experience and computing power of a single vehicle, is always introduced to cases where the data from participants have the same structure, wasting massive heterogeneous data from vehicles of different brands. Therefore, in this paper, we propose a quantum collective learning and many-to-many matching game-based scheme in the metaverse for CAVs. The environment is simulated in the metaverse, which has its own time clock system, thereby expanding sample size and speeding up the sampling process. And we quantify the quality of intelligence in collective learning from the perspective of feature diversity. It is the cornerstone of collective learning between heterogeneous vehicles, facilitating maximum utilization of data with different structures. Then, we formulate the distributed vehicles selection problem as a many-to-many matching game and use Gale–Shapely algorithm to solve it. Also, we formulate the spectrum resource allocation problem as a discrete Markov decision process (MDP) and adopt a quantum-inspired reinforcement learning (QRL) algorithm to find the optimal policy to achieve the high revenue of the system. In simulations, the performance of the proposed scheme is compared with existing methods. [ABSTRACT FROM AUTHOR]

With the integration of Artificial Intelligence (AI) and Internet of Things (IoT), the Federated Edge Learning (FEL), a promising computing framework is developing. However, there are still unsolved issues on communication efficiency and data security due to the huge models and unreliable transmission links. To address these issues, this paper proposes a novel federated edge learning system, called LightFed, where the edge nodes upload only vital partial local models, and successfully achieve lightweight communication and model aggregation. First, a novel model aggregation method Model Splitting and Splicing (MSS) and a Selective Parameter Transmission (SPT) scheme are proposed. By detecting the updating gradients of local parameters and filtering significant parameters, selective rotated transmission and efficient aggregation of local models are achieved. Second, a Training Filling Model (TFM) is proposed to infer the total data distribution of edge nodes, and train a filling model to mitigate the unbalanced training data without violating the data privacy of individual users. Moreover, a blockchain-powered confusion transmission mechanism is proposed for defending the attacks from external adversaries and protecting the model information. Finally, extensive experimental results demonstrate that our LightFed significantly outperforms the existing FEL systems in terms of communication efficiency and privacy security. [ABSTRACT FROM AUTHOR]

As an emulous alternative to deep neural networks, Deep Forest emerges with features like low complexity, fewer hyper-parameters, and good robustness, which are predominantly desired in distributed computing applications and ecosystems. Recently, an efficient distributed Deep Forest system, named ForestLayer, was proposed, designing a fine-grained sub-Forest-based task-parallel algorithm to improve the parallel computing efficiency of Deep Forest. However, the sub-Forest splitting of ForestLayer is static and one-off without adaptability to the computing environment, nevertheless, the size of splitting granularity has a significant impact on the system performance. To further improve the computing efficiency and scalability of the distributed Deep Forest, in this paper, we propose a novel distributed Deep Forest algorithm, named BLB-gcForest (Bag of Little Bootstraps-gcForest), which augments the gcForest (multi-Grained Cascade Forest) approach for constructing Deep Forest. BLB-gcForest carries out parallel computation for each tree in sub-Forests at a finer parallel granularity and integrates with the Bag of Little Bootstraps (BLB) mechanism to reduce massive transmitted feature instances for Cascade Forest Layers, utterly improving both computation efficiency and communication efficiency. Moreover, to solve the problem of the forest splitting granularity, we further design an adaptive sub-Forest splitting algorithm to ensure the maximum resource utilization for parallel computation of each sub-Forest. Experimental results on four well-known large-scale datasets, namely YEAST, LETTER, MNIST, CIFAR10, show that the training efficiency of BLB-gcForest achieves up to 20.3x and 1.64x speedups compared with the state-of-the-art gcForest and ForestLayer, respectively while guaranteeing higher accuracy and better robustness [ABSTRACT FROM AUTHOR]

Time series classification models have been garnering significant importance in the research community. However, not much research has been done on generating adversarial samples for these models. These adversarial samples can become a security concern. In this paper, we propose utilizing an adversarial transformation network (ATN) on a distilled model to attack various time series classification models. The proposed attack on the classification model utilizes a distilled model as a surrogate that mimics the behavior of the attacked classical time series classification models. Our proposed methodology is applied onto 1-nearest neighbor dynamic time warping (1-NN DTW) and a fully convolutional network (FCN), all of which are trained on 42 University of California Riverside (UCR) datasets. In this paper, we show both models were susceptible to attacks on all 42 datasets. When compared to Fast Gradient Sign Method, the proposed attack generates a larger faction of successful adversarial black-box attacks. A simple defense mechanism is successfully devised to reduce the fraction of successful adversarial samples. Finally, we recommend future researchers that develop time series classification models to incorporating adversarial data samples into their training data sets to improve resilience on adversarial samples. [ABSTRACT FROM AUTHOR]

With substantial amount of time, resources and human (team) efforts invested to explore and develop successful deep neural networks (DNN), there emerges an urgent need to protect these inventions from being illegally copied, redistributed, or abused without respecting the intellectual properties of legitimate owners. Following recent progresses along this line, we investigate a number of watermark-based DNN ownership verification methods in the face of ambiguity attacks, which aim to cast doubts on the ownership verification by forging counterfeit watermarks. It is shown that ambiguity attacks pose serious threats to existing DNN watermarking methods. As remedies to the above-mentioned loophole, this paper proposes novel passport-based DNN ownership verification schemes which are both robust to network modifications and resilient to ambiguity attacks. The gist of embedding digital passports is to design and train DNN models in a way such that, the DNN inference performance of an original task will be significantly deteriorated due to forged passports. In other words, genuine passports are not only verified by looking for the predefined signatures, but also reasserted by the unyielding DNN model inference performances. Extensive experimental results justify the effectiveness of the proposed passport-based DNN ownership verification schemes. Code is available at https://github.com/kamwoh/DeepIPR. [ABSTRACT FROM AUTHOR]

In order to retain more feature information of local areas on a point cloud, local grouping and subsampling are the necessary data structuring steps in most hierarchical deep learning models. Due to the disorder nature of the points in a point cloud, the significant time cost may be consumed when grouping and subsampling the points, which consequently results in poor scalability. This paper proposes a fast data structuring method called PSNet (Point Structuring Net). PSNet transforms the spatial features of the points and matches them to the features of local areas in a point cloud. PSNet achieves grouping and sampling at the same time while the existing methods process sampling and grouping in two separate steps (such as using FPS plus kNN). PSNet performs feature transformation pointwise while the existing methods uses the spatial relationship among the points as the reference for grouping. Thanks to these features, PSNet has two important advantages: 1) the grouping and sampling results obtained by PSNet is stable and permutation invariant; and 2) PSNet can be easily parallelized. PSNet can replace the data structuring methods in the mainstream point cloud deep learning models in a plug-and-play manner. We have conducted extensive experiments. The results show that PSNet can improve the training and inference speed significantly while maintaining the model accuracy. [ABSTRACT FROM AUTHOR]

Wide models such as generalized linear models and factorization-based models have been extensively used in various predictive applications, e.g., recommendation, CTR prediction, and image recognition. Due to the memory bounded property of the models, the performance improvement on CPU is reaching the limitation. GPU is known to have many computation units and high memory bandwidth, and becomes a promising platform for training machine learning models. However, the GPU training for the wide models is far from optimal due to the sparsity and irregularity in wide models. The existing GPU-based wide models are even slower than the ones using CPU. The classical training schema of the wide models does not optimized for the GPU architecture, which suffers from large amount of random memory accesses and redundant read/write of intermediate values. In this paper, we propose an efficient GPU-training framework for the large-scale wide models, named cuWide. To fully benefit from the memory hierarchy of GPU, cuWide applies a new flow-based schema for training, which leverages the spatial and temporal locality of wide models to drastically reduce the amount of communication with GPU global memory. To do so, we adopt a bigraph computation model to efficiently realize the flow-based schema and exploit three flexible interfaces for programming. Further, we use the 2D partition of mini-batch (in sample and feature dimensions) with proposed graph abstraction to optimize GPU memory access for sparse data, and apply several spatial-temporal caching mechanisms (importance-based model caching and cross-stage accumulation caching mechanisms) to achieve a high performance kernel. To efficiently implement cuWide, we also propose several GPU-oriented optimizations, including feature-oriented data layout to enhance the data locality, replication mechanism to reduce update conflicts in shared memory, and multi-stream scheduling to overlap data transferring and kernel computing. We show that cuWide can be up to more than 20× faster than the state-of-the-art GPU solutions and multi-core CPU solutions. [ABSTRACT FROM AUTHOR]

The paper presents several novel residual recurrent neural network (RNN) models for digital predistortion of radio-frequency power amplifiers. Then, unstructured and structured neural network (NN) pruning algorithms are proposed to reduce the per-sample computational complexity and the required memory, without compromising their modeling performance. The mathematical equations of the complexity and the memory are derived to analyze the hardware cost and design low cost models. An experimental platform is developed to measure the modeling performance in terms of adjacent channel leakage ratio (ACLR), normalized mean square error, error vector magnitude (EVM) and bit error rate. The platform uses an envelope tracking PA excited by 80 MHz IEEE 802.11ac signals. Compared with the state-of-the-art RNN model in (Sun et al., 2019), the proposed models reduce the per-sample complexity and required memory by $94\hbox{--}99$ %, they also improve the ACLR and EVM by at least 3 dB and 1 dB at 0.45 dB and 3.15 dB gain compression, respectively. Furthermore, an observation path with time alignment is used to remove the transmit latency in the state-of-the-art approach. At 3.15 dB gain compression, the proposed models outperform the state-of-the-art feedforward and convolutional NN models by at least 1.3 dB in EVM, outperform the general memory polynomial model by at least 3.3 dB in EVM. The above evidences indicate that the proposed RNN models can be implemented with relatively low hardware cost and achieve the best EVM in high power efficiency scenario. [ABSTRACT FROM AUTHOR]

Domain adaptation is to transfer the shared knowledge learned from the source domain to a new environment, i.e., target domain. One common practice is to train the model on both labeled source-domain data and unlabeled target-domain data. Yet the learned models are usually biased due to the strong supervision of the source domain. Most researchers adopt the early-stopping strategy to prevent over-fitting, but when to stop training remains a challenging problem since the lack of the target-domain validation set. In this paper, we propose one efficient bootstrapping method, called Adaboost Student, explicitly learning complementary models during training and liberating users from empirical early stopping. Adaboost Student combines deep model learning with the conventional training strategy, i.e., adaptive boosting, and enables interactions between learned models and the data sampler. We adopt one adaptive data sampler to progressively facilitate learning on hard samples and aggregate “weak” models to prevent over-fitting. Extensive experiments show that (1) Without the need to worry about the stopping time, AdaBoost Student provides one robust solution by efficient complementary model learning during training. (2) AdaBoost Student is orthogonal to most domain adaptation methods, which can be combined with existing approaches to further improve the state-of-the-art performance. We have achieved competitive results on three widely-used scene segmentation domain adaptation benchmarks. [ABSTRACT FROM AUTHOR]

An emerging computational paradigm, named federated edge learning (FEL), enables intelligent computing at the network edge with the feature of preserving data privacy for edge devices. Given their constrained resources, it becomes a great challenge to achieve high execution performance for FEL. Most of the state-of-the-arts concentrate on enhancing FEL from the perspective of system operation procedures, taking few precautions during the composition step of the FEL system. Though a few recent studies recognize the importance of FEL formation and propose server-centric device selection schemes, the impact of data sizes is largely overlooked. In this paper, we take advantage of game theory to depict the decision dilemma among edge devices regarding whether to participate in FEL or not given their heterogeneous sizes of local datasets. For realizing both the individual and global optimization, the server is employed to solve the participation dilemma, which requires accurate information collection for devices’ local datasets. Hence, we utilize mechanism design to enable truthful information solicitation. With the help of correlated equilibrium, we derive a decision making strategy for devices from the global perspective, which can achieve the long-term stability and efficacy of FEL. For scalability consideration, we optimize the computational complexity of the basic solution to the polynomial level. Lastly, extensive experiments based on both real and synthetic data are conducted to evaluate our proposed mechanisms, with experimental results demonstrating the performance advantages. [ABSTRACT FROM AUTHOR]

Artificial intelligence (AI)-powered autonomous vehicles (AVs) can integrate different machine learning (ML) techniques to build up a complex autonomous driving system. However, single AV intelligence is not enough to cope with ever-changing driving environments. The underlying reason is that, with current neural network design and training algorithms, it is challenging for the driving model to generalize to diverse driving environments all at once due to sample inefficiency and the curse of dimensionality. Powerful computing resources and massive amount of data can be used to train a good driving model offline. However, the driving model obtained offline might fail in corner case scenarios. In this paper, we propose an intelligence networking framework among AVs assisted by multi-access edge computing (MEC) with end-to-end learning for demonstration. In this framework, driving road is divided into segments and data is collected for each road segment separately. Assisted by MEC networks, a continuously updated driving model is produced in near real-time for each road segment when the environment changes. By dividing the road into segments, we aim to reduce the burden of generalization since a single model only needs to adapt to a specific road segment. Simulation results show that our solutions can produce updated driving model for each road segment to adapt to environmental changes better than the existing scheme. Upcoming AVs can then adapt to changing environments by downloading the updated driving model. [ABSTRACT FROM AUTHOR]

We give an overview of recent developments in the modeling of radiowave propagation, based on machine learning (ML) algorithms. We identify the input and output specification and the architecture of the model as the main challenges associated with ML-driven propagation models. Relevant papers are discussed and categorized based on their approach to each of these challenges. Emphasis is given on presenting the prospects and open problems in this promising and rapidly evolving area. [ABSTRACT FROM AUTHOR]

In an Internet of Vehicle (IoV) system, federated learning (FL) is a new approach to process real-time vehicle data in a distributed way, which can improve the driving experience and service quality. However, due to the high mobility and uncertainty of vehicles, the existing federated learning protocols are difficult to meet the full requirements from an IoV system, such as efficient resource allocation, high precision learning, and fast convergence of learning algorithm. To solve the above problems, in this paper, we propose a semi-synchronous federated learning (Semi-SynFed) protocol, to improve the performance of machine learning at Internet of Vehicles. In Semi-SynFed, we first select appropriate nodes to participate the aggregation by its computing capacity, network capacity and learning value of training samples. Meanwhile, the dynamic waiting time technique is designed to adjust the server waiting time at each round dynamically, which makes federated learning process work much more efficient. Finally, a dynamic aggregation scheme is designed to aggregate model parameters in an asynchronous way. To evaluate the reliability of Semi-SynFed, we establish a simulation of IoVs based on the real world data. In the comparative experiments, the results show that the Semi-SynFed outperforms the existing federated learning protocols in terms of convergence speed and resource consumption. [ABSTRACT FROM AUTHOR]

Geographical topic models have been used to mine geo-tagged documents for topical region and geographical topics, and also have applications in recommendations, user mobility modeling, event detection, etc. Existing studies focus on learning effective geographical topic models while ignoring the efficiency issue. However, it is very expensive to train geographical topic models — it may take days to train a geographical topic model of a small scale on a collection of documents with millions of word tokens. In this paper, we propose the first distributed solution, called ${\sf PGeoTopic}$ PGeoTopic , for training geographical topic models. The proposed solution comprises several novel technical components to increase parallelism, reduce memory requirement, and reduce communication cost. Experiments show that our approach for mining geographical topic models is scalable with both model size and data size on distributed systems. [ABSTRACT FROM AUTHOR]

Decentralized sparsity learning has attracted a significant amount of attention recently due to its rapidly growing applications. To obtain the robust and sparse estimators, a natural idea is to adopt the non-smooth median loss combined with a $\ell _1$ sparsity regularizer. However, most of the existing methods suffer from slow convergence performance caused by the double non-smooth objective. To accelerate the computation, in this paper, we proposed a decentralized surrogate median regression (deSMR) method for efficiently solving the decentralized sparsity learning problem. We show that our proposed algorithm enjoys a linear convergence rate with a simple implementation. We also investigate the statistical guarantee, and it shows that our proposed estimator achieves a near-oracle convergence rate without any restriction on the number of network nodes. Moreover, we establish the theoretical results for sparse support recovery. Thorough numerical experiments and real data study are provided to demonstrate the effectiveness of our method. [ABSTRACT FROM AUTHOR]

Radar radio source (RRS) recognition plays an important role in the fields of military electronic support systems (ESM) and civilian autonomous driving. The rapid development of machine learning technology, especially deep learning, has effectively and efficiently improved RRS intelligent recognition performances when operating in the increasingly complex electromagnetic environment. However, the data sampling limitation and computation cost are still severe challenges in real RRS recognition scenarios. In this paper, we propose a novel network based on meta-transfer learning, called RRSARNet, to achieve effective adaptive RRS recognition in the context of low signal-to-noise ratio (SNR). First, by using the short-time Fourier transform, a six-type small samples RRS simulation dataset with different SNR levels is constructed. Then, a novel RRSARNet, based on metric learning, is proposed, which consists of a four-layer embedding module and a four-layer relational module. Finally, the RRS dataset is divided into training, supporting and testing subsets, which are used to train and test the RRSARNet in a meta-transfer learning method. Experiments on the RRS dataset show that the proposed RRSARNet can achieve an overall accuracy (OA) above 96% and 99% when the SNR is above −15 dB and −10 dB, respectively. Even when the SNR is -30 dB, OA can reach more than 70%. For 5-way 1-shot and 5-way 5-shot experiments, the inference time of an image is about 0.043 and 0.140 milliseconds, respectively. Besides, experiments on the RRS simulation dataset and the two benchmark datasets, the RRSARNet performs better or more competitive than many existing state-of-the-art technologies in terms of recognition accuracy. [ABSTRACT FROM AUTHOR]

Recent workat the U.S. Army CCDC Ground Vehicle Systems Center has shown that machine learning classifiers can quickly construct high-fidelity mobility maps. Training these classifiers, on the other hand, is still a challenge, since each data instance is labeled by performing a computationally intensive, physics-based simulation. In this paper we introduce an active learning framework, based on the query-by-bagging algorithm, that substantially reduces the number of simulations needed to train a classifier. Experimental results suggest that our sampling algorithm can train a neural network, with higher accuracy, using less than half the number of simulations when compared to random sampling. [ABSTRACT FROM AUTHOR]

Automated machine learning (AutoML) has achieved remarkable progress on various tasks, which is attributed to its minimal involvement of manual feature and model designs. However, most of existing AutoML pipelines only touch parts of the full machine learning pipeline, e.g., neural architecture search or optimizer selection. This leaves potentially important components such as data cleaning and model ensemble out of the optimization, and still results in considerable human involvement and suboptimal performance. The main challenges lie in the huge search space assembling all possibilities over all components, as well as the generalization ability over different tasks like image, text, and tabular etc. In this paper, we present a first-of-its-kind fully AutoML pipeline, to comprehensively automate data preprocessing, feature engineering, model generation/selection/training and ensemble for an arbitrary dataset and evaluation metric. Our innovation lies in the comprehensive scope of a learning pipeline, with a novel “life-long” knowledge anchor design to fundamentally accelerate the search over the full search space. Such knowledge anchors record detailed information of pipelines and integrates them with an evolutionary algorithm for joint optimization across components. Experiments demonstrate that the result pipeline achieves state-of-the-art performance on multiple datasets and modalities. Specifically, the proposed framework was extensively evaluated in the NeurIPS 2019 AutoDL challenge, and won the only champion with a significant gap against other approaches, on all the image, video, speech, text and tabular tracks. [ABSTRACT FROM AUTHOR]

When applying machine learning techniques to the Internet of things, aggregating massive amount of data seriously reduce the system efficiency. To tackle this challenge, a distributed learning framework called federated learning has been proposed. Due to the parallel training structure, the performance of federated learning suffers from the straggler effect. In this paper, to mitigate the straggler effect, we propose a novel learning scheme, edge-assisted federated learning (EAFL), which utilizes edge computing to reduce the computational burdens for stragglers in federated learning. It enables stragglers to offload partial computation to the edge server, and leverages the server's idle computing power to assist clients in model training. The offloading data size is optimized to minimize the learning delay of the system. Based on the optimized data size, a threshold-based offloading strategy for EAFL is proposed. Moreover, we extend EAFL to a dynamic scenario where clients may be offline after several update rounds. By grouping clients into different sets, we formulate the new EAFL delay optimization problem and derive the corresponding offloading strategy for the dynamic scenario. Simulation results are presented to show that EAFL has lower system delay than the original federated learning scheme. [ABSTRACT FROM AUTHOR]