Search

The article reports that With the continued growth of IoT devices and their deployment, manually managing and connecting them is impractical and presents multiple challenges. To that end, Zero Touch Networks that rely on software-based modules instead of dedicated propriety hardware become a viable potential solution. The overall aim of zero-touch networks is for machines to learn how to become more autonomous so that we can delegate complex, mundane tasks to them.

Multi-Task Learning (MTL) is a learning paradigm in machine learning and its aim is to leverage useful information contained in multiple related tasks to help improve the generalization performance of all the tasks. In this paper, we give a survey for MTL from the perspective of algorithmic modeling, applications and theoretical analyses. For algorithmic modeling, we give a definition of MTL and then classify different MTL algorithms into five categories, including feature learning approach, low-rank approach, task clustering approach, task relation learning approach and decomposition approach as well as discussing the characteristics of each approach. In order to improve the performance of learning tasks further, MTL can be combined with other learning paradigms including semi-supervised learning, active learning, unsupervised learning, reinforcement learning, multi-view learning and graphical models. When the number of tasks is large or the data dimensionality is high, we review online, parallel and distributed MTL models as well as dimensionality reduction and feature hashing to reveal their computational and storage advantages. Many real-world applications use MTL to boost their performance and we review representative works in this paper. Finally, we present theoretical analyses and discuss several future directions for MTL. [ABSTRACT FROM AUTHOR]

Benefiting from the rapid progress of wireless communication and artificial intelligence, multi-agent collaboration opens up new opportunities for various fields. To facilitate multi-agent acting as a group, effective communication plays a crucial role. Recently, many efforts based on multi-agent reinforcement learning have been made to enable effective multi-agent communication under limited bandwidth or noisy channel. However, current methods do not explore wireless resource allocation strategy explicitly. Moreover, due to ignoring task-relevant significance of information, traditional wireless resource allocation schemes may fail to guarantee the transmission efficiency and reliability for multi-agent communication. To this end, in this paper, we propose a task-oriented communication principle for multi-agent communication. We model the task-oriented channel allocation problem as a decentralized partially observable Markov decision process and propose a multi-agent reinforcement learning framework as a solution. Specifically, we design a novel variational information bottleneck to extract task-relevant information from local observation. Furthermore, a task-oriented channel allocation mechanism is developed to choose the allocation pattern with maximum expected gain. Finally, a double attention mechanism is developed to motivate the efficient utilization of task-relevant information. Experimental results show that our method can improve the effectiveness and efficiency of multi-agent communication, enhancing collaboration performance compared to baselines. [ABSTRACT FROM AUTHOR]

The accuracy of artificial intelligence (AI) models is crucial for connected and autonomous vehicles (CAVs). However, in reality, model training under less frequent weather faces the problem of insufficient sampling. Also, in the real world, weather, sunlight, etc., can only change with the speed of the real-time clock, so the traditional sampling process is very slow. Moreover, currently, collective learning, which can make up the limited experience and computing power of a single vehicle, is always introduced to cases where the data from participants have the same structure, wasting massive heterogeneous data from vehicles of different brands. Therefore, in this paper, we propose a quantum collective learning and many-to-many matching game-based scheme in the metaverse for CAVs. The environment is simulated in the metaverse, which has its own time clock system, thereby expanding sample size and speeding up the sampling process. And we quantify the quality of intelligence in collective learning from the perspective of feature diversity. It is the cornerstone of collective learning between heterogeneous vehicles, facilitating maximum utilization of data with different structures. Then, we formulate the distributed vehicles selection problem as a many-to-many matching game and use Gale–Shapely algorithm to solve it. Also, we formulate the spectrum resource allocation problem as a discrete Markov decision process (MDP) and adopt a quantum-inspired reinforcement learning (QRL) algorithm to find the optimal policy to achieve the high revenue of the system. In simulations, the performance of the proposed scheme is compared with existing methods. [ABSTRACT FROM AUTHOR]

Crowdsourcing is a hotspot research field which can facilitate machine learning by collecting labels to train models. Consequently, the state-of-the-art research efforts in crowdsourcing focus on truth inference or label integration, to remove inconsistent labels or to alleviate biased labeling. In turn, the integrated labels will be used to fine-tune machine learning models. Particularly, in this paper, we change the target of truth inference in crowdsourcing from discrete labels to multiple comments given by online participants, that is, the integration of the crowdsourced comments. For such a goal, we propose a Self-play and Sentiment-Emphasized Comment Integration Framework (SSECIF), based on deep Q-learning, with three unique features. First, our framework SSECIF can generate the comment integration in a totally self-play way, without relying on the ground truth generated by human effort. Second, the integrated comment generated by SSECIF can include salient content with low redundancy. Third, the proposed framework SSECIF has emphasized, with a higher intensity, the sentiment in the integrated comment, in order to reflect the attitude or opinion more obviously. Extensive evaluation on real-world datasets demonstrates that SSECIF has achieved the best overall performance in terms of both effectiveness and efficiency, compared with the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

This paper proposes the design of two coordinated wide-area damping controllers (CWADCs) for damping low frequency oscillations (LFOs), while accounting for the uncertainties present in the power system. The controllers based on Deep Neural Network (DNN) and Deep Reinforcement Learning (DRL), respectively, coordinate the operation of different local damping controls such as power system stabilizers (PSSs), static VAr compensators (SVCs), and supplementary damping controllers for DC lines (DC-SDCs). The DNN-CWADC learns to make control decisions using supervised learning; the training dataset consisting of polytopic controllers designed with the help of linear matrix inequality (LMI)-based mixed $H_2/H_\infty$ optimization. The DRL-CWADC learns to adapt to the system uncertainties based on its continuous interaction with the power system environment by employing an advanced version of the state-of-the-art deep deterministic policy gradient (DDPG) algorithm referred to as bounded exploratory control-based DDPG (BEC-DDPG). The studies performed on a 33 machine, 127 bus equivalent model of the Western Electricity Coordinating Council (WECC) system-embedded with different types of damping controls demonstrate the effectiveness of the proposed CWADCs. [ABSTRACT FROM AUTHOR]

Portfolio Selection is an important real-world financial task and has attracted extensive attention in artificial intelligence communities. This task, however, has two main difficulties: (i) the non-stationary price series and complex asset correlations make the learning of feature representation very hard; (ii) the practicality principle in financial markets requires controlling both transaction and risk costs. Most existing methods adopt handcraft features and/or consider no constraints for the costs, which may make them perform unsatisfactorily and fail to control both costs in practice. In this paper, we propose a cost-sensitive portfolio selection method with deep reinforcement learning. Specifically, a novel two-stream portfolio policy network is devised to extract both price series patterns and asset correlations, while a new cost-sensitive reward function is developed to maximize the accumulated return and constrain both costs via reinforcement learning. We theoretically analyze the near-optimality of the proposed reward, which shows that the growth rate of the policy regarding this reward function can approach the theoretical optimum. We also empirically evaluate the proposed method on real-world datasets. Promising results demonstrate the effectiveness and superiority of the proposed method in terms of profitability, cost-sensitivity and representation abilities. [ABSTRACT FROM AUTHOR]

Recently, unmanned aerial vehicle (UAV)-enabled multi-access edge computing (MEC) has been introduced as a promising edge paradigm for the future space-aerial-terrestrial integrated communications. Due to the high maneuverability of UAVs, such a flexible paradigm can improve the communication and computation performance for multiple user equipments (UEs). In this paper, we consider the sum power minimization problem by jointly optimizing resource allocation, user association, and power control in an MEC system with multiple UAVs. Since the problem is nonconvex, we propose a centralized multi-agent reinforcement learning (MARL) algorithm to solve it. However, the centralized method ignores essential issues like distributed framework and privacy concern. We then propose a multi-agent federated reinforcement learning (MAFRL) algorithm in a semi-distributed framework. Meanwhile, we introduce the Gaussian differentials to protect the privacy of all UEs. Simulation results show that the semi-distributed MAFRL algorithm achieves close performances to the centralized MARL algorithm and significantly outperform the benchmark schemes. Moreover, the semi-distributed MAFRL algorithm costs 23 $\%$ lower opeartion time than the centralized algorithm. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel adaptive importance sampling (AIS) algorithm for probabilistic inference. The sampler learns a proposal distribution adaptation strategy by framing AIS as a reinforcement learning problem. Under this structure, the proposal distribution of the sampler is treated as an agent whose state is controlled using a parameterized policy. At each iteration of the algorithm, the agent earns a reward that is related to its contribution to the variance of the AIS estimator of the normalization constant of the target distribution. Policy gradient methods are employed to learn a locally optimal policy that maximizes the expected value of the sum of all rewards. Numerical simulations on two different examples demonstrate promising results for the future application of the proposed method to complex Bayesian models. [ABSTRACT FROM AUTHOR]

In this paper, we propose a channel-wise interaction based binary convolutional neural networks (CI-BCNN) approach for efficient inference. Conventional binary convolutional neural networks usually apply the xnor and bitcount operations in the binary convolution with notable quantization errors, which obtain opposite signs of pixels in binary feature maps compared to their full-precision counterparts and lead to significant information loss. In our proposed CI-BCNN method, we exploit the channel-wise interactions with the prior knowledge which aims to alleviate inconsistency of signs in binary feature maps and preserves the information of input samples during inference. Specifically, we mine the channel-wise interactions by using a reinforcement learning model, and impose channel-wise priors on the intermediate feature maps to correct inconsistent signs through the interacted bitcount. Since CI-BCNN mines the channel-wise interactions in a large search space where each channel may correlate with others, the search deficiency caused by sparse interactions obstacles the agent to obtain the optimal policy. To address this, we further present a hierarchical channel-wise interaction based binary convolutional neural networks (HCI-BCNN) method to shrink the search space via hierarchical reinforcement learning. Moreover, we propose a denoising interacted bitcount operation in binary convolution by smoothing the channel-wise interactions, so that noise in channel-wise priors can be alleviated. Extensive experimental results on the CIFAR-10 and ImageNet datasets demonstrate the effectiveness of the proposed CI-BCNN and HCI-BCNN. [ABSTRACT FROM AUTHOR]

Resilience of a distribution system can be enhanced by efficient restoration of critical load following a major outage. Existing models include optimization approaches that consider available information without incorporating the inherent asynchrony of data arrival during execution of the restoration plan. Failure to consider the asynchronous nature of information arrival can lead to underutilization of critical resources. Moreover, analytical models become computationally inefficient for large scale systems. On the other hand, artificial intelligence (AI)-based tools have demonstrated efficient results for power system applications. In this paper, it is proposed a Reinforcement Learning (RL) model that learns how to efficiently restore a distribution system after a major outage. The proposed approach is based on a Monte Carlo Tree Search to expedite the training process. The proposed model strategy provides a robust decision-making tool for asynchronous and partial information scenarios. The results, validated with the IEEE 13-bus test feeder and IEEE 8500-node distribution test feeder, demonstrate the effectiveness and scalability of the proposed method. [ABSTRACT FROM AUTHOR]

Recently neural architecture search (NAS) has raised great interest in both academia and industry. However, it remains challenging because of its huge and non-continuous search space. Instead of applying evolutionary algorithm or reinforcement learning as previous works, this paper proposes a direct sparse optimization NAS (DSO-NAS) method. The motivation behind DSO-NAS is to address the task in the view of model pruning. To achieve this goal, we start from a completely connected block, and then introduce scaling factors to scale the information flow between operations. Next, sparse regularizations are imposed to prune useless connections in the architecture. Lastly, an efficient and theoretically sound optimization method is derived to solve it. Our method enjoys both advantages of differentiability and efficiency, therefore it can be directly applied to large datasets like ImageNet and tasks beyond classification. Particularly, on the CIFAR-10 dataset, DSO-NAS achieves an average test error 2.74 percent, while on the ImageNet dataset DSO-NAS achieves 25.4 percent test error under 600M FLOPs with 8 GPUs in 18 hours. As for semantic segmentation task, DSO-NAS also achieve competitive result compared with manually designed architectures on the PASCAL VOC dataset. Code is available at https://github.com/XinbangZhang/DSO-NAS. [ABSTRACT FROM AUTHOR]