Search

The explosive growth of Internet-of-Things (IoT) applications such as smart cities and Industry 4.0 have led to drastic increase in demand for wireless bandwidth, hence motivating the rapid development of new techniques for enhancing spectrum utilization needed by new generation wireless communication technologies. Among others, dynamic spectrum access (DSA) is one of the most widely accepted approaches. In this paper, as an enhancement of existing works, we take into consideration of inter-node collaborations in a dynamic spectrum environment. Typically, in such distributed circumstances, intelligent dynamic spectrum access almost invariably relies on self-learning to achieve dynamic spectrum access improvement. Whereas, this paper proposes a DSA scheme based on deep reinforcement learning to enhance spectrum and access efficiency. Unlike traditional Q-learning-based DSA, we introduce the following to enhance the spectrum efficiency in dynamic IoT spectrum environments. First, deep double Q-learning is adopted to perform local self-spectrum-learning for IoT terminals in order to achieve better dynamic access accuracy. Second, to accelerate learning convergence, federated learning (FL) in edge nodes is used to improve the self-learning. Third, multiple secondary users, who do not interfere with each other and have similar operation condition, are clustered for federated learning to enhance the efficiency of deep reinforcement learning. Comparing with the traditional distributed DSA with deep learning, the proposed scheme has faster access convergence speed due to the characteristic of global optimization for federated learning. Based on this, a framework of federated deep reinforcement learning (FDRL) for DSA is proposed. Furthermore, this scheme preserves privacy of IoT users in that FDRL only requires model parameters to be uploaded to edge servers. Simulations are performed to show the effectiveness of theproposed FDRL-based DSA framework. [ABSTRACT FROM AUTHOR]

Radio signal recognition has a wide application in future communication systems and the vehicular communication, whose core is the extraction of signal features such as electromagnetic fingerprints. With the rapid development of artificial intelligence technology, deep learning has made amazing breakthroughs in image recognition, speech recognition and other fields. Deep learning is applied to electromagnetic fingerprint extraction in this paper. Firstly, thousands of the downlink aircraft communications addressing and reporting system (ACARS) signals used for communication between civil aircraft and airport tower are collected and generated. Then a pre-transformation network suitable for electromagnetic signals is constructed to convert one-dimensional signals into two-dimensional feature maps, and afterwards the feature maps are input into the convolutional neural network (CNN) for classification. By adopting the attention modules, the classification results were improved by a few percentage points over the baseline with a little cost. The method proposed in this paper achieves an accuracy rate of 94.1% and can obtain the aircraft type in a shorter time than traditional method. Moreover, the robustness of the proposed model in response to additive Gaussian white noise (AWGN) and phase deviation is studied and tested. [ABSTRACT FROM AUTHOR]

This paper proposes a deep reinforcement learning-based video streaming scheme for mobility-aware vehicular networks, e.g., vehicles on the highway. We consider infrastructure-assisted and mmWave-based scenarios in which the macro base station (MBS) cannot directly provide the streaming service to vehicles due to the short range of mmWave beams so that small mmWave base stations (mBSs) along the road deliver the desired videos to users. Motivated by the fact that a video stream consists of sequential chunks and smooth streaming has to be provided while users pass through multiple mBSs at high speed, the MBS proactively pushes some video chunks of the desired contents to mBSs. This is done to support vehicles that are currently covered and/or will be by each mBS. We formulate the dynamic video delivery scheme that adaptively determines 1) which content, 2) what quality and 3) how many chunks to be proactively delivered from the MBS to mBSs using Markov decision process (MDP). Since it is difficult for the MBS to track all the channel conditions and the network states have extensive dimensions, we adopt the deep deterministic policy gradient (DDPG) algorithm for the DRL-based video delivery scheme. This paper finally shows that the DRL agentlearns a streaming policy that pursues high average quality while limiting packet drops, avoiding playback stalls, reducing quality fluctuations and saving backhaul usage. [ABSTRACT FROM AUTHOR]

This paper studies the allocation of shared resources between vehicle-to-infrastructure (V2I) and vehicle-to-vehicle (V2V) links in vehicle-to-everything (V2X) communications. In existing algorithms, dynamic vehicular environments and quantization of continuous power become the bottlenecks for providing an effective and timely resource allocation policy. In this paper, we develop two algorithms to deal with these difficulties. First, we propose a deep reinforcement learning (DRL)-based resource allocation algorithm to improve the performance of both V2I and V2V links. Specifically, the algorithm uses deep Q-network (DQN) to solve the sub-band assignment and deep deterministic policy-gradient (DDPG) to solve the continuous power allocation problem. Second, we propose a meta-based DRL algorithm to enhance the fast adaptability of the resource allocation policy in the dynamic environment. Numerical results demonstrate that the proposed DRL-based algorithm can significantly improve the performance compared to the DQN-based algorithm that quantizes continuous power. In addition, the proposed meta-based DRL algorithm can achieve the required fast adaptation in the new environment with limited experiences. [ABSTRACT FROM AUTHOR]

The Internet of vehicles (IoV) is a large information interaction network that collects information on vehicles, roads and pedestrians. One of the important uses of vehicle networks is to meet the entertainment needs of driving users through communication between vehicles and roadside units (RSUs). Due to the limited storage space of RSUs, determining the content cached in each RSU is a key challenge. With the development of 5G and video editing technology, short video systems have become increasingly popular. Current widely used cache update methods, such as partial file precaching and content popularity- and user interest-based determination, are inefficient for such systems. To solve this problem, this paper proposes a QoE-driven edge caching method for the IoV based on deep reinforcement learning. First, a class-based user interest model is established. Compared with the traditional file popularity- and user interest distribution-based cache update methods, the proposed method is more suitable for systems with a large number of small files. Second, a quality of experience (QoE)-driven RSU cache model is established based on the proposed class-based user interest model. Third, a deep reinforcement learning method is designed to address the QoE-driven RSU cache update issue effectively. The experimental results verify the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

This paper proposes a cloud-assisted joint charging scheduling and energy management framework for unmanned aerial vehicle (UAV) networks. For charging the UAVs those are extremely power hungry, charging towers are considered for plug-and-play charging during run-time operations. The charging towers should be cost-effective, thus it is equipped with photovoltaic power generation and energy storage systems functionalities. Furthermore, the towers should be cooperative for more cost-effectiveness by intelligent energy sharing. Based on the needs and setting, this paper proposes 1) charging scheduling between UAVs and towers and 2) cooperative energy managements among towers. For charging scheduling, the UAVs and towers should be scheduled for maximizing charging energy amounts and the scheduled pairs should determine charging energy allocation amounts. Here, two decisions are correlated, i.e., it is a non-convex problem. We re-formulate the non-convex to convex for guaranteeing optimal solutions. Lastly, the cooperative energy sharing among towers is designed and implemented with multi-agent deep reinforcement learning and then intelligent energy sharing can be realized. We can observe that the two methods are related and it should be managed, coordinated, and harmonized by a centralized orchestration manager under the consideration of fairness, energy-efficiency, and cost-effectiveness. Our data-intensive performance evaluation verifies that our proposed framework achieves desired performance. [ABSTRACT FROM AUTHOR]

In this paper, we design an Anomaly Detection (AD) framework for mobile data traffic, capable of identifying different types of anomalous events generated by flash crowds in metropolitan areas. We state the problem using a semi-supervised approach and exploit the great performance of different Recurrent Neural Network (RNN) models to learn the temporal context of input sequences. Our proposal processes real traffic traces from the unencrypted LTE Physical Downlink Control Channel (PDCCH) of an operative network, gathered during an extensive measurement campaign in two major cities in Spain. The AD framework is designed to perform: i) a-posteriori analysis to understand users’ behavior and urban environment variations; ii) real-time analysis to automatically and on-the-fly alert urban anomalies; and iii) estimation of the duration of the periods identified as anomalous. Numerical results show the higher performance of our AD framework compared to classic AD algorithms and confirm that the proposed framework predicts anomalous behaviours with high accuracy and regardless of their cause. [ABSTRACT FROM AUTHOR]

Computation offloading is an effective method in mobile edge computing (MEC) to relieve user equipment (UE) from the limited computation resource and battery capacity. Meanwhile, simultaneous wireless information and power transmission (SWIPT) can be applied to MEC to extend the operating time of the equipment. However, in multi-user network environment, diverse computation task requirements and changeable network channel states make it challenging to obtain offloading strategy timely and accurately. To address the issue, we propose an intelligent computation offloading scheme (iCOS) based on enhanced priority deep deterministic policy gradient (EPDDPG) algorithm to minimize the energy consumption of all the UEs by jointly optimizing the offloading decision, the central processing unit (CPU) frequency and the power split ratio in a dynamic SWIPT-MEC network. In particular, we improve the traditional fully-connected network structure to obtain both discrete and continuous action outputs, and accelerate neural network parameter updates by using prioritized experience tuples. Furthermore, we use dynamic voltage and frequency scaling (DVFS) technology to dynamically adjust the CPU frequency of local computing, and employ SWIPT technology to balance the charging and communication according to the obtained strategy. Simulation results show that the algorithm proposed in this paper can effectively reduce the energy cost of UEs, and complete more computation tasks within the delay limit. [ABSTRACT FROM AUTHOR]

This paper is concerned with the efficient movement control of multiple drone cells for communication coverage. Although many works have been developed to cope with this problem, but only few of them have investigated the three-dimensional (3-D) continuous movement control of multiple drone cells. In this paper, a problem of 3-D continuous movement control of multiple drone cells is formulated with an objective of maximizing the energy-efficient communication coverage of drone-cell networks while preserving the network connectivity. To mitigate this challenging problem, an energy-efficiency and continuous-movement-control algorithm (E $^2$ CMC), which is based on an emerging deep reinforcement learning method, is proposed. In E $^2$ CMC, an energy-efficiency reward function considering the energy consumption, the sum of quality-of-service (QoS) requirements of users as well as the coverage fairness is first designed. Next, E $^2$ CMC learns to adjust drone cells’ locations continuously by interacting with an environment in a sequence of observations, actions, and rewards. Furthermore, E $^2$ CMC will reduce the reward drastically as long as the networks are disconnected. Simulation results verify the superiority of the proposed learning algorithm on deploying multiple drone cells to provide the communication coverage. [ABSTRACT FROM AUTHOR]

Unmanned aerial vehicles (UAVs) control information delivery is a critical communication with stringent requirements in terms of reliability and latency. In this context, link adaptation plays an essential role in the fulfillment of the required performance in terms of decode error probability and delay. Link adaptation is usually based on channel quality indicator (CQI) feedback information from the user equipment that should represent the current state of the channel. However, measurement, scheduling and processing delays introduce a CQI aging effect, that is a mismatch between the current channel state and its CQI representation. Using outdated CQI values may lead to the selection of a wrong modulation and coding scheme, with a detrimental effect on performance. This is particularly relevant in ultra reliable and low latency communications (URLLC), where the control of the reliability can be negatively impacted, and it is more evident when the channel is fast varying as the case of UAVs. This paper analyzes the effects of CQI aging on URLLCs, considering transmissions under the finite blocklength regime, that characterizes such communications type. A deep learning approach is investigated to predict the next CQI from the knowledge of past reports, and performance in terms of decode error probability and throughput is given. The results show the benefit of CQI proposed prediction mechanism also in comparison with previously proposed methods. [ABSTRACT FROM AUTHOR]

Automatic modulation classification (AMC) plays an important role in both civilian and military applications. Today, increasingly more researchers apply a deep learning framework in AMC. However, few papers take into account that a typical deep model is difficult to deploy on resource constrained devices. In this paper, we propose a new filter-level pruning technique based on activation maximization (AM) that omits the less important convolutional filter. Compared to other network pruning techniques, the convolutional neural network pruned via the AM method achieves equal or higher classification accuracy in the RadioML2016.10a dataset. [ABSTRACT FROM AUTHOR]

Characterization and linearization of RF power amplifiers (PAs) are key issues of fifth-generation wireless communication systems, especially when high peak-to-average ratio waveforms are introduced. Recently, deep learning methods have achieved great success in numerous domains including wireless physical-layer. However, there has been limited work in using deep learning for PAs behavioral modeling and linearization. In this paper, we make a bridge between memory effects of the nonlinear PAs and memory of bidirectional long short-term memory (BiLSTM) neural networks. We then build a BiLSTM-based behavioral modeling architecture and its accompanying digital predistortion (DPD) model by reconciling a non causality concern. Next, an additional model is proposed in this paper to mitigate uncertainty of the tested PA when transforming phases. The experimental results demonstrate the effectiveness of the proposed scheme, in which the adequately trained networks are capable of characterizing the PA, and the artificial intelligence-based DPD shows promising linearization performance when considering the tested PAs inherent unpredictability. [ABSTRACT FROM AUTHOR]

Channel allocation has a direct and profound impact on the performance of vehicle-to-everything (V2X) networks. Considering the dynamic nature of vehicular environments, it is appealing to devise a blended strategy to perform effective resource sharing. In this paper, we exploit deep learning techniques predict vehicles’ mobility patterns. Then we propose an architecture consisting of centralized decision making and distributed channel allocation to maximize the spectrum efficiency of all vehicles involved. To achieve this, we leverage two deep reinforcement learning techniques, namely deep Q-network (DQN) and advantage actor-critic (A2C) techniques. In addition, given the time varying nature of the user mobility, we further incorporate the long short-term memory (LSTM) into DQN and A2C techniques. The combined system tracks user mobility, varying demands and channel conditions and adapt resource allocation dynamically. We verify the performance of the proposed methods through extensive simulations and prove the effectiveness of the proposed LSTM-DQN and LSTM-A2C algorithms using real data obtained from California state transportation department. [ABSTRACT FROM AUTHOR]

Due to the scarcity in the wireless spectrum and limited energy resources especially in mobile applications, efficient resource allocation strategies are critical in wireless networks. Motivated by the recent advances in deep reinforcement learning (DRL), we address multi-agent DRL-based joint dynamic channel access and power control in a wireless interference network. We first propose a multi-agent DRL algorithm with centralized training (DRL-CT) to tackle the joint resource allocation problem. In this case, the training is performed at the central unit (CU) and after training, the users make autonomous decisions on their transmission strategies with only local information. We demonstrate that with limited information exchange and faster convergence, DRL-CT algorithm can achieve 90% of the performance achieved by the combination of weighted minimum mean square error (WMMSE) algorithm for power control and exhaustive search for dynamic channel access. In the second part of this paper, we consider distributed multi-agent DRL scenario in which each user conducts its own training and makes its decisions individually, acting as a DRL agent. Finally, as a compromise between centralized and fully distributed scenarios, we consider federated DRL (FDRL) to approach the performance of DRL-CT with the use of a central unit in training while limiting the information exchange and preserving privacy of the users in the wireless system. Via simulation results, we show that proposed learning frameworks lead to efficient adaptive channel access and power control policies in dynamic environments. [ABSTRACT FROM AUTHOR]

Past few years have witnessed the widespread adoption of unmanned aerial vehicles (UAVs) in target tracking for regional monitor and strike. Most existing target tracking approaches rely on the target motion frames obtained by the camera equipped, or on ideally assuming a pre-set target trajectory. However, in practice, the real trajectory of the target cannot be perfectly known to the UAVs in advance, and also the target may intelligently adjust its flying strategy according to the environment. Besides, the limited flight performance, as well as information capture and processing capability, of a single UAV can hardly fulfill high tracking success rate requirements. To address aforementioned issues, this paper proposes an end-to-end cooperative multi-agent reinforcement learning (MARL) scheme, where UAVs are enabled to make intelligent flight decisions for cooperative target tracking, on the basis of the past and current states of the target. In order to reduce power consumption and prolong the lifetime of the UAV tracking system, the propulsion power consumption model and energy saving strategy are introduced. Moreover, to further increase the detection coverage, spatial information entropy is introduced in the tracking algorithm. Simulation results show that our proposed algorithm outperfoms the deep reinforcement learning baselines in terms of the mean episode rewards, while also yields high performances with respect to tracking success rates, power saving efficiency and detection coverage. [ABSTRACT FROM AUTHOR]

One of the pillars of the 5G architecture is network slicing, in which hardware, radio, and power resources are virtualized as a logical network taking into account the requirements of diverse applications. While ensuring performance isolation among different slices, resource allocation in 5G Radio Access Networks (RANs) is associated with different challenges due to network dynamics and the different applications’ requirements. In this paper, we have considered the allocation of power and radio resources to rate-based as well as resource-based users. We propose an energy-efficient deep reinforcement learning-assisted resource allocation (EE-DRL-RA) method for RAN slicing in 5G networks. The main idea of the proposed method is to exploit a collaborative learning framework that includes deep reinforcement learning (DRL) and deep learning (DL) to decide on resource allocation in the RAN. Specifically, we use DL for decision-making on resource allocation on a large time-scale and DRL for decision-making on resource allocation on a small time-scale. The asynchronous advantage actor-critic (A3C) and the stacked and bidirectional long-short-term-memory (SBiLSTM) network are used as DRL and supervised DL methods, respectively. Furthermore, we determine the optimal power and resource blocks (RBs) for rate-based users by formulating the energy-efficient power allocation (EE-PA) problem as a non-convex optimization problem and solve it by an efficient iterative algorithm. Our proposed approach is unique in that it simultaneously allocates power and RBs while ensuring slice isolation with low computational and time complexity. Simulation results show that EE-DRL-RA yields better performance compared to a state-of-the-art published method in terms of convergence speed, computational complexity, energy efficiency, and the number of accepted users as well as the degree of inter-slice isolation. [ABSTRACT FROM AUTHOR]

Knowledge transfer is a promising concept to achieve real-time decision-making for autonomous vehicles. This paper constructs a transfer deep reinforcement learning (RL) framework to transform the driving tasks in the intersection environments. The driving missions at the unsignalized intersection are cast into a left turn, right turn, and running straight for automated vehicles. The goal of the autonomous ego vehicle (AEV) is to drive through the intersection situation efficiently and safely. This objective promotes the studied vehicle to increase its speed and avoid crashing other vehicles. The decision-making policy learned from one driving task is transferred through three transfer rules in another driving mission and evaluated. Simulation results reveal that the decision-making strategies related to similar tasks are transferable and have a high success rate. It indicates that the presented control framework could reduce time consumption and realize online implementation. Therefore, the transfer RL concept is helpful for establishing the real-time decision-making policy for autonomous vehicles. [ABSTRACT FROM AUTHOR]

In this paper, we study the three-dimensional (3D) path planning problem for cellular-connected unmanned aerial vehicle (UAV) taking into account the impact of 3D antenna radiation patterns. The cellular-connected UAV has a mission to travel from an initial location to a destination. In this process, there is a trade-off between the flight time and the expected communication outages duration, which requires planning a suitable route to avoid the weak coverage region. However, as the existing cellular networks are primarily designed for ground coverage, the communication coverage in the sky tends to be intermittent and irregular due to the potential for severe interference and obstructions (such by buildings), which brings great challenges to 3D path planning. To address this challenge, we first construct a 3D coverage map that stores the expected outage probability over each locations. We then propose a multi-step dueling DDQN (multi-step D3QN) based algorithm to design the local optimal UAV path by leveraging the constructed coverage map. In this algorithm, the UAV acts as the agent to learn the appropriate action to complete the flight mission. Numerical results show the effectiveness of the proposed algorithm for connectivity-aware UAV path planning and the superiority of 3D path design over its 2D counterparts. [ABSTRACT FROM AUTHOR]

Air ground integrated mobile cloud computing (MCC) provides unmanned aerial vehicles (UAVs) the capability to act as an aerial relay with more flexibility and resilience. In the cloud computing architecture, the data generated by ground users (GUs) can be offloaded to the remote server for fast processing. However, the heterogeneity of mobile tasks makes the data size distributed among GUs unbalanced. Besides, the energy efficiency of UAVs movement should be carefully considered for sustainable flight and obstacle avoidance. In general, such a joint trajectory issue can hardly be formulated as a convex optimization in unpredictable and dynamic environments. This paper proposes a potential game combined multi-agent deep deterministic policy gradient (MADDPG) approach to optimize multiple UAVs' trajectory with the consideration of GUs' offloading delay, energy efficiency as well as obstacle avoidance system. In specific, we first model the issue as a mixed integer non-linear problem (MINP), in which the service assignment between multi-user and multi-UAV is solved by potential game. The convergence to a Nash Equilibrium (NE) can be achieved by distributive service assignment update with infinite iteration. Then, we optimize the trajectory with obstacle avoidance at each UAV by MADDPG approach, which has a great advantage of centralized-training and decentralized-execution to reduce the global synchronized communication overhead. UAVs movement can be optimized in continuity rather than other deep reinforcement learning (DRL) approaches generating discrete simple actions. Experiments demonstrate the proposed game-combined learning algorithm can minimize the offloading delay, enhance UAVs’ energy efficiency and avoid the obstacles at the same time. [ABSTRACT FROM AUTHOR]

In this paper, we consider an underwater linear network, where an autonomous underwater vehicle (AUV) gathers data from a set of underwater devices. The AUV monitors a set of physical processes, where the status of each process can be sensed by one or more devices and each device is capable of sensing one or more processes. The AUV needs to maintain freshness of its information status about the monitored processes. To quantify the freshness of the information at the AUV, we consider the concept of the age of information (AoI), which represents the amount of time elapsed since the most recently delivered update information was generated. A framework is proposed to optimize the AUV's linear movement trajectory and scheduling of process status updates with the objective of minimizing the normalized weighted sum of the average AoI of the monitored physical processes. The formulated optimization problem is a non-convex mixed integer problem, which cannot be solved by the standard optimization techniques. We develop a solution approach based on the technique of deep reinforcement learning (DRL). Specifically, we leverage an actor-critic DRL approach to find the optimum locations and stopping time of the data gathering points. Simulation results illustrate that the proposed framework maintains robustness under different scenarios and provides better performance when compared with baseline and $K$ -means clustering approaches. [ABSTRACT FROM AUTHOR]

We are seeing a growth in the number of connected vehicles in Vehicular Ad-hoc Networks (VANETs) to achieve the goal of Intelligent Transportation System (ITS). This is leading to a connected vehicular network scenario with vehicles continuously broadcasting data to other vehicles on the road and the roadside network infrastructure. The presence of a large number of communicating vehicles greatly increases the number and types of possible anomalies in the network. Existing works provide solutions addressing specific anomalies in the network only. However, since there can be a multitude of anomalies possible in the network, there is a need for better anomaly detection frameworks that can address this unprecedented scenario. In this paper, we propose an anomaly detection framework for VANETs based on deep neural networks (DNNs) using a sequence reconstruction and thresholding algorithm. In this framework, the DNN architectures are deployed on the roadside units (RSUs) which receive the broadcast vehicular data and run anomaly detection tasks to classify a particular message sequence as anomalous or genuine. Multiple DNN architectures are implemented in this experiment and their performance is compared using key evaluation metrics. Performance comparison of the proposed framework is also drawn against the prior work in this area. Our best performing deep learning-based scheme detects anomalous sequences with an accuracy of 98%, a great improvement over the set benchmark. [ABSTRACT FROM AUTHOR]

In vehicle trajectory prediction, the difficulty in modeling the interaction relationship between vehicles lies in constructing the interaction structure between the vehicles in the traffic scene. Majority of existing models only focus on the interaction between the historical trajectory of the vehicle and the surrounding vehicles in the spatial domain, and do not pay attention to the interaction between the vehicle and the non-Euclidean correlation structure (graph structure) that exists in the environment. In order to overcome the deficiencies in the existing models, this paper proposes the Environment-Attention Network model (EA-Net) to obtain the full interactive information between the vehicle and its driving environment. In the proposed model, a new type of parallel structure consisting of Graph Attention network (GAT) and Convolutional social pooling containing Squeeze-and-Extraction mechanism (SE-CS), is constructed as the environmental feature extraction module and embedded in LSTM encoder-decoder. This structure solves the limitation of the dimension and structure influence when constructing the interaction model between the vehicle and the surrounding environment, making the extracted feature information comprehensive and effective. The prediction accuracy of the model with the RMSE loss function is tested on two public datasets– NGSIM and highD, and compared with several state-of-the-art trajectory prediction algorithm models. The results show that the prediction accuracy of the proposed Environment-Attention Network in the two datasets is more than 20% higher than that of the single structure model, which indicates that the proposed model proposed has superior performance and better adaptability to different traffic environments compared with the existing models. [ABSTRACT FROM AUTHOR]

Network slicing (NS) management devotes to providing various services to meet distinct requirements over the same physical communication infrastructure and allocating resources on demands. Considering a dense cellular network scenario that contains several NS over multiple base stations (BSs), it remains challenging to design a proper real-time inter-slice resource management strategy, so as to cope with frequent BS handover and satisfy the fluctuations of distinct service requirements. In this paper, we propose to formulate this challenge as a multi-agent reinforcement learning (MARL) problem in which each BS represents an agent. Then, we leverage graph attention network (GAT) to strengthen the temporal and spatial cooperation between agents. Furthermore, we incorporate GAT into deep reinforcement learning (DRL) and correspondingly design an intelligent real-time inter-slice resource management strategy. More specially, we testify the universal effectiveness of GAT for advancing DRL in the multi-agent system, by applying GAT on the top of both the value-based method deep Q-network (DQN) and a combination of policy-based and value-based method advantage actor-critic (A2C). Finally, we verify the superiority of the GAT-based MARL algorithms through extensive simulations. [ABSTRACT FROM AUTHOR]

To meet the execution requirements of delay-sensitive services in vehicular edge computing (VEC) networks, vehicular services need to be offloaded to edge computing nodes. For complex, large-scale services, the services need to be migrated if the services are not completed before the vehicles leave the coverage of edge computing nodes. Trust and resource matching between areas thus become major problems. This paper studies the decision model of vehicular service offloading and migration. First, software-defined network (SDN) technology is introduced into the traditional network architecture, and a two-layer distributed SDN-controlled VEC network architecture is designed, which is divided into a domain control layer and an area control layer. In this framework, we use the consortium blockchain as a carrier to share network topology information between SDN controllers to prevent information leakage. We then established a service offloading and migration optimization problem model to minimize service execution delay, reduce energy consumption and maximize the throughput of the blockchain system. We describe the problem model as a Markov Decision Process (MDP), introduce a deep reinforcement learning (DRL) algorithm named asynchronous advantage actor-critic (A3C) and design a dynamic service offloading and migration algorithm (DSOMA) based on A3C to solve the problem. Simulation results show that DSOMA can increase the throughput of the blockchain system, and DSOMA is superior to the deep Q-learning (DQN) algorithm and greedy offloading algorithm in reducing service execution delay and system energy consumption. [ABSTRACT FROM AUTHOR]

Hybrid precoding is used in millimeter wave (mmWave) massive multiple-input multiple-output (MIMO) systems instead of full digital precoding to reduce the cost and power consumption. Conventional hybrid precoding however requires complex channel estimation before beamforming. In this paper, we propose a low complexity low-cost channel estimation-free deep direct beamforming method composed of two basic steps-beam impulse and beamforming for time division duplex (TDD). This method is built upon randomized singular value decomposition (RSVD) and can directly complete the process of beamforming. Furthermore, a realistic dataset of mmWave channel-DeepMIMO is adopted to evaluate and determine the practicability of this method. The simulation results demonstrate that the proposed deep direct beamforming achieves high performance while its complexity and power consumption are very low. Compared with other methods without perfect channel status information (CSI), the proposed method costs less and requires lower computational resources while achieves higher spectral efficiency. Furthermore, the RSVD can be easily combined with conventional beamforming methods hence the proposed method is easily extendable to other applications. [ABSTRACT FROM AUTHOR]

In this paper, we investigate the problem of wireless service provisioning through a rotary-wing UAV which can serve as an aerial base station (BS) to communicate with multiple ground terminals (GTs) in a boost demand area. Our objective is to optimize the UAV control for maximizing the UAV.s energy efficiency, in which both aerodynamic energy and communication energy are considered while ensuring the communication requirements for each GT and backhaul link between the UAV and the terrestrial BS. The mobility of the UAV and GTs lead to time-varying channel conditions that make the environment dynamic. We formulate a nonconvex optimization for controlling the UAV considering the practical angle-dependent Rician fading channels between the UAV and GTs, and between the UAV and the terrestrial BS. Traditional optimization approaches are not able to handle the dynamic environment and high complexity of the problem in real-time. We propose to use a deep reinforcement learning-based approach namely Deep Deterministic Policy Gradient (DDPG) to solve the formulated nonconvex problem of UAV control with continuous action space that takes into account the real-time of the environment including time-varying UAV-ground channel conditions, available onboard energy of the UAV, and the communication requirement of the GTs. However, the DDPG method may not achieve good performance in an unstable environment and will face a large number of hyperparameters. We extend our approach to use the Trust Region Policy Optimization (TRPO) method that can improve the performance of the UAV compared to the DDPG method in such a dynamic environment. [ABSTRACT FROM AUTHOR]

Recently, inferring lane change intention has received considerable attention. Due to the high nonlinearity and complexity of traffic contexts, traditional methods cannot satisfy the requirements of long-term prediction tasks and lack the ability of capturing nonlinear temporal dependencies. This paper proposes an intention inference model based on Recurrent Neural Networks (RNN), to tackle time series prediction problems. Considering dynamic interaction among surrounding vehicles, our model takes the sequence motion information of surrounding vehicles as inputs and calculates the congestion of different lanes, integrated with vehicle states of the object vehicle. To illustrate the availability of the proposed RNN intention inference model, a motion planning controller considering intention was developed. A Nonlinear Model Predictive Control (NMPC) was established to planning a safe, sub-optimal path for autonomous driving vehicle under collision avoidance constraints. The experiments on the proposed model were conducted, based on two RNN structure Long-Short Term Memory (LSTM) and Generalized Recurrent Unit (GRU), by Tensorflow with NGSIM data. The motion planning controller is modeled and simulated by Carsim with Simulink for some typical scenarios. Subsequently, experimental results demonstrate that RNN achieves best performance, inferring intention with 96% accuracy, compared with other approaches. [ABSTRACT FROM AUTHOR]

State-of-the-art drone technologies have severe flight time limitations due to weight constraints, which inevitably lead to a relatively small amount of available energy. Therefore, frequent battery replacement or recharging is necessary in applications such as delivery, exploration, or support to the wireless infrastructure. Mobile charging stations (i.e., mobile stations with charging equipment) for outdoor ad-hoc battery charging is one of the feasible solutions to address this issue. However, the ability of these platforms to charge the drones is limited in terms of the number and charging time. This paper designs an auction-based mechanism to control the charging schedule in multi-drone setting. In this paper, charging time slots are auctioned, and their assignment is determined by a bidding process. The main challenge in developing this framework is the lack of prior knowledge on the distribution of the number of drones participating in the auction. Based on optimal second-price-auction, the proposed formulation, then, relies on deep learning algorithms to learn such distribution online. Numerical results from extensive simulations show that the proposed deep-learning-based approach provides effective battery charging control in multi-drone scenarios. [ABSTRACT FROM AUTHOR]

In this paper, we study a multiuser mobile edge computing (MEC) network, where tasks from users can be partially offloaded to multiple computational access points (CAPs). We consider practical cases where task characteristics and computational capability at the CAPs may be time-varying, thus, creating a dynamic offloading problem. To deal with this problem, we first formulate it as a Markov decision process (MDP), and then introduce the state and action spaces. We further design a novel offloading strategy based on the deep Q network (DQN), where the users can dynamically fine-tune the offloading proportion in order to ensure the system performance measured by the latency and energy consumption. Simulation results are finally presented to verify the advantages of the proposed DQN-based offloading strategy over conventional ones. [ABSTRACT FROM AUTHOR]

This paper tackles the joint user association (UA) and power control (PC) problem for dynamic heterogeneous networks (HetNets), where the number of active users varies over time. Since the joint UA and PC problem is a mixed-integer problem, it is challenging to develop a joint UA and PC algorithm that closely achieves the optimal proportional fairness (PF) performance while satisfying the tight computation time budget of recent 3GPP standards. Although previous studies have reduced the computation time on the UA and PC by employing a fully-connected-neural network (FCN), the FCN architecture is not generally applicable to dynamic HetNets because of a varying number of users. In pursuit of scalable neural network design, we propose an unsupervised learning-based UA and PC algorithm using a recurrent neural network (RNN). Notably, in the RNN-based UA, we relax the combinatorial UA variable into a low-dimensional continuous variable, the length of which is invariant to the number of users. The optimality loss in the variable relaxation is upper-bounded by the negligibly small duality gap. Through extensive experiments, we show that the proposed scheme has a 10% higher PF performance than conventional optimization-based methods with the same number of flops and closely achieves the optimal PF performance for various numbers of users. [ABSTRACT FROM AUTHOR]

The flow fluctuations in the highly dynamic Internet of Vehicles (IoV) make the IoV difficult to provide reliable and scalable wireless network services for the emerging applications in the 5 G and beyond era. The software-defined networks (SDN) could feasibly manage and optimize the network according to the network load. Controller placement is a critical problem in SDN to achieve its robustness and flexibility with the changes of network status. Motivated by the advantages of SDN and Mobile-edge computing (MEC), this paper aims at enhancing the performance of IoV with the assistance of these two. Specifically, we consider a three-layer hierarchical SDN control plane for the IoV, where the SDN controllers are placed at the edge of networks. Under this framework, we investigate a multi-objective optimization problem on controller placement problem including delay, load balancing, and path reliability. To efficiently solve the formulated NP-hard problems, we develop an algorithm based on multi-agent deep Q-learning networks (MADQN) because of its advantages for large-scale combinatorial optimization. At last, we use multi-process technology to accelerate the operation of the algorithm, so as to complete the algorithm iteration faster. Numerical results show that the proposed methods achieve better performances than three baselines. [ABSTRACT FROM AUTHOR]

The efficiency of link adaptation in wireless communications relies greatly on the accuracy of channel knowledge and transmission mode selection. In this paper, a novel deep learning based link adaptation framework is proposed for the orthogonal frequency-division multiplexing (OFDM) systems with compressed-sensing-assisted index modulation, termed as OFDM-CSIM, communicating over millimeter-wave (mmWave) channels. To achieve link adaptation, a novel multi-layer sparse Bayesian learning (SBL) algorithm is proposed for accurately and instantaneously providing the required channel state information. Meanwhile, a deep neural networks (DNN)-assisted adaptive modulation algorithm is proposed to choose the best possible transmission mode to maximize the achievable throughput. Simulation results show that the proposed multi-layer SBL algorithm enables more accurate channel estimation than the conventional techniques. The DNN-based adaptive modulator is capable of achieving a higher throughput than the learning-assisted solution based on the $k$ nearest neighbor ($k$ -NN) algorithm, and also the classic average signal-to-noise ratio (SNR)-based solutions. Moreover, analysis shows that both the multi-layer SBL algorithm and the DNN-assisted adaptive modulator achieve better performance than their respective conventional counterparts while at a significantly lower computational complexity cost. [ABSTRACT FROM AUTHOR]

Extracting channel state information (CSI) from WiFi signals is of proved high-effectiveness in locating human locations in a device-free manner. However, existing localization/positioning systems are mainly trained and deployed in a fixed environment, and thus they are likely to suffer from substantial performance declines when immigrating to new environments. In this paper, we address the fundamental problem of WiFi-based cross-environment indoor localization using a semi-supervised approach, in which we only have access to the annotations of the source environment while the data in the target environments are un-annotated. This problem is of high practical values in enabling a well-trained system to be scalable to new environments without tedious human annotations. To this end, a deep neural forest is introduced which unifies the ensemble learning with the representation learning functionalities from deep neural networks in an end-to-end trainable fashion. On top of that, an adversarial training strategy is further employed to learn environment-invariant feature representations for facilitating more robust localization. Extensive experiments on real-world datasets demonstrate the superiority of the proposed methods over state-of-the-art baselines. Compared with the best-performing baseline, our model excels with an average 12.7% relative improvement on all six evaluation settings. [ABSTRACT FROM AUTHOR]

As an emerging technology in 5G, ultra-dense unmanned aerial vehicles (UAVs) network can significantly improve the system capacity and networks coverage. However, it is still a challenge to reduce interference and improve energy efficiency (EE) of UAVs. In this paper, we investigate a downlink power control problem to maximize the EE in an ultra-dense UAV network. Firstly, the power control problem is formulated as a discrete mean field game (MFG) to imitate the interactions among a large number of UAVs, and then the MFG framework is transformed into a Markov decision process (MDP) to obtain the equilibrium solution of the MFG due to the dense deployment of UAVs. Specifically, a deep reinforcement learning-based MFG (DRL-MFG) algorithm is proposed to suppress the interference and maximize the EE by using deep neural networks (DNN) to explore the optimal power strategy for UAVs. The numerical results show that the UAVs can effectively interact with the environment to obtain the optimal power control strategy. Compared with the benchmarks algorithms, the DRL-MFG algorithm converges faster to the solution of MFG and improves the EE of UAVs. Moreover, the impact of the transmit power on EE under the different heights of the UAVs is also analyzed. [ABSTRACT FROM AUTHOR]

The Internet of Vehicles (IoV) as a promising application of Internet of Things (IoT) has played a significant role in autonomous driving, by connecting intelligent vehicles. Autonomous driving needs to process the mass environmental sensing data in coordination with surrounding vehicles, and makes an accurate driving judgment accordingly. Since the vehicles always have limited computing resources, processing these data in parallel with efficient task scheduling is one of the most important topics. Most current work focuses on formulating special scenarios and service requirements as optimization problems. However, the complicated and dynamic environment of vehicular computing is hard to model, predict and control, making those previous methods unscalable and unable to reflect the real scenario. In this paper, a Multi-task Deep reinforcement learning approach for scalable parallel Task Scheduling (MDTS) is firstly devised. For avoiding the curse of dimensionality when coping with complex parallel computing environments and jobs with diverse properties, we extend the action selection in Deep Reinforcement Learning (DRL) to a multi-task decision, where the output branches of multi- task learning are fine-matched to parallel scheduling tasks. Child tasks of a job are accordingly assigned to distributed nodes without any human knowledge while the resource competition among parallel tasks is leveraged through shared neural network layers. Moreover, we design an appropriate reward function to optimize multiple metrics simultaneously, with emphasis on specific scenarios. Extensive experiments show that the MDTS significantly increases the overall reward compared with least- connection scheduling and particle swarm optimization algorithm from −16.71, −0.67 to 2.93, respectively. [ABSTRACT FROM AUTHOR]

Deep learning (DL) has recently aroused substantial concern due to its successful implementations in many fields. Currently, there are few studies on the applications of DL in the automatic modulation classification (AMC), which plays a critical role in non-cooperation communications. Besides, most previous work ignores the feature interaction, and only considers spatial or temporal attributes of signals. Combining the advantages of the convolutional neural network (CNN), and the long short-term memory (LSTM), this paper addresses the AMC using CNN-LSTM based dual-stream structure, which efficiently explores the feature interaction, and the spatial-temporal properties of raw complex temporal signals. Specifically, a preprocessing step is first implemented to convert signals into the temporal in-phase/quadrature (I/Q) format, and the amplitude/phase (A/P) representation, which facilitates the acquirement of more effective features for classification. To extract features from each signal pattern, each stream is composed of CNN, and LSTM (denoted as CNN-LSTM). Most importantly, the features learned from two streams interact in pairs, which increases the diversity of features and thereby further improves the performance. Finally, some comparisons with previous work are performed. The experimental results not only demonstrate the superior performance of the proposed method compared with the existing state-of-the-art methods, but also reveal the potential of DL-based approaches for AMC. [ABSTRACT FROM AUTHOR]

Rear-end collision prediction has gained an increasing attention for safety improvement in smart cities. It is urgent to design efficient warning strategies for rear-end collisions which is one of the main causes of traffic accidents. The existing researches have been conducted to predict the collisions. Learning-based methods are proposed to solve this complicated issue, which the traditional methods are difficult to solve. However, due to some limitations in terms of the feature extraction and prediction performance, back-propagation learning methods are facing challenge. In this paper, we proposed a novel Rear-end Collision Prediction Mechanism with deep learning method (RCPM), in which a convolutional neural network model is established. In RCPM, the dataset is smoothed and expanded based on genetic theory to alleviate the class imbalance problem. The preprocessed dataset is divided into training and testing sets as the input to train our convolutional neural network model. The experimental results show that compared with the Honda, Berkeley and multi-layer perception neural-network-based algorithms, RCPM effectively improves performance to predict rear-end collisions. [ABSTRACT FROM AUTHOR]

Chaos-based communications can be applied to high-speed vehicular information transmissions thanks to the anti-jamming and anti-interference capabilities of chaotic transmissions. In traditional practical chaotic systems, reference chaotic signals are required to be delivered to remove complex chaotic synchronization circuits. However, the direct transmission of reference signals will degrade the security performances, while interferences and noises imposed on the reference signals due to imperfect channel conditions will deteriorate the reliability performances. In order to enhance the reliability and security performances over vehicular channels such as the railway channel and the channels undergoing fast fadings, in this paper, we propose a deep learning (DL) aided intelligent OFDM-DCSK transceiver. In this design, no reference chaotic signals are delivered, and we propose to utilize the time-delay neural network (TDNN) to learn the chaotic maps, followed by the long short-term memory (LSTM) units to extract and exploit the correlations between chaotic modulated signals, and multiple fully connected layers (FCLs) to estimate the user bit data. With the aid of the constructed deep neural network (DNN), after the offline neural network training, the receiver can recover the transmitted information with lower bit error rate (BER) and enhance security performances. Theoretical performance is then analyzed for the proposed intelligent transceiver. Simulation results validate the proposed design, and demonstrate that the intelligent DL-based OFDM-DCSK system can achieve better BER and security performances over fast fading and railway channels compared with the benchmark systems. [ABSTRACT FROM AUTHOR]

Dynamic topology, fast-changing channels and the time sensitivity of safety-related services present challenges to the status quo of resource allocation for cellular-underlaying vehicle-to-vehicle (V2V) communications. In this paper, we investigate a novel federated multi-agent deep reinforcement learning (FedMARL) approach for the decentralized joint optimization of channel selection and power control for V2V communication. The approach takes advantage of both deep reinforcement learning (DRL) and federated learning (FL), satisfying the reliability and delay requirements of V2V communication while maximizing the transmit rates of cellular links. Specifically, we elaborately construct individual V2V agent implement by the dueling double deep Q-network (D3QN), and design the reward function to train V2V agents collaboratively. As a result, each agent individually optimizes channel selection and power level based on its local observations, including the instantaneous channel state information (CSI) of corresponding V2V link, the instantaneous co-channel interference from the cellular link, the previous channels selections of nearby V2V pairs, and the queue backlog at the V2V transmitter. Another important aspect is that we incorporate FL to alleviate the training instability problem induced by cooperative multi-agent environment. The local DRL models of different V2V agents are federated periodically, addressing the limitations of partial observability on the entire network status for individual agent, and accelerating the training process of multi-agent learning. Validated via simulations, the proposed FedMARL scheme shows superiority to the baselines in terms of the cellular sum-rate and the V2V packet delivery rate. [ABSTRACT FROM AUTHOR]

In multi-cell massive multiple-input multiple-output (MIMO) systems, channel estimation is deteriorated by pilot contamination and the effects of pilot contamination become more severe due to hardware impairments. In this paper, we propose a joint pilot design and channel estimation based on deep residual learning in order to mitigate the effects of pilot contamination under the consideration of hardware impairments. We first investigate a conventional linear minimum mean square error (LMMSE) based channel estimator to suppress the interference caused by pilot contamination. After that, a deep learning based pilot design is proposed to minimize the mean square error (MSE) of LMMSE channel estimation, which is utilized to the joint pilot design and channel estimator for transfer learning approach. For the channel estimator, we use a deep residual learning which extracts the features of interference caused by pilot contamination and eliminates them to estimate the channel information. Simulation results demonstrate that the proposed joint pilot design and channel estimation method can effectively reduce the effect of pilot contamination as well as outperforms the conventional approach in multi-cell massive MIMO scenarios. Moreover, the joint pilot design and channel estimation method using transfer learning further enhances the estimation performance when the prior knowledge of pilot contamination cannot be exploited. [ABSTRACT FROM AUTHOR]

In recent years, the growing development of Connected Autonomous Vehicles (CAV), Intelligent Transport Systems (ITS), and 5G communication networks have led to the advent of Autonomous Intersection Management (AIM) systems. AIMs present a new paradigm for CAV control in future cities, taking control of CAVs in scenarios where cooperation is necessary and allowing safe and efficient traffic flows, eliminating traffic signals. So far, the development of AIM algorithms has been based on basic control algorithms, without the ability to adapt or keep learning new situations. To solve this, in this paper we present a new advanced AIM approach based on end-to-end Multi-Agent Deep Reinforcement Learning (MADRL) and trained using Curriculum through Self-Play, called advanced Reinforced AIM (adv.RAIM). adv.RAIM enables the control of CAVs at intersections in a collaborative way, autonomously learning complex real-life traffic dynamics. In addition, adv.RAIM provides a new way to build smarter AIMs capable of proactively controlling CAVs in other highly complex scenarios. Results show remarkable improvements when compared to traffic light control techniques (reducing travel time by 59% or reducing time lost due to congestion by 95%), as well as outperforming other recently proposed AIMs (reducing waiting time by 56%), highlighting the advantages of using MADRL. [ABSTRACT FROM AUTHOR]

For vehicle-to-network communications, handover (HO) management enables vehicles to maintain the connection with the network while transiting through coverage areas of different base stations (BSs). However, the high mobility of vehicles means shorter connection periods with each BS that leads to frequent HOs, hence raises the necessity for optimal HO decision making for high quality infotainment services. Machine learning is capable of capturing underlying pattern via data driven methods to find optimal solutions to complex problems, and much learning-based HO optimization research has been conducted focusing on specific network setups. However, attention still needs to be paid to the actual deployment aspect and standardized datasets or simulation environments for evaluation. This paper proposes a deep reinforcement learning-based HO algorithm using the input parameters that are configurable in the existing measurement report of cellular networks. The performance of the proposed algorithm is evaluated using the well-known network simulator ns-3 with its official LTE module. A realistic network setup in the city center of Glasgow (U.K.) is configured with vehicle trajectories generated by the routes mobility model using Google Maps Directions API. Evaluation results reveal that the proposed algorithm significantly outperforms the A3 RSRP baseline with an average of 25.72% packet loss reduction per HO, suggesting significant improvement in quality of service of phone call and video streaming, etc. The proposed algorithm also has a small implementation cost compared to some state-of-the-art and should be deployed by a software update to a local BS controller. [ABSTRACT FROM AUTHOR]

To deal with the deep learning-based automatic modulation classification (AMC) in the scenario that the training dataset are distributed over a network without gathering the data at a centralized location, the decentralized learning-based AMC (DecentAMC) had been presented. However, there exists frequent model parameter uploading and downloading in DecentAMC method, which cause high communication overhead. In this paper, an innovative learning framework are proposed for AMC (named DeEnAMC), in which the framework is realized by utilizing the combination of decentralized learning and ensemble learning. Our results show that the proposed DeEnAMC reduces communication overhead while keeping a similar classification performance to DecentAMC. [ABSTRACT FROM AUTHOR]

From adapting to the propagation environment to reconstructing the propagation environment, reconfigurable intelligent surface (RIS) changes the design paradigm of wireless communications. To reconstruct the propagation environment, joint beamforming of RIS and multi-input multi-output (MIMO) is crucial. Unfortunately, due to the coupling effect of the active beamforming of MIMO and passive beamforming of RIS, it is difficult to find the optimal solution to the joint beamforming problem, so a serious performance loss will be caused. In this paper, inspired by the end-to-end (E2E) learning of communication system, we propose the E2E learning based RIS-aided communication system to mitigate the performance loss via deep learning techniques. The key idea is to simultaneously optimize the signal processing functions at base station (BS), RIS, and user, including active beamforming for BS and user, passive beamforming for RIS. This way is able to avoid the performance loss caused by alternately optimizing each function of the RIS-aided system. Specifically, we firstly utilize a deep neural network (DNN) to realize the modulation and beamforming for BS and utilize another DNN to realize the demodulation and combining for user. Then, the RIS passive beamforming is also represented by trainable parameters, which could be simultaneously optimized with the DNNs at BS and user. Simulation results show that the proposed E2E learning based RIS-aided communication system could achieve the better bit error rate (BER) performance than traditional RIS-aided communication systems. [ABSTRACT FROM AUTHOR]

Recently, model-free power control approaches have been developed to achieve the near-optimal performance of cell-free (CF) massive multiple-input multiple-output (MIMO) with affordable computational complexity. In particular, deep reinforcement learning (DRL) is one of such promising techniques for realizing effective power control. In this paper, we propose a model-free method adopting the deep deterministic policy gradient algorithm (DDPG) with feedforward neural networks (NNs) to solve the downlink max-min power control problem in CF massive MIMO systems. Our result shows that compared with the conventional convex optimization algorithm, the proposed DDPG method can effectively strike a performance-complexity trade-off obtaining 1,000 times faster implementation speed and approximately the same achievable user rate as the optimal solution produced by conventional numerical convex optimization solvers, thereby offering effective power control implementations for large-scale systems. Finally, we extend the DDPG algorithm to both the max-sum and the max-product power control problems, while achieving better performance than that achieved by the conventional deep learning algorithm. [ABSTRACT FROM AUTHOR]

Intelligent traffic light control is one of the modern approaches to solve traffic congestion, where reinforcement learning is a widely used method. Conventionally, reinforcement learning is used to determine whether to change the current phase (or choose a traffic phase) after each small interval. One major drawback of these approaches is that it makes the current traffic light phase duration uncertain before the current phase terminates. Directly determining the duration of the traffic light phase can effectively avoid this shortcoming. An adaptive traffic light timing system is proposed in this paper which can directly control the phase duration. In the proposed system, the Q-learning algorithm is employed and the action space is defined as all possible phase durations. In addition, the reward function is redefined to guide the agent to balance more traffic metrics, and the state is redefined to reduce the state space. Finally, the proposed system is evaluated by equal, unequal, and complex traffic scenarios. Results show that the proposed system has a better performance compared with other methods in controlling traffic lights, even on complex traffic scenarios. [ABSTRACT FROM AUTHOR]

The vehicular edge computing is considered an enabling technology for intelligent and connected vehicles since the optimization of communication and computing on edge has a significant impact on driving safety and efficiency. In this paper, with the road traffic assignment to “proactively” reshape the spatiotemporal distribution of resource demands, we investigate the joint service migration and mobility optimization problem for vehicular edge computing. The goal is to meet the service delay requirements of vehicular edge computing with minimum migration cost and travel time. As service migration and mobility optimization are coupled, the joint scheduling problem suffers from the curse of dimensionality, which cannot be solved in real time by centralized algorithms. To this end, a multi-agent deep reinforcement learning (MADRL) algorithm is proposed to maximize the composite utility of communication, computing, and route planning in a distributed way. In the MADRL algorithm, a two-branch convolution based deep Q-network is constructed to coordinate migration action and routing action. Extensive experimental results show that the proposed algorithm is scalable and substantially reduces service delay, migration cost and travel time as compared with the existing baselines. [ABSTRACT FROM AUTHOR]

Wireless virtual reality (VR) is predicted to become a killer application in 5G and beyond, which provides an immersive experience and revolutionizes the way people communicate. It is well-known that rendering is the key performance bottleneck in wireless VR systems, especially for VR games. However, real-time rendering and data correlation are ignored by most researchers. In this paper, we propose an adaptive VR framework that enables high-quality wireless VR in future mmWave-enabled wireless networks with mobile edge computing (MEC), where real-time VR rendering tasks can be offloaded to MEC servers adaptively and the caching capability of MEC servers enables further performance improvement. First, we formulate the addressed problem to maximize the quality of experience (QoE) of the users, where association, adaptive offloading mode selection, and caching policy are jointly optimized. Considering the high complexity of the addressed problem, we then propose a distributed learning approach consisting of an offline training phase and an online running phase, which maintains scalability and adaptation capability. The offline phase is based on deep reinforcement learning (DRL) while the latter utilizes game theory. At last, simulation results show the superiority of the proposed algorithm over the other baseline algorithms in terms of QoE utility values, latency, and convergence time. [ABSTRACT FROM AUTHOR]

With the increasing number of vehicles and traffic accidents, driving safety has become an important factor that affects human daily life. As the primary cause of driving accidents, driving fatigue could be prevented by a sensing and alarm system built in the vehicle. In this paper, we propose an effective, low-cost driving fatigue detection system to sense driver's nodding movements using commodity RFID. The system measures the phase difference between two RFID tags attached to the back of a hat worn by the driver. To accurately extract nodding features, we propose an effective approach to mitigate the environment noise, the interference caused by surrounding movements, and the cumulative error caused by the frequency hopping offset in FCC-compliant RFID systems. A long short-term memory (LSTM) autoencoder is utilized to detect nodding movements using calibrated data. The highly accurate detection performance of the proposed system is validated by extensive experiments in various real driving scenarios. [ABSTRACT FROM AUTHOR]

With the popularization of Wi-Fi signals and urgent demands for passive human action recognition, wireless sensing based activity recognition has been a hot topic in recent years. Most existing researches rely on traditional hand-crafted features, and limited work focuses on how to effectively extract deep features with spatial-temporal information. In this paper, we develop an accurate device-free action recognition system utilizing a Commodity Off-The-Shelf (COTS) router and propose a novel deep learning framework (termed two-stream network) mining spatial-temporal cues in channel state information (CSI). Specifically, an entire action sample is segmented into a series of coherent sub-activity clips. Then we try to capture the complementary features on appearance from the original CSI clips and motion between CSI frames. The spatial and temporal information are processed with separate networks which are then integrated for the final recognition task. The extensive experiments are implemented on the data collected from two indoor environments, respectively reaching 97.6% and 96.9% recognition accuracies. [ABSTRACT FROM AUTHOR]