Search

Radio signal recognition has a wide application in future communication systems and the vehicular communication, whose core is the extraction of signal features such as electromagnetic fingerprints. With the rapid development of artificial intelligence technology, deep learning has made amazing breakthroughs in image recognition, speech recognition and other fields. Deep learning is applied to electromagnetic fingerprint extraction in this paper. Firstly, thousands of the downlink aircraft communications addressing and reporting system (ACARS) signals used for communication between civil aircraft and airport tower are collected and generated. Then a pre-transformation network suitable for electromagnetic signals is constructed to convert one-dimensional signals into two-dimensional feature maps, and afterwards the feature maps are input into the convolutional neural network (CNN) for classification. By adopting the attention modules, the classification results were improved by a few percentage points over the baseline with a little cost. The method proposed in this paper achieves an accuracy rate of 94.1% and can obtain the aircraft type in a shorter time than traditional method. Moreover, the robustness of the proposed model in response to additive Gaussian white noise (AWGN) and phase deviation is studied and tested. [ABSTRACT FROM AUTHOR]

The explosive growth of Internet-of-Things (IoT) applications such as smart cities and Industry 4.0 have led to drastic increase in demand for wireless bandwidth, hence motivating the rapid development of new techniques for enhancing spectrum utilization needed by new generation wireless communication technologies. Among others, dynamic spectrum access (DSA) is one of the most widely accepted approaches. In this paper, as an enhancement of existing works, we take into consideration of inter-node collaborations in a dynamic spectrum environment. Typically, in such distributed circumstances, intelligent dynamic spectrum access almost invariably relies on self-learning to achieve dynamic spectrum access improvement. Whereas, this paper proposes a DSA scheme based on deep reinforcement learning to enhance spectrum and access efficiency. Unlike traditional Q-learning-based DSA, we introduce the following to enhance the spectrum efficiency in dynamic IoT spectrum environments. First, deep double Q-learning is adopted to perform local self-spectrum-learning for IoT terminals in order to achieve better dynamic access accuracy. Second, to accelerate learning convergence, federated learning (FL) in edge nodes is used to improve the self-learning. Third, multiple secondary users, who do not interfere with each other and have similar operation condition, are clustered for federated learning to enhance the efficiency of deep reinforcement learning. Comparing with the traditional distributed DSA with deep learning, the proposed scheme has faster access convergence speed due to the characteristic of global optimization for federated learning. Based on this, a framework of federated deep reinforcement learning (FDRL) for DSA is proposed. Furthermore, this scheme preserves privacy of IoT users in that FDRL only requires model parameters to be uploaded to edge servers. Simulations are performed to show the effectiveness of theproposed FDRL-based DSA framework. [ABSTRACT FROM AUTHOR]

This paper proposes a deep reinforcement learning-based video streaming scheme for mobility-aware vehicular networks, e.g., vehicles on the highway. We consider infrastructure-assisted and mmWave-based scenarios in which the macro base station (MBS) cannot directly provide the streaming service to vehicles due to the short range of mmWave beams so that small mmWave base stations (mBSs) along the road deliver the desired videos to users. Motivated by the fact that a video stream consists of sequential chunks and smooth streaming has to be provided while users pass through multiple mBSs at high speed, the MBS proactively pushes some video chunks of the desired contents to mBSs. This is done to support vehicles that are currently covered and/or will be by each mBS. We formulate the dynamic video delivery scheme that adaptively determines 1) which content, 2) what quality and 3) how many chunks to be proactively delivered from the MBS to mBSs using Markov decision process (MDP). Since it is difficult for the MBS to track all the channel conditions and the network states have extensive dimensions, we adopt the deep deterministic policy gradient (DDPG) algorithm for the DRL-based video delivery scheme. This paper finally shows that the DRL agentlearns a streaming policy that pursues high average quality while limiting packet drops, avoiding playback stalls, reducing quality fluctuations and saving backhaul usage. [ABSTRACT FROM AUTHOR]

This paper studies the allocation of shared resources between vehicle-to-infrastructure (V2I) and vehicle-to-vehicle (V2V) links in vehicle-to-everything (V2X) communications. In existing algorithms, dynamic vehicular environments and quantization of continuous power become the bottlenecks for providing an effective and timely resource allocation policy. In this paper, we develop two algorithms to deal with these difficulties. First, we propose a deep reinforcement learning (DRL)-based resource allocation algorithm to improve the performance of both V2I and V2V links. Specifically, the algorithm uses deep Q-network (DQN) to solve the sub-band assignment and deep deterministic policy-gradient (DDPG) to solve the continuous power allocation problem. Second, we propose a meta-based DRL algorithm to enhance the fast adaptability of the resource allocation policy in the dynamic environment. Numerical results demonstrate that the proposed DRL-based algorithm can significantly improve the performance compared to the DQN-based algorithm that quantizes continuous power. In addition, the proposed meta-based DRL algorithm can achieve the required fast adaptation in the new environment with limited experiences. [ABSTRACT FROM AUTHOR]

This paper proposes a cloud-assisted joint charging scheduling and energy management framework for unmanned aerial vehicle (UAV) networks. For charging the UAVs those are extremely power hungry, charging towers are considered for plug-and-play charging during run-time operations. The charging towers should be cost-effective, thus it is equipped with photovoltaic power generation and energy storage systems functionalities. Furthermore, the towers should be cooperative for more cost-effectiveness by intelligent energy sharing. Based on the needs and setting, this paper proposes 1) charging scheduling between UAVs and towers and 2) cooperative energy managements among towers. For charging scheduling, the UAVs and towers should be scheduled for maximizing charging energy amounts and the scheduled pairs should determine charging energy allocation amounts. Here, two decisions are correlated, i.e., it is a non-convex problem. We re-formulate the non-convex to convex for guaranteeing optimal solutions. Lastly, the cooperative energy sharing among towers is designed and implemented with multi-agent deep reinforcement learning and then intelligent energy sharing can be realized. We can observe that the two methods are related and it should be managed, coordinated, and harmonized by a centralized orchestration manager under the consideration of fairness, energy-efficiency, and cost-effectiveness. Our data-intensive performance evaluation verifies that our proposed framework achieves desired performance. [ABSTRACT FROM AUTHOR]

This paper tackles the joint user association (UA) and power control (PC) problem for dynamic heterogeneous networks (HetNets), where the number of active users varies over time. Since the joint UA and PC problem is a mixed-integer problem, it is challenging to develop a joint UA and PC algorithm that closely achieves the optimal proportional fairness (PF) performance while satisfying the tight computation time budget of recent 3GPP standards. Although previous studies have reduced the computation time on the UA and PC by employing a fully-connected-neural network (FCN), the FCN architecture is not generally applicable to dynamic HetNets because of a varying number of users. In pursuit of scalable neural network design, we propose an unsupervised learning-based UA and PC algorithm using a recurrent neural network (RNN). Notably, in the RNN-based UA, we relax the combinatorial UA variable into a low-dimensional continuous variable, the length of which is invariant to the number of users. The optimality loss in the variable relaxation is upper-bounded by the negligibly small duality gap. Through extensive experiments, we show that the proposed scheme has a 10% higher PF performance than conventional optimization-based methods with the same number of flops and closely achieves the optimal PF performance for various numbers of users. [ABSTRACT FROM AUTHOR]

The flow fluctuations in the highly dynamic Internet of Vehicles (IoV) make the IoV difficult to provide reliable and scalable wireless network services for the emerging applications in the 5 G and beyond era. The software-defined networks (SDN) could feasibly manage and optimize the network according to the network load. Controller placement is a critical problem in SDN to achieve its robustness and flexibility with the changes of network status. Motivated by the advantages of SDN and Mobile-edge computing (MEC), this paper aims at enhancing the performance of IoV with the assistance of these two. Specifically, we consider a three-layer hierarchical SDN control plane for the IoV, where the SDN controllers are placed at the edge of networks. Under this framework, we investigate a multi-objective optimization problem on controller placement problem including delay, load balancing, and path reliability. To efficiently solve the formulated NP-hard problems, we develop an algorithm based on multi-agent deep Q-learning networks (MADQN) because of its advantages for large-scale combinatorial optimization. At last, we use multi-process technology to accelerate the operation of the algorithm, so as to complete the algorithm iteration faster. Numerical results show that the proposed methods achieve better performances than three baselines. [ABSTRACT FROM AUTHOR]

The efficiency of link adaptation in wireless communications relies greatly on the accuracy of channel knowledge and transmission mode selection. In this paper, a novel deep learning based link adaptation framework is proposed for the orthogonal frequency-division multiplexing (OFDM) systems with compressed-sensing-assisted index modulation, termed as OFDM-CSIM, communicating over millimeter-wave (mmWave) channels. To achieve link adaptation, a novel multi-layer sparse Bayesian learning (SBL) algorithm is proposed for accurately and instantaneously providing the required channel state information. Meanwhile, a deep neural networks (DNN)-assisted adaptive modulation algorithm is proposed to choose the best possible transmission mode to maximize the achievable throughput. Simulation results show that the proposed multi-layer SBL algorithm enables more accurate channel estimation than the conventional techniques. The DNN-based adaptive modulator is capable of achieving a higher throughput than the learning-assisted solution based on the $k$ nearest neighbor ($k$ -NN) algorithm, and also the classic average signal-to-noise ratio (SNR)-based solutions. Moreover, analysis shows that both the multi-layer SBL algorithm and the DNN-assisted adaptive modulator achieve better performance than their respective conventional counterparts while at a significantly lower computational complexity cost. [ABSTRACT FROM AUTHOR]

Extracting channel state information (CSI) from WiFi signals is of proved high-effectiveness in locating human locations in a device-free manner. However, existing localization/positioning systems are mainly trained and deployed in a fixed environment, and thus they are likely to suffer from substantial performance declines when immigrating to new environments. In this paper, we address the fundamental problem of WiFi-based cross-environment indoor localization using a semi-supervised approach, in which we only have access to the annotations of the source environment while the data in the target environments are un-annotated. This problem is of high practical values in enabling a well-trained system to be scalable to new environments without tedious human annotations. To this end, a deep neural forest is introduced which unifies the ensemble learning with the representation learning functionalities from deep neural networks in an end-to-end trainable fashion. On top of that, an adversarial training strategy is further employed to learn environment-invariant feature representations for facilitating more robust localization. Extensive experiments on real-world datasets demonstrate the superiority of the proposed methods over state-of-the-art baselines. Compared with the best-performing baseline, our model excels with an average 12.7% relative improvement on all six evaluation settings. [ABSTRACT FROM AUTHOR]

Chaos-based communications can be applied to high-speed vehicular information transmissions thanks to the anti-jamming and anti-interference capabilities of chaotic transmissions. In traditional practical chaotic systems, reference chaotic signals are required to be delivered to remove complex chaotic synchronization circuits. However, the direct transmission of reference signals will degrade the security performances, while interferences and noises imposed on the reference signals due to imperfect channel conditions will deteriorate the reliability performances. In order to enhance the reliability and security performances over vehicular channels such as the railway channel and the channels undergoing fast fadings, in this paper, we propose a deep learning (DL) aided intelligent OFDM-DCSK transceiver. In this design, no reference chaotic signals are delivered, and we propose to utilize the time-delay neural network (TDNN) to learn the chaotic maps, followed by the long short-term memory (LSTM) units to extract and exploit the correlations between chaotic modulated signals, and multiple fully connected layers (FCLs) to estimate the user bit data. With the aid of the constructed deep neural network (DNN), after the offline neural network training, the receiver can recover the transmitted information with lower bit error rate (BER) and enhance security performances. Theoretical performance is then analyzed for the proposed intelligent transceiver. Simulation results validate the proposed design, and demonstrate that the intelligent DL-based OFDM-DCSK system can achieve better BER and security performances over fast fading and railway channels compared with the benchmark systems. [ABSTRACT FROM AUTHOR]

In this paper, we design an Anomaly Detection (AD) framework for mobile data traffic, capable of identifying different types of anomalous events generated by flash crowds in metropolitan areas. We state the problem using a semi-supervised approach and exploit the great performance of different Recurrent Neural Network (RNN) models to learn the temporal context of input sequences. Our proposal processes real traffic traces from the unencrypted LTE Physical Downlink Control Channel (PDCCH) of an operative network, gathered during an extensive measurement campaign in two major cities in Spain. The AD framework is designed to perform: i) a-posteriori analysis to understand users’ behavior and urban environment variations; ii) real-time analysis to automatically and on-the-fly alert urban anomalies; and iii) estimation of the duration of the periods identified as anomalous. Numerical results show the higher performance of our AD framework compared to classic AD algorithms and confirm that the proposed framework predicts anomalous behaviours with high accuracy and regardless of their cause. [ABSTRACT FROM AUTHOR]

Dynamic topology, fast-changing channels and the time sensitivity of safety-related services present challenges to the status quo of resource allocation for cellular-underlaying vehicle-to-vehicle (V2V) communications. In this paper, we investigate a novel federated multi-agent deep reinforcement learning (FedMARL) approach for the decentralized joint optimization of channel selection and power control for V2V communication. The approach takes advantage of both deep reinforcement learning (DRL) and federated learning (FL), satisfying the reliability and delay requirements of V2V communication while maximizing the transmit rates of cellular links. Specifically, we elaborately construct individual V2V agent implement by the dueling double deep Q-network (D3QN), and design the reward function to train V2V agents collaboratively. As a result, each agent individually optimizes channel selection and power level based on its local observations, including the instantaneous channel state information (CSI) of corresponding V2V link, the instantaneous co-channel interference from the cellular link, the previous channels selections of nearby V2V pairs, and the queue backlog at the V2V transmitter. Another important aspect is that we incorporate FL to alleviate the training instability problem induced by cooperative multi-agent environment. The local DRL models of different V2V agents are federated periodically, addressing the limitations of partial observability on the entire network status for individual agent, and accelerating the training process of multi-agent learning. Validated via simulations, the proposed FedMARL scheme shows superiority to the baselines in terms of the cellular sum-rate and the V2V packet delivery rate. [ABSTRACT FROM AUTHOR]

In recent years, the growing development of Connected Autonomous Vehicles (CAV), Intelligent Transport Systems (ITS), and 5G communication networks have led to the advent of Autonomous Intersection Management (AIM) systems. AIMs present a new paradigm for CAV control in future cities, taking control of CAVs in scenarios where cooperation is necessary and allowing safe and efficient traffic flows, eliminating traffic signals. So far, the development of AIM algorithms has been based on basic control algorithms, without the ability to adapt or keep learning new situations. To solve this, in this paper we present a new advanced AIM approach based on end-to-end Multi-Agent Deep Reinforcement Learning (MADRL) and trained using Curriculum through Self-Play, called advanced Reinforced AIM (adv.RAIM). adv.RAIM enables the control of CAVs at intersections in a collaborative way, autonomously learning complex real-life traffic dynamics. In addition, adv.RAIM provides a new way to build smarter AIMs capable of proactively controlling CAVs in other highly complex scenarios. Results show remarkable improvements when compared to traffic light control techniques (reducing travel time by 59% or reducing time lost due to congestion by 95%), as well as outperforming other recently proposed AIMs (reducing waiting time by 56%), highlighting the advantages of using MADRL. [ABSTRACT FROM AUTHOR]

In multi-cell massive multiple-input multiple-output (MIMO) systems, channel estimation is deteriorated by pilot contamination and the effects of pilot contamination become more severe due to hardware impairments. In this paper, we propose a joint pilot design and channel estimation based on deep residual learning in order to mitigate the effects of pilot contamination under the consideration of hardware impairments. We first investigate a conventional linear minimum mean square error (LMMSE) based channel estimator to suppress the interference caused by pilot contamination. After that, a deep learning based pilot design is proposed to minimize the mean square error (MSE) of LMMSE channel estimation, which is utilized to the joint pilot design and channel estimator for transfer learning approach. For the channel estimator, we use a deep residual learning which extracts the features of interference caused by pilot contamination and eliminates them to estimate the channel information. Simulation results demonstrate that the proposed joint pilot design and channel estimation method can effectively reduce the effect of pilot contamination as well as outperforms the conventional approach in multi-cell massive MIMO scenarios. Moreover, the joint pilot design and channel estimation method using transfer learning further enhances the estimation performance when the prior knowledge of pilot contamination cannot be exploited. [ABSTRACT FROM AUTHOR]

For vehicle-to-network communications, handover (HO) management enables vehicles to maintain the connection with the network while transiting through coverage areas of different base stations (BSs). However, the high mobility of vehicles means shorter connection periods with each BS that leads to frequent HOs, hence raises the necessity for optimal HO decision making for high quality infotainment services. Machine learning is capable of capturing underlying pattern via data driven methods to find optimal solutions to complex problems, and much learning-based HO optimization research has been conducted focusing on specific network setups. However, attention still needs to be paid to the actual deployment aspect and standardized datasets or simulation environments for evaluation. This paper proposes a deep reinforcement learning-based HO algorithm using the input parameters that are configurable in the existing measurement report of cellular networks. The performance of the proposed algorithm is evaluated using the well-known network simulator ns-3 with its official LTE module. A realistic network setup in the city center of Glasgow (U.K.) is configured with vehicle trajectories generated by the routes mobility model using Google Maps Directions API. Evaluation results reveal that the proposed algorithm significantly outperforms the A3 RSRP baseline with an average of 25.72% packet loss reduction per HO, suggesting significant improvement in quality of service of phone call and video streaming, etc. The proposed algorithm also has a small implementation cost compared to some state-of-the-art and should be deployed by a software update to a local BS controller. [ABSTRACT FROM AUTHOR]

Computation offloading is an effective method in mobile edge computing (MEC) to relieve user equipment (UE) from the limited computation resource and battery capacity. Meanwhile, simultaneous wireless information and power transmission (SWIPT) can be applied to MEC to extend the operating time of the equipment. However, in multi-user network environment, diverse computation task requirements and changeable network channel states make it challenging to obtain offloading strategy timely and accurately. To address the issue, we propose an intelligent computation offloading scheme (iCOS) based on enhanced priority deep deterministic policy gradient (EPDDPG) algorithm to minimize the energy consumption of all the UEs by jointly optimizing the offloading decision, the central processing unit (CPU) frequency and the power split ratio in a dynamic SWIPT-MEC network. In particular, we improve the traditional fully-connected network structure to obtain both discrete and continuous action outputs, and accelerate neural network parameter updates by using prioritized experience tuples. Furthermore, we use dynamic voltage and frequency scaling (DVFS) technology to dynamically adjust the CPU frequency of local computing, and employ SWIPT technology to balance the charging and communication according to the obtained strategy. Simulation results show that the algorithm proposed in this paper can effectively reduce the energy cost of UEs, and complete more computation tasks within the delay limit. [ABSTRACT FROM AUTHOR]

To deal with the deep learning-based automatic modulation classification (AMC) in the scenario that the training dataset are distributed over a network without gathering the data at a centralized location, the decentralized learning-based AMC (DecentAMC) had been presented. However, there exists frequent model parameter uploading and downloading in DecentAMC method, which cause high communication overhead. In this paper, an innovative learning framework are proposed for AMC (named DeEnAMC), in which the framework is realized by utilizing the combination of decentralized learning and ensemble learning. Our results show that the proposed DeEnAMC reduces communication overhead while keeping a similar classification performance to DecentAMC. [ABSTRACT FROM AUTHOR]

From adapting to the propagation environment to reconstructing the propagation environment, reconfigurable intelligent surface (RIS) changes the design paradigm of wireless communications. To reconstruct the propagation environment, joint beamforming of RIS and multi-input multi-output (MIMO) is crucial. Unfortunately, due to the coupling effect of the active beamforming of MIMO and passive beamforming of RIS, it is difficult to find the optimal solution to the joint beamforming problem, so a serious performance loss will be caused. In this paper, inspired by the end-to-end (E2E) learning of communication system, we propose the E2E learning based RIS-aided communication system to mitigate the performance loss via deep learning techniques. The key idea is to simultaneously optimize the signal processing functions at base station (BS), RIS, and user, including active beamforming for BS and user, passive beamforming for RIS. This way is able to avoid the performance loss caused by alternately optimizing each function of the RIS-aided system. Specifically, we firstly utilize a deep neural network (DNN) to realize the modulation and beamforming for BS and utilize another DNN to realize the demodulation and combining for user. Then, the RIS passive beamforming is also represented by trainable parameters, which could be simultaneously optimized with the DNNs at BS and user. Simulation results show that the proposed E2E learning based RIS-aided communication system could achieve the better bit error rate (BER) performance than traditional RIS-aided communication systems. [ABSTRACT FROM AUTHOR]

Recently, model-free power control approaches have been developed to achieve the near-optimal performance of cell-free (CF) massive multiple-input multiple-output (MIMO) with affordable computational complexity. In particular, deep reinforcement learning (DRL) is one of such promising techniques for realizing effective power control. In this paper, we propose a model-free method adopting the deep deterministic policy gradient algorithm (DDPG) with feedforward neural networks (NNs) to solve the downlink max-min power control problem in CF massive MIMO systems. Our result shows that compared with the conventional convex optimization algorithm, the proposed DDPG method can effectively strike a performance-complexity trade-off obtaining 1,000 times faster implementation speed and approximately the same achievable user rate as the optimal solution produced by conventional numerical convex optimization solvers, thereby offering effective power control implementations for large-scale systems. Finally, we extend the DDPG algorithm to both the max-sum and the max-product power control problems, while achieving better performance than that achieved by the conventional deep learning algorithm. [ABSTRACT FROM AUTHOR]

Intelligent traffic light control is one of the modern approaches to solve traffic congestion, where reinforcement learning is a widely used method. Conventionally, reinforcement learning is used to determine whether to change the current phase (or choose a traffic phase) after each small interval. One major drawback of these approaches is that it makes the current traffic light phase duration uncertain before the current phase terminates. Directly determining the duration of the traffic light phase can effectively avoid this shortcoming. An adaptive traffic light timing system is proposed in this paper which can directly control the phase duration. In the proposed system, the Q-learning algorithm is employed and the action space is defined as all possible phase durations. In addition, the reward function is redefined to guide the agent to balance more traffic metrics, and the state is redefined to reduce the state space. Finally, the proposed system is evaluated by equal, unequal, and complex traffic scenarios. Results show that the proposed system has a better performance compared with other methods in controlling traffic lights, even on complex traffic scenarios. [ABSTRACT FROM AUTHOR]

Unmanned aerial vehicles (UAVs) control information delivery is a critical communication with stringent requirements in terms of reliability and latency. In this context, link adaptation plays an essential role in the fulfillment of the required performance in terms of decode error probability and delay. Link adaptation is usually based on channel quality indicator (CQI) feedback information from the user equipment that should represent the current state of the channel. However, measurement, scheduling and processing delays introduce a CQI aging effect, that is a mismatch between the current channel state and its CQI representation. Using outdated CQI values may lead to the selection of a wrong modulation and coding scheme, with a detrimental effect on performance. This is particularly relevant in ultra reliable and low latency communications (URLLC), where the control of the reliability can be negatively impacted, and it is more evident when the channel is fast varying as the case of UAVs. This paper analyzes the effects of CQI aging on URLLCs, considering transmissions under the finite blocklength regime, that characterizes such communications type. A deep learning approach is investigated to predict the next CQI from the knowledge of past reports, and performance in terms of decode error probability and throughput is given. The results show the benefit of CQI proposed prediction mechanism also in comparison with previously proposed methods. [ABSTRACT FROM AUTHOR]

Unmanned aerial vehicles (UAVs) play a critical role for 5 G/B5G communications. In this paper, we investigate the physical layer security issue for ground transmissions with the help of a UAV. Specifically, we consider the scenario without direct transmissions and employ the UAV as a mobile relay to amplify and forward the confidential information. Meanwhile, the receiver transmits cooperative jamming attempting to disrupt eavesdropping. We investigate the UAV deployment and receiver jamming strategy jointly for the optimized secrecy rate, while considering the channel uncertainties. In particular, for the case of complete channel state information (CSI), the jamming power is obtained by bi-section search, and we propose a data-trained deep neural network (DNN) to approximate the optimal UAV deployment. Meanwhile, for the case with partial CSI, we design a deep Q network (DQN) to learn the optimal UAV deployment, due to the increased complexity to determine the jamming strategy with channel statistics. Simulation results demonstrate that both the learned DQN and trained DNN model effectively approximate the optimal deployment of the UAV. Moreover, compared with the baselines, our proposal significantly improves the security performance of legitimate transmissions. [ABSTRACT FROM AUTHOR]

In this paper, we propose a robust non-cooperative spectrum sensing algorithm based on deep learning over Rayleigh fading channel. We conduct noise cancellation on the received sensing data using the stacked convolutional auto-encoder (SCAE) as a pre-processing step. The series of the denoised signal in the time domain is then fed into the proposed Hybrid CNN-SA-GRU (H-CSG) network. The proposed network combines convolutional neural network (CNN), self-attention (SA) modules and gate recurrent unit (GRU). It can extract input features from spatial and temporal domains. The proposed algorithm has been shown to be effective and robust in detecting weak signals at the low signal-to-noise ratio (SNR) level. [ABSTRACT FROM AUTHOR]

Autonomous vehicles have been highlighted as a major growth area for future transportation systems and the deployment of large numbers of these vehicles is expected when safety and legal challenges are overcome. To meet the necessary safety standards, effective collision avoidance technologies are required to ensure that the number of accidents are kept to a minimum. As large numbers of autonomous vehicles, operating together on roads, can be regarded as a swarm system, we propose a bio-inspired collision avoidance strategy using virtual pheromones; an approach that has evolved effectively in nature over many millions of years. Previous research using virtual pheromones showed the potential of pheromone-based systems to maneuver a swarm of robots. However, designing an individual controller to maximise the performance of the entire swarm is a major challenge. In this paper, we propose a novel deep reinforcement learning (DRL) based approach that is able to train a controller that introduces collision avoidance behaviour. To accelerate training, we propose a novel sampling strategy called Highlight Experience Replay and integrate it with a Deep Deterministic Policy Gradient algorithm with noise added to the weights and biases of the artificial neural network to improve exploration. To evaluate the performance of the proposed DRL-based controller, we applied it to navigation and collision avoidance tasks in three different traffic scenarios. The experimental results showed that the proposed DRL-based controller outperformed the manually-tuned controller in terms of stability, effectiveness, robustness and ease of tuning process. Furthermore, the proposed Highlight Experience Replay method outperformed than the popular Prioritized Experience Replay sampling strategy by taking 27% of training time average over three stages. [ABSTRACT FROM AUTHOR]

In this paper, we investigate an unmanned aerial vehicle (UAV) communication system, where the trajectories of multi-UAVs are designed for the data collection mission of IoT nodes. We aim at minimizing the mission time with constraints of UAV’s maximum speed and acceleration, the collision avoidance, and communication interference among UAVs. We propose a three-step approach to solve this problem, which is based on the K-means algorithm, and Deep Reinforcement Learning (DRL) with a distributed manner and a centralized manner. The mutual influences like collision avoidance and interference among UAVs are explicitly expressed in our algorithm. Numerical results show the advantage of our proposed approach. [ABSTRACT FROM AUTHOR]

Nowadays, the autonomous driving transportation systems are at the heart of both academic and industry research for the distinguished advantages including increased network capacity, enhanced punctuality, greater flexibility and improved overall safety level. With the responsibility of transporting passengers in a safe, comfortable and efficient way, the decision making method plays a critical position in the autonomous driving of high-speed trains. Focusing on solving the autonomous decision making problem, this paper proposes a novel learning based framework by combining the deep learning technology with the distributed tracking control approach. To cope with the data insufficiency problem in training the deep learning network, a generative adversarial network (GAN) based data argumentation scheme is proposed to generate data samples that have the same distribution with actual data samples, and a hybrid learning network is constructed to predict the speed trajectory from the multi-attribute data with both temporal sequences and static features. Then, based on the model predictive control (MPC) scheme, a distributed tracking control model is formulated to minimize the tracking deviations and balance the performance of punctuality, energy-efficiency and riding comfort. Further, the dual decomposition technique is adopted to deal with the coupling constraints for the safe distance headway such that the separation for the autonomous driving of high-speed trains is achieved. Finally, simulation experiments based on actual scenarios of the Beijing-Shanghai high-speed railway are conducted to illustrate the effectiveness of our methods. [ABSTRACT FROM AUTHOR]

Autonomous pilot is crucial in integrally promoting the autonomy of an unmanned surface vehicle (USV). However, the integration mechanism of decision and control is still unclear within the entire autonomy. In this paper, by organically bridging path planning and tracking, an autonomous pilot framework with waypoints generation, path smoothing and policy guidance of a USV in congested waters is established, for the first time. Incorporating elite and diversity operations into the genetic algorithm (GA), an elite-duplication GA (EGA) strategy is devised to optimally generate sparse waypoints in a constrained space. The B-spline technique is further deployed to make flexibly smooth interpolation facilitating path smoothing supported by optimal sparse-waypoints. Seamlessly bridged by the parametric smooth path, deep reinforcement learning (DRL) technique is resorted to continuously extract in-depth pilotage policies, i.e., mappings from path tracking errors, collision risks and control constraints to continuous control forces/torques. Eventually, the entire spline-bridged EGA-DRL (SED) framework merits autonomous global-pilotage and local-reaction in an organically modular manner. Comprehensive validations and comparisons in various real-world geographies demonstrate the effectiveness and superiority of the proposed SED autonomous pilot framework. [ABSTRACT FROM AUTHOR]

Considering the continuance feature of the spectrum idle state observed from our spectrum measurements, in this paper, we propose a novel SKipping bAsed specTrum sEnsing (SKATE) scheme for cognitive radio (CR) transmissions. Instead of sensing at each time slot, we skip the sensing process in some following slots if a band is captured in an idle state, so that the sensing cost could be reduced significantly. For the STAKE scheme, how to determine the number of slots to skip is the key, where more slots could bring more benefits, but also with a higher probability to cause collisions. Since the perfect knowledge on spectrum environment might be hardly obtainable in practice, to enable the device to predict the spectrum state effectively and achieve the optimal joint decision on both band selection and skipped slots, we develop a reinforcement learning (RL) based solution for the SKATE scheme to make it learn the optimal strategy asymptotically. Both real and simulated spectrum data are adopted to demonstrate the effectiveness of the proposed RL based SKATE scheme. [ABSTRACT FROM AUTHOR]

Channel allocation has a direct and profound impact on the performance of vehicle-to-everything (V2X) networks. Considering the dynamic nature of vehicular environments, it is appealing to devise a blended strategy to perform effective resource sharing. In this paper, we exploit deep learning techniques predict vehicles’ mobility patterns. Then we propose an architecture consisting of centralized decision making and distributed channel allocation to maximize the spectrum efficiency of all vehicles involved. To achieve this, we leverage two deep reinforcement learning techniques, namely deep Q-network (DQN) and advantage actor-critic (A2C) techniques. In addition, given the time varying nature of the user mobility, we further incorporate the long short-term memory (LSTM) into DQN and A2C techniques. The combined system tracks user mobility, varying demands and channel conditions and adapt resource allocation dynamically. We verify the performance of the proposed methods through extensive simulations and prove the effectiveness of the proposed LSTM-DQN and LSTM-A2C algorithms using real data obtained from California state transportation department. [ABSTRACT FROM AUTHOR]

In this paper, we develop centralized and distributed dynamic resource allocation schemes for an intelligent reflecting surface (IRS) aided energy harvesting (EH) system that optimize transmit power of a source node and phase-shift of passive reflecting elements of IRS. The source node randomly harvests renewable energy from the surrounding environment and performs data transmission with the harvested energy while satisfying statistical packet delay constraints in terms of maximum acceptable delay-outage probability. Our developed schemes do not require the statistical distributions of channel and energy profiles to be known and apply deep reinforcement learning (DRL) algorithm. Simulation results demonstrate the effectiveness of the proposed resource control scheme for IRS-aided EH system in different channel and energy conditions. [ABSTRACT FROM AUTHOR]

Due to the scarcity in the wireless spectrum and limited energy resources especially in mobile applications, efficient resource allocation strategies are critical in wireless networks. Motivated by the recent advances in deep reinforcement learning (DRL), we address multi-agent DRL-based joint dynamic channel access and power control in a wireless interference network. We first propose a multi-agent DRL algorithm with centralized training (DRL-CT) to tackle the joint resource allocation problem. In this case, the training is performed at the central unit (CU) and after training, the users make autonomous decisions on their transmission strategies with only local information. We demonstrate that with limited information exchange and faster convergence, DRL-CT algorithm can achieve 90% of the performance achieved by the combination of weighted minimum mean square error (WMMSE) algorithm for power control and exhaustive search for dynamic channel access. In the second part of this paper, we consider distributed multi-agent DRL scenario in which each user conducts its own training and makes its decisions individually, acting as a DRL agent. Finally, as a compromise between centralized and fully distributed scenarios, we consider federated DRL (FDRL) to approach the performance of DRL-CT with the use of a central unit in training while limiting the information exchange and preserving privacy of the users in the wireless system. Via simulation results, we show that proposed learning frameworks lead to efficient adaptive channel access and power control policies in dynamic environments. [ABSTRACT FROM AUTHOR]

Automatic modulation classification (AMC) plays an important role in both civilian and military applications. Today, increasingly more researchers apply a deep learning framework in AMC. However, few papers take into account that a typical deep model is difficult to deploy on resource constrained devices. In this paper, we propose a new filter-level pruning technique based on activation maximization (AM) that omits the less important convolutional filter. Compared to other network pruning techniques, the convolutional neural network pruned via the AM method achieves equal or higher classification accuracy in the RadioML2016.10a dataset. [ABSTRACT FROM AUTHOR]

Characterization and linearization of RF power amplifiers (PAs) are key issues of fifth-generation wireless communication systems, especially when high peak-to-average ratio waveforms are introduced. Recently, deep learning methods have achieved great success in numerous domains including wireless physical-layer. However, there has been limited work in using deep learning for PAs behavioral modeling and linearization. In this paper, we make a bridge between memory effects of the nonlinear PAs and memory of bidirectional long short-term memory (BiLSTM) neural networks. We then build a BiLSTM-based behavioral modeling architecture and its accompanying digital predistortion (DPD) model by reconciling a non causality concern. Next, an additional model is proposed in this paper to mitigate uncertainty of the tested PA when transforming phases. The experimental results demonstrate the effectiveness of the proposed scheme, in which the adequately trained networks are capable of characterizing the PA, and the artificial intelligence-based DPD shows promising linearization performance when considering the tested PAs inherent unpredictability. [ABSTRACT FROM AUTHOR]

Past few years have witnessed the widespread adoption of unmanned aerial vehicles (UAVs) in target tracking for regional monitor and strike. Most existing target tracking approaches rely on the target motion frames obtained by the camera equipped, or on ideally assuming a pre-set target trajectory. However, in practice, the real trajectory of the target cannot be perfectly known to the UAVs in advance, and also the target may intelligently adjust its flying strategy according to the environment. Besides, the limited flight performance, as well as information capture and processing capability, of a single UAV can hardly fulfill high tracking success rate requirements. To address aforementioned issues, this paper proposes an end-to-end cooperative multi-agent reinforcement learning (MARL) scheme, where UAVs are enabled to make intelligent flight decisions for cooperative target tracking, on the basis of the past and current states of the target. In order to reduce power consumption and prolong the lifetime of the UAV tracking system, the propulsion power consumption model and energy saving strategy are introduced. Moreover, to further increase the detection coverage, spatial information entropy is introduced in the tracking algorithm. Simulation results show that our proposed algorithm outperfoms the deep reinforcement learning baselines in terms of the mean episode rewards, while also yields high performances with respect to tracking success rates, power saving efficiency and detection coverage. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel deep learning (DL)-based codebook design method for generalized space shift keying (GSSK) systems. In this DL-based method, the transmitter and receiver of GSSK systems are designed based on deep neural network (DNN). By training the DNN in an end-to-end manner, the DL-based method can adaptively generate suitable binary codewords and combine them into a codebook for GSSK systems. Simulation results show that the proposed DL-based method obtains better performance compared to conventional approaches. [ABSTRACT FROM AUTHOR]

In this paper, collaborative task offloading in edge computing is studied, where computation requesters can offload tasks to not only the edge server, but also nearby smartphone users. By considering the fact that smartphone users may not always be willing to provide such computation service because of the consumption of their own energy and resources, a truthful mechanism is designed to provide incentive to smartphone users. The design aims to maximize the net revenue of the service provider and addresses more practical, but more complicated, scenarios of unknown a prior distribution information on smartphone users’ private information. To tackle this high computational complexity, which makes the traditional mechanism design methods infeasible, a new approach, called truthful deep mechanism, is proposed by leveraging a multi-task machine learning model, where inherently inter-connected collaborator selection and pricing policy determination are decided by designing two deep neural networks. The numerical results show that the proposed deep truthful mechanism can ensure a convergence to a stable state and can satisfy all required economical properties, including individual rationality, incentive compatibility, and budget balance. [ABSTRACT FROM AUTHOR]

In this paper, we introduce channel prediction to wireless communications aided by intelligent reflecting surface (IRS) in order to reduce the overhead brought by full channel estimation. Hopefully, fine-grained control of reflection coefficients on IRS can still be maintained with sufficiently accurate channel prediction. Innovatively, we consider channel prediction as an analogy to image super-resolution in computer vision since elements on an IRS are normally two-dimensionally arranged, which makes the popular deep learning techniques efficient solution. To this end, we first solve the element control problem via semi-definite relaxation (SDR) and then design an end-to-end deep-learning model for channel prediction. With estimation for only part of the entire channel state information (CSI), the total channel estimation overhead can be significantly reduced. We visually show that the purposed deep-learning model maintains considerably high accuracy for channel prediction, which makes the proposed channel-prediction method outperforms the existing group-based method in terms of achievable rate. Moreover, the tradeoff between channel estimation overhead and channel prediction accuracy is also investigated with numerical results. [ABSTRACT FROM AUTHOR]

One of the pillars of the 5G architecture is network slicing, in which hardware, radio, and power resources are virtualized as a logical network taking into account the requirements of diverse applications. While ensuring performance isolation among different slices, resource allocation in 5G Radio Access Networks (RANs) is associated with different challenges due to network dynamics and the different applications’ requirements. In this paper, we have considered the allocation of power and radio resources to rate-based as well as resource-based users. We propose an energy-efficient deep reinforcement learning-assisted resource allocation (EE-DRL-RA) method for RAN slicing in 5G networks. The main idea of the proposed method is to exploit a collaborative learning framework that includes deep reinforcement learning (DRL) and deep learning (DL) to decide on resource allocation in the RAN. Specifically, we use DL for decision-making on resource allocation on a large time-scale and DRL for decision-making on resource allocation on a small time-scale. The asynchronous advantage actor-critic (A3C) and the stacked and bidirectional long-short-term-memory (SBiLSTM) network are used as DRL and supervised DL methods, respectively. Furthermore, we determine the optimal power and resource blocks (RBs) for rate-based users by formulating the energy-efficient power allocation (EE-PA) problem as a non-convex optimization problem and solve it by an efficient iterative algorithm. Our proposed approach is unique in that it simultaneously allocates power and RBs while ensuring slice isolation with low computational and time complexity. Simulation results show that EE-DRL-RA yields better performance compared to a state-of-the-art published method in terms of convergence speed, computational complexity, energy efficiency, and the number of accepted users as well as the degree of inter-slice isolation. [ABSTRACT FROM AUTHOR]

In this paper, we propose a deep neural network (DNN) aided codebook based beamforming scheme for frequency-division-duplex (FDD) millimeter-Wave (mmWave) massive multiple-input multiple-output (MIMO) systems under multipath. Different from the time-division-duplex (TDD) systems, for FDD systems the reciprocity between the down-link (DL) and up-link (UL) channel state information (CSI) does not hold in general, requiring an extra CSI feedback stage. Based on the previous theoretical analysis and measurements, however, partial reciprocities, including the spatial directional angles and number of propagation paths, do exist for FDD mmWave systems. Given such partial reciprocities, we propose a beamforming scheme based on a predefined codebook. Furthermore, we consider the channels with multipath where multiple codewords can be selected for some mobile stations (MSs) to utilize more than one propagation paths, which improves the link reliability. We then propose the corresponding beamforming strategy that not only identifies the number of paths, but also reconstructs the individual single-path channels which are easier to deal with at the beamforming stage. To select the optimal codewords for these single-path channels with lower complexity, we then present a DNN-aided approach that predicts the optimal codewords. The integration of DNN resolves not only the issue of power leakage, but also angular ambiguity in the codebook. Simulation results demonstrate that the performance of the proposed DNN-aided approach is very close to that of an exhaustive search. [ABSTRACT FROM AUTHOR]

Radar detection and communication are two essential sub-tasks for the operation of next-generation autonomous vehicles (AVs). The forthcoming proliferation of faster 5G networks utilizing mmWave has raised concerns on interference with automotive radar sensors, which has led to a body of research on Joint Radar-Communication (JRC). This paper considers the problem of time-sharing for JRC, with the additional simultaneous objective of minimizing the average age of information (AoI) transmitted by a JRC-equipped AV. We first formulate the problem as a Markov Decision Process (MDP). We then propose a more general multi-agent system, with an appropriate medium access control (MAC) protocol, which is formulated as a partially observed Markov game (POMG). To solve the POMG, we propose a multi-agent extension of the Proximal Policy Optimization (PPO) algorithm, along with algorithmic features to enhance learning from raw observations. Simulations are run with a range of environmental parameters to mimic variations in real-world operation. The results show that the chosen deep reinforcement learning methods allow the agents to obtain strong performance with minimal a priori knowledge about the environment. [ABSTRACT FROM AUTHOR]

Knowledge transfer is a promising concept to achieve real-time decision-making for autonomous vehicles. This paper constructs a transfer deep reinforcement learning (RL) framework to transform the driving tasks in the intersection environments. The driving missions at the unsignalized intersection are cast into a left turn, right turn, and running straight for automated vehicles. The goal of the autonomous ego vehicle (AEV) is to drive through the intersection situation efficiently and safely. This objective promotes the studied vehicle to increase its speed and avoid crashing other vehicles. The decision-making policy learned from one driving task is transferred through three transfer rules in another driving mission and evaluated. Simulation results reveal that the decision-making strategies related to similar tasks are transferable and have a high success rate. It indicates that the presented control framework could reduce time consumption and realize online implementation. Therefore, the transfer RL concept is helpful for establishing the real-time decision-making policy for autonomous vehicles. [ABSTRACT FROM AUTHOR]

In this paper, we study the three-dimensional (3D) path planning problem for cellular-connected unmanned aerial vehicle (UAV) taking into account the impact of 3D antenna radiation patterns. The cellular-connected UAV has a mission to travel from an initial location to a destination. In this process, there is a trade-off between the flight time and the expected communication outages duration, which requires planning a suitable route to avoid the weak coverage region. However, as the existing cellular networks are primarily designed for ground coverage, the communication coverage in the sky tends to be intermittent and irregular due to the potential for severe interference and obstructions (such by buildings), which brings great challenges to 3D path planning. To address this challenge, we first construct a 3D coverage map that stores the expected outage probability over each locations. We then propose a multi-step dueling DDQN (multi-step D3QN) based algorithm to design the local optimal UAV path by leveraging the constructed coverage map. In this algorithm, the UAV acts as the agent to learn the appropriate action to complete the flight mission. Numerical results show the effectiveness of the proposed algorithm for connectivity-aware UAV path planning and the superiority of 3D path design over its 2D counterparts. [ABSTRACT FROM AUTHOR]

The development of Autonomous Vehicles (AVs) envisions the promising technology of future Intelligent Transportation Systems (ITS). However, the complex road structures and increased vehicles cause traffic congestion and road safety, which eventually leads to horrible accidents. Cooperative driving of AVs, a groundbreaking initiative of vehicle platooning, epitomizes the next wave in vehicular technology through minimizing accident risks, transport times, costs, energy, and fuel consumption. However, the traditional machine learning-based platooning approaches fail to regulate the policy with the dynamic feature of AVs. This paper proposes a hybrid Deep Reinforcement learning and Genetic algorithm for Smart-Platooning (DRG-SP) the AVs. The leverage of the deep reinforcement learning mechanism addresses the computational complexity and accommodates the high dynamic platoon environments. Adopting the Genetic Algorithm in Deep Reinforcement learning overcomes the slow convergence problem and offers long-term performance. The simulation results reveal that the Smart-Platooning effectively forms and maintains the platoons by minimizing traffic congestion and fuel consumption. [ABSTRACT FROM AUTHOR]

Currently, self-driving, emerging as a key automatic application, has brought a huge potential for the provision of in-vehicle services (e.g., automatic path planning) to mitigate urban traffic congestion and enhance travel safety. To provide high-quality vehicular services with stringent delay constraints, edge computing (EC) enables resource-hungry self-driving vehicles (SDVs) to offload computation-intensive tasks to the edge servers (ESs). In addition, caching highly reusable contents decreases the redundant transmission time and improves the quality of services (QoS) of SDVs, which is envisioned as a supplement to the computation offloading. However, the high mobility and time-varying requests of SDVs make it challenging to provide reliable offloading decisions while guaranteeing the resource utilization of content caching. To this end, in this paper we propose a collaborative computation offloading and content caching method, named CoPace, by leveraging deep reinforcement learning (DRL) in EC for self-driving system. Specifically, we first introduce OSTP to predict the future time-varying content popularity, taking into account the temporal-spatial attributes of requests. Moreover, a DRL-based algorithm is developed to jointly optimize the offloading and caching decisions, as well as the resource allocation (i.e., computing and communication resources) strategies. Extensive experiments with real-world datasets in Shanghai, China, are conducted to evaluate the performance, which demonstrates that CoPace is both effective and well-performed. [ABSTRACT FROM AUTHOR]

Air ground integrated mobile cloud computing (MCC) provides unmanned aerial vehicles (UAVs) the capability to act as an aerial relay with more flexibility and resilience. In the cloud computing architecture, the data generated by ground users (GUs) can be offloaded to the remote server for fast processing. However, the heterogeneity of mobile tasks makes the data size distributed among GUs unbalanced. Besides, the energy efficiency of UAVs movement should be carefully considered for sustainable flight and obstacle avoidance. In general, such a joint trajectory issue can hardly be formulated as a convex optimization in unpredictable and dynamic environments. This paper proposes a potential game combined multi-agent deep deterministic policy gradient (MADDPG) approach to optimize multiple UAVs' trajectory with the consideration of GUs' offloading delay, energy efficiency as well as obstacle avoidance system. In specific, we first model the issue as a mixed integer non-linear problem (MINP), in which the service assignment between multi-user and multi-UAV is solved by potential game. The convergence to a Nash Equilibrium (NE) can be achieved by distributive service assignment update with infinite iteration. Then, we optimize the trajectory with obstacle avoidance at each UAV by MADDPG approach, which has a great advantage of centralized-training and decentralized-execution to reduce the global synchronized communication overhead. UAVs movement can be optimized in continuity rather than other deep reinforcement learning (DRL) approaches generating discrete simple actions. Experiments demonstrate the proposed game-combined learning algorithm can minimize the offloading delay, enhance UAVs’ energy efficiency and avoid the obstacles at the same time. [ABSTRACT FROM AUTHOR]

IEEE 802.11p standard defines wireless technology protocols that enable vehicular transportation and manage traffic efficiency. A major challenge in the development of this technology is ensuring communication reliability in highly dynamic vehicular environments, where the wireless communication channels are doubly selective, thus making channel estimation and tracking a relevant problem to investigate. In this paper, a novel deep learning (DL)-based weighted interpolation estimator is proposed to accurately estimate vehicular channels especially in high mobility scenarios. The proposed estimator is based on modifying the pilot allocation of the IEEE 802.11p standard so that more transmission data rates are achieved. Extensive numerical experiments demonstrate that the developed estimator significantly outperforms the recently proposed DL-based frame-by-frame estimators in different vehicular scenarios, while substantially reducing the overall computational complexity. [ABSTRACT FROM AUTHOR]

In this paper, we consider an underwater linear network, where an autonomous underwater vehicle (AUV) gathers data from a set of underwater devices. The AUV monitors a set of physical processes, where the status of each process can be sensed by one or more devices and each device is capable of sensing one or more processes. The AUV needs to maintain freshness of its information status about the monitored processes. To quantify the freshness of the information at the AUV, we consider the concept of the age of information (AoI), which represents the amount of time elapsed since the most recently delivered update information was generated. A framework is proposed to optimize the AUV's linear movement trajectory and scheduling of process status updates with the objective of minimizing the normalized weighted sum of the average AoI of the monitored physical processes. The formulated optimization problem is a non-convex mixed integer problem, which cannot be solved by the standard optimization techniques. We develop a solution approach based on the technique of deep reinforcement learning (DRL). Specifically, we leverage an actor-critic DRL approach to find the optimum locations and stopping time of the data gathering points. Simulation results illustrate that the proposed framework maintains robustness under different scenarios and provides better performance when compared with baseline and $K$ -means clustering approaches. [ABSTRACT FROM AUTHOR]

In this paper, we present a novel unified framework to protect multi-function wireless systems from jamming attacks. Examples of such multi-function system include joint radar and communication (JRC) systems and simultaneous wireless information and power transfer (SWIPT) systems. By abstracting the system functionalities as a joint optimization problem of multiple queues, we achieve effective resistance against jammers for the multi-functions simultaneously. We incorporate different anti-jamming techniques into one framework. Deception mechanism is adopted to lure the jammer to attack and make its actions more predictable, and ambient backscatter technology is used to leverage the jamming signals. Since conventional Markov decision process (MDP) has only one decision epoch at every time slot, it cannot be used to model the deception strategy which needs two decision epochs to leverage the jamming signals. We therefore formulate the problem using an advanced two-step MDP. After that, a deep reinforcement learning algorithm with a prioritized double deep Q-Learning architecture is proposed to learn optimal strategies in different system states. We show that by jointly considering the multi-functions of the system with potential jamming attacks during design phase, significant improvement can be achieved for both of the system functionalities. [ABSTRACT FROM AUTHOR]

We are seeing a growth in the number of connected vehicles in Vehicular Ad-hoc Networks (VANETs) to achieve the goal of Intelligent Transportation System (ITS). This is leading to a connected vehicular network scenario with vehicles continuously broadcasting data to other vehicles on the road and the roadside network infrastructure. The presence of a large number of communicating vehicles greatly increases the number and types of possible anomalies in the network. Existing works provide solutions addressing specific anomalies in the network only. However, since there can be a multitude of anomalies possible in the network, there is a need for better anomaly detection frameworks that can address this unprecedented scenario. In this paper, we propose an anomaly detection framework for VANETs based on deep neural networks (DNNs) using a sequence reconstruction and thresholding algorithm. In this framework, the DNN architectures are deployed on the roadside units (RSUs) which receive the broadcast vehicular data and run anomaly detection tasks to classify a particular message sequence as anomalous or genuine. Multiple DNN architectures are implemented in this experiment and their performance is compared using key evaluation metrics. Performance comparison of the proposed framework is also drawn against the prior work in this area. Our best performing deep learning-based scheme detects anomalous sequences with an accuracy of 98%, a great improvement over the set benchmark. [ABSTRACT FROM AUTHOR]

In vehicle trajectory prediction, the difficulty in modeling the interaction relationship between vehicles lies in constructing the interaction structure between the vehicles in the traffic scene. Majority of existing models only focus on the interaction between the historical trajectory of the vehicle and the surrounding vehicles in the spatial domain, and do not pay attention to the interaction between the vehicle and the non-Euclidean correlation structure (graph structure) that exists in the environment. In order to overcome the deficiencies in the existing models, this paper proposes the Environment-Attention Network model (EA-Net) to obtain the full interactive information between the vehicle and its driving environment. In the proposed model, a new type of parallel structure consisting of Graph Attention network (GAT) and Convolutional social pooling containing Squeeze-and-Extraction mechanism (SE-CS), is constructed as the environmental feature extraction module and embedded in LSTM encoder-decoder. This structure solves the limitation of the dimension and structure influence when constructing the interaction model between the vehicle and the surrounding environment, making the extracted feature information comprehensive and effective. The prediction accuracy of the model with the RMSE loss function is tested on two public datasets– NGSIM and highD, and compared with several state-of-the-art trajectory prediction algorithm models. The results show that the prediction accuracy of the proposed Environment-Attention Network in the two datasets is more than 20% higher than that of the single structure model, which indicates that the proposed model proposed has superior performance and better adaptability to different traffic environments compared with the existing models. [ABSTRACT FROM AUTHOR]