Search

The continuous development of communication technology and various deep learning models has led to the invention and application of many anti-interference technologies in the field of communication countermeasures. The existing communication interference models have defects such as low anti-interference rate and low accuracy in communication spectrum prediction. To solve these problems, this study attempts to construct a Convolutional Neural Networks Long Short Term Memory (CNN-LSTM) and apply it to the communication jamming system for spectrum state prediction. Firstly, the framework of the communication interference system using the USRP RIO radio platform software was designed, and based on it, the communication interference channel was optimized using reinforcement learning Q-learning algorithm. Next, to further predict the signal spectrum state during the communication process, neural networks are utilized to construct a communication spectrum state prediction model. According to the optimization effect of communication interference channel and network spectrum prediction effect tested, the communication model under the Q-learning algorithm can achieve a 100% effective interference probability in fixed communication strategies. The Convolutional Neural Networks-1 Long-Short Term Memory-2 model has a prediction accuracy of 95.2% and can accurately predict changes in the communication spectrum. In summary, the Convolutional Neural Networks-1 Long-Short Term Memory-2 network constructed by this paper can provide new solutions and achieve good results for communication spectrum prediction.

Packet routing is a fundamental problem in wireless networks in which routers decide the next hop for each packet in order to deliver it to its destination as quickly as possible. In order to overcome the shortcoming of non optimal forwarding path caused by fixed forwarding mode in geographic location-based routing algorithm, we investigated a new efficient packet routing strategy which combined with Deep Reinforcement Learning (DRL) algorithm (Proximal Policy Optimization: PPO) to minimize the hops and reduce the probability of encountering “routing hole” during the forwarding of the packets in complex networks. Each node in our network can make routing decisions to learn a routing policy that can not only use the greedy mode to maintain the efficiency of routing policy but also reduce the use of peripheral forwarding mode. The learning process of the DRL agent is based on the status information of the packet being transmitted and information of neighbor nodes within the communication range. Extensive simulations over dynamic network topologies and the number of nodes have shown that our packets routing agent can learn the optimal policy in terms of average packets delivery rate and the number of hops compared with Greedy Perimeter Stateless Routing (GPSR) protocol. The performance of PPO in large-scale action space is also verified, which provides a basis for the future researches to combine PPO technique with packet routing optimization.

Reinforcement learning has recently made great progress in various challenging domains such as board game of Go and MOBA game of StarCraft II. Policy gradient based reinforcement learning method has become the mainstream due to its effectiveness and simplicity both in discrete and continuous scenarios. However, policy gradient methods commonly involve function approximation and work in an on-policy fashion, which leads to high variance and low sample efficiency. This paper introduces a novel policy gradient method to improve the sample efficiency via a pair of trajectory based prioritized replay buffers and reduce the variance in training with a target network whose weights are updated in a “soft” manner. We evaluate our method on the reinforcement learning suit of Open AI Gym tasks, and the results show that the proposed method can learn more steadily and achieve higher performance than existing methods.

The unmanned aerial vehicle (UAV) has drawn attention from the military and researchers worldwide, which has advantages such as robust survivability and execution ability. Mobility models are usually used to describe the movement of nodes in drone networks. Different mobility models have been proposed for different application scenarios; currently, there is no unified mobility model that can be adapted to all scenarios. The mobility of nodes is an essential characteristic of mobile ad hoc networks (MANETs), and the motion state of nodes significantly impacts the network’s performance. Currently, most related studies focus on the establishment of mathematical models that describe the motion and connectivity characteristics of the mobility models with limited universality. In this study, we use a backpropagation neural network (BPNN) to explore the relationship between the motion characteristics of mobile nodes and the performance of routing protocols. The neural network is trained by extracting five indicators that describe the relationship between nodes and the global features of nodes. Our model shows good performance and accuracy of classification on new datasets with different motion features, verifying the correctness of the proposed idea, which can help the selection of mobility models and routing protocols in different application scenarios having the ability to avoid repeated experiments to obtain relevant network performance. This will help in the selection of mobility models for drone networks and the setting and optimization of routing protocols in future practical application scenarios.

Aiming at anti Unmanned Aerial Vehicle (UAV) swarm, this paper studies the detection and suppression mechanisms of emergence in cooperative flight. Cooperative fly is one of the critical operations for UAV swarm in both military and civilian utilities, which allows individual UAVs to distributed adjust their velocity to head for a common destination as well as avoid a collision. This process is viewed as the emergence of complex systems. An emergence detection algorithm based on double thresholds is proposed. It simultaneously monitors the cooperative flight process and system connectivity to accurately identify the occurrence, achievement, or failure of cooperative fly, which provides a solid prerequisite for the suppression mechanism. For suppression, in-band radio interference is designed under the constraint of average power, and the effect is modeled from the perspective of degrading the communication performance of the target system. It is found that low-intensity continuous interference can effectively delay the cooperative fly process and has better concealment, while medium-intensity continuous interference can rapidly stop that process. Based on the above analysis, for the first time, two countermeasures for the UAV swarm’s cooperative fly are designed for the operation intent of delay and disruption of the target UAC swarms, respectively. Simulation results show the effectiveness of the countermeasures.