Search

In this paper, we study a autonomous aerial vehicle (AAV) inspection system. In this system, the AAV flies to all inspection points in a certain area for patrol inspection, and the energy of inspection AAV is limited. Our goal is to optimize the communication quality of the AAV by planning the inspection sequence and flight trajectory, so as to ensure that the AAV can complete the inspection task and minimize the outage time subject to limited energy of the AAV. To solve this problem, we propose a hybrid algorithm, which consists of simulated annealing (SA) algorithm and Dueling Double Deep Q Network (D3QN) algorithm. The SA algorithm is used to obtain the inspection sequence of the AAV with the most energy saving. On this basis, the D3QN algorithm is used to optimize the flight trajectory of the energy-limited inspection AAV. To prove the effectiveness of the sub-optimal solution obtained by our proposed algorithm, we use several algorithms as a comparison. Numerical results show that the proposed algorithm is effective in optimizing the communication quality of the inspection AAV with limited energy, and its performance is improved by about 15%-50% compared with other benchmarks.

With the Internet of Vehicles (IOV), a lot of self-driving vehicles (SDVs) need to handle a variety of tasks but have very seriously limited computing and storage resources, meaning they cannot complete intensive tasks timely. In this paper, a joint edge computing and caching based on a Dueling Double Deep Q Network (D3QN) is proposed to solve the problem of the multi-task joint edge calculation and caching process. Firstly, the processes of offloading tasks and caching them to the base station are modeled as optimization problems to maximize system revenues, which are limited by system latency and energy consumption as well as cache space for computing task constraints. Moreover, we also take into account the negative impact of the number of unfinished tasks in relation to the optimization problem—the higher the number of unfinished tasks, the lower the system revenue. Secondly, we use the D3QN algorithm together with the cache models to solve the formulated NP-hard problem and select the optimal caching and offloading action by adopting an e-greedy strategy. Moreover, two cache models are proposed in this paper to cache tasks, namely the active cache, based on the popularity of the task, and passive cache, based on the D3QN algorithm. Additionally, tasks which deal with cache space are updated by computing the expulsion value based on type of popularity. Finally, simulation results show that the proposed algorithm has good performance in terms of the latency and energy consumption of the system and that it improves utilization of cache space and reduces the probability of unfinished tasks. Compared to the Deep Q Network with caching policy, with the Double Deep Q Network with caching policy and Dueling Deep Q Network with caching policy, the system revenue of the proposed algorithm is improved by 65%, 35% and 66%, respectively. The scenario of the IOV proposed in this article can be expanded to larger-scale IOV systems by increasing the number of SDVs and base stations, and the content caching and download functions of the Internet of Things can also be achieved through collaboration between multiple base stations. However, only the cache model is focused on in this article, and the design of the replacement model is not good enough, resulting in a low utilization of cache resources. In future work, we will analyze how to make joint decisions based on multi-agent collaboration for caching, offloading and replacement in IOV scenarios with multiple heterogeneous services to support different Vehicle-to-Everything services. [ABSTRACT FROM AUTHOR]

Fault diagnosis is subject to the challenge of implementing model learning in the presence of small samples and imbalanced data (i.e., variable operating conditions), which is a fundamental and crucial problem that hinders their applications in real industrial scenarios. Herein, a novel deep reinforcement learning strategy (SIMC-PERDRL) that combines SimCLR and elevated prioritized experience replay (PER) is proposed for machinery fault quantitative diagnosis in non-ideal data scenarios. First, unsupervised contrastive learning pre-trains the feature extraction layer to mine optimal discriminative features with more optimal intra-class compactness and inter-class separability to reduce inter-class overlap. Second, the experience priority is quantified by reward and TD error to enhance the learning frequency of rare high-value samples; the reward function is skillfully constructed using adaptive unbalanced distribution, which immensely increases the agent's sensitivity to minorities, and enhances the model's domain adaptability by dynamically fine-tuning the agent's decision through real-time feedback. Moreover, ResNet utilizes the Convolutional Block Attention Module (CBAM) to construct a deep Q-network; thus, the agent's learning ability of critical fault features is enhanced. Finally, SIMC-PERDRL was validated online using three rotating machinery datasets. The results indicate that the method can automatically realize accurate qualitative identification under different rotational speeds, different loads, and class unbalanced conditions, with excellent effectiveness, stability, and versatility. [ABSTRACT FROM AUTHOR]