Search

The study proposes a method for the trajectory tracking control of a fixed-wing unmanned aerial vehicle (UAV) based on the deep deterministic policy gradient (DDPG). First, the problem of controlling the trajectory of a fixed-wing UAV is combined with the reinforcement learning framework and transformed into a Markov decision process, and a DDPG agent is established in the framework of TensorFlow. Second, we conducted simulations to train and optimize the model in a 3D environment of trajectory tracking control and obtained an integrated DDPG-based trajectory tracking controller that can regulate functions ranging from the state of flight of the UAV to rudder control. Third, we constructed a digital simulation system to verify the proposed method while considering the influence of parametric uncertainties, measurement-induced noise, and delays in the response of the control system. The effectiveness and robustness of the proposed DDPG controller were verified by comparing its performance with that of traditional proportional-integral-derivative (PID) control. Unmanned aerial vehicles have gained widespread use across numerous sectors of society. However, the control process for fixed-wing UAVs is more intricate and challenging than that for rotor type UAVs due to the coupled and nonlinear nature of their dynamic models. To address this issue, decoupling and splitting hierarchical control methods are commonly employed. However, these methods can introduce errors and affect flight performance. In contrast, artificial intelligence, particularly deep learning and reinforcement learning, has made significant advancements and found successful applications in areas such as robot control and gaming. To address the challenges posed by complex forms, excessive measurement information, and manual parameter tuning associated with traditional control approaches for fixed-wing UAVs, this paper introduces a deep reinforcement learning algorithm, i.e., the deep deterministic policy gradient, into the control process of fixed-wing UAVs. The paper presents a trajectory tracking control policy for UAVs, which efficiently achieves the task of tracking a desired trajectory without the need for selecting multiple working state nodes or designing linear control laws. Further, through simulation verification, the proposed policy demonstrates adaptability to interference caused by parameter bias, measurement noise, and control system response delays while achieving control accuracy comparable with classical control methods. [ABSTRACT FROM AUTHOR]

Traditional aerodynamic modeling methods have some difficulties in multiparameter unsteady state-space modeling and control law design, which raise significant challenges in flight control. The rapid development of machine learning provides a new idea for unsteady aerodynamic modeling and control law design, which has significant value for theoretical research and engineering application. In this paper, an unsteady aerodynamic model environment was established based on deep neural network (DNN), and a dynamic computational fluid dynamics (CFD) virtual environment was taken as an approximation of the real environment. A deep reinforcement learning (DRL) longitudinal-axis flight control method was studied on the basis of both environments. The deep deterministic policy gradient (DDPG) algorithm was used to implement flight control, and the effects of different constraints in the reward function on the results of DRL were studied at the same time. The results show that the DDPG agent effectively achieves longitudinal-axis flight control in the model environment. The DDPG agent trained in the model environment was also used for longitudinal-axis flight control in the dynamic CFD virtual environment, and this method provides a reference for the coupling of dynamic CFD and DRL. The control results from the model environment and the dynamic CFD virtual environment are compared, and results show that the DDPG agent has good robustness in both environments. The results of this study suggest that machine learning and deep reinforcement learning can effectively solve complicated multiconstraint, uncertain, and sophisticated control tasks. In this paper, a DNN-based unsteady aerodynamic modeling method was established based on dynamic computational fluid dynamics calculation and machine learning, and the results show that the DNN-based unsteady aerodynamic model has advantages in dealing with highly nonlinear data under unsteady conditions. In addition, the deep deterministic policy gradient algorithm can be well applied in longitudinal-axis flight control, and the agent can be trained in DNN-based unsteady aerodynamic model environment. The agent was also used for longitudinal-axis flight control in the dynamic computational fluid dynamics virtual environment, and the results show that the agent has good robustness in both environments. This provides the possibility for the agent to achieve longitudinal-axis flight control in a real environment. [ABSTRACT FROM AUTHOR]