Multi-Ship Collision Avoidance Method based on Markov Decision Process

The maritime industry has been incorporating advanced technology to enhance mission planning and ensure safe navigation, including autonomous collision avoidance systems that follow International Regulations for Preventing Collisions at Sea (COLREG). Ongoing research in this field encompasses a wide range of approaches, from optimal control analysis to heuristic and metaheuristic methods, and solutions based on artificial intelligence. In this study, we propose an autonomous collision avoidance algorithm for ships based on Markov decision process. This work focuses on the development of a COLREG-compliant autonomous collision avoidance algorithm for ships using a Markov decision process. The algorithm considers the subject ship's position and aims to resolve potential collision conflicts with target ships while keeping the vessel on its initial trajectory, in compliance with regulations. The system is modeled as a Markov decision process using the ship's three coordinates position as states, actions generated from degrees-of-freedom, and constraints such as safe path, trip cost, and respect for rules to design the reward. The proposed policy search algorithm is implemented using python and its convergence and efficiency are tested through multiple scenarios.