Search

Facing the future requirements of distributed warfare at sea, the strategic planning of missile penetration is firstly analyzed based on the background of intelligent missile salvo penetration against surface ships in distributed warfare scenario. Secondly, a strategic planning method of intelligent missile penetration based on reward-shaping reinforcement learning is designed by using multi-class reward function. Then, the operation scenario of the missile penetration ship is constructed on the Mozi joint operation simulation system. The comparison experiment shows that the success rate of the intelligent missile penetration attack controlled by the model learned by the reward molding method is 79%, which verifies the effectiveness of the reward-based reinforcement learning method. Finally, after action review, it is found that there are emerging four kinds of penetration strategies of intelligent missiles in the reward shaping experiment, including concentrated and roundabout attack, scattered penetration multi-direction attack, group delay attack and cruise detection guide attack.

The breakthrough and progress of intelligent gaming technology with deep reinforcement learning as the core in the field of games provide a method reference for the research of agents in sea-air wargames. The architecture design of the agent is the primary core key problem that needs to be solved, and a good architecture can reduce the complexity and difficulty of training and accelerate the convergence of policies. A stochastic game model of sea-air cross-domain cooperative decision-making has been proposed, and its corresponding equilibrium solution concepts have been analyzed. Based on the analysis of typical agent frameworks, aiming at the decision-making gaming process of sea-air wargames, and then an agent bi-level architecture based on multi-Agent hierarchical reinforcement learning is proposed, which can effectively solve the problems of collaboration and dimensional disaster. The key technologies are analyzed from four aspects: force coordination, agent network design, adversary modeling and training mechanism. Hoping to provide architectural guidance for the subsequent design and implementation of sea-air wargaming agents.

Multi-agent systems are a cutting-edge research concept in the field of distributed artificial intelligence. Traditional multi-agent reinforcement learning methods mainly focus on topics such as group behavior emergence, multi-agent cooperation and coordination, communication and communication between agents, opponent modeling and prediction. However, they still face challenges such as observable environment, non-stationary opponent strategies, high dimensionality of decision space, and difficulty in understanding credit allocation. How to design multi-agent reinforcement learning methods that meet the large number and scale of intelligent agents and adapt to multiple different application scenarios is a cutting-edge topic in this field. This article first outlined the relevant research progress of multi-agent reinforcement learning. Secondly, a comprehensive overview and induction of multi-agent learning methods with multiple types and paradigms were conducted from the perspectives of scalability and population adaptation. Four major categories of scalable learning methods were systematically sorted out, including set permutation invariance, attention, graph and network theory, and mean field theory. There were four major categories of population adaptive reinforcement learning methods: transfer learning, course learning, meta learning, and meta game, and typical application scenarios were provided. Finally, the frontier research directions were prospected from five aspects: benchmark platform development, two-layer optimization architecture, adversarial strategy learning, human-machine collaborative value alignment and adaptive game decision-making loop, providing reference for the research on relevant frontier key issues of multi-agent reinforcement learning in multimodal environments.