Transactions of Nanjing University of Aeronautics & Astronautics
A New Intelligent Decision-Making Method for Air-Sea Joint Operation Based on Deep Reinforcement Learning
- Article
- Figures
- Metrics
- Preview PDF
- Reference
- Related
- Cited by
- Materials
Abstract:
Aiming at the difficulty of air-sea joint operation in complex multi-equipment combat with high uncertainty， a new intelligent decision-making method for air-sea joint operation based on deep reinforcement learning is proposed. To uniformly represent the input and output of complex networks and their corresponding relations， various networks are utilized， e.g.， perceptron， deep long-short term memory network and actor critical structure. Aiming at the instability of policy network learning process and the defects of the proximal policy optimization（PPO） algorithm， an improved proximate policy optimization algorithm is proposed. To enhance the variability of opponent’s strategy in the process of policy network self-learning， a baseline policy model selection method based on model performance and model diversity is proposed. The experiments demonstrate that the proposed method is effective and stable in air-sea joint operation decision. In the 4th Wargaming Competition hosted by Chinese Institute of Command and Control， the winning rate in more than 100 rounds against regular decision-making algorithm and human confrontation was 97%， which was about 20% higher than that of regular decision-making algorithms.
Keywords:
Project Supported:
This work was supported by the National Natural Science Foundation of China （Nos.62073102，62203145） and the China Postdoctoral Science Foundation （No.2022M710948）.
Clc Number:
TP273
CHINA AERO-SCIENCES
Mobile website
