Transactions of Nanjing University of Aeronautics & Astronautics
UAV Formation Beyond-Visual-Range Air Combat Decision Based on Multi-agent Reinforcement Learning
- Article
- Figures
- Metrics
- Preview PDF
- Reference
- Related
- Cited by
- Materials
Abstract:
With the rapid evolution of weapon systems towards precision and intelligence， unmanned aerial combat has increasingly transitioned from close-range engagements to beyond-visual-range （BVR） operations. This paper addresses the challenges of learning effective missile launch strategies in BVR air combat， where the long delay between weapon launch and target hit leads to sparse and delayed reward problems. This paper first extends the multi-agent proximal policy optimization （MAPPO） framework to incorporate expert rule-based launch control， resulting in MAPPO with launch constraints （MAPPO-LC）. This method ensures that missile launch decisions satisfy tactical constraints on distance， altitude and timing while providing the learning process with a viable starting policy. Building upon this baseline， this paper introduces MAPPO with reward return （MAPPO-RR）， a reward return mechanism that explicitly identifies missile launch and hit events as key decision nodes， and return the hit reward to the launch step. This reward redistribution method mitigates the delayed reward problem， and significantly accelerates policy convergence in multi-agent BVR scenarios. Experimental evaluation demonstrates that the MAPPO-RR algorithm achieves a win rate exceeding 75% and exhibits a sampling efficiency over 55% higher than that of the baseline method.
Keywords:
Project Supported:
This work was supported by the National Natural Science Foundation of China （No.52272382）.
Clc Number:
V279
Articles
Mobile website
