The phrase
"battle system" refers to the rules and mechanics that govern how battles or fights take place in a game, such as in a video game or a tabletop game. It includes things like the turn order, actions that can be taken, and how damage or victories are determined.
Full definition