A Reinforcement Learning–Based Simulated Environment for Tactical Modeling in Offensive Football Scenarios

In recent years, the study of performance evaluation and tactical decision-making has been deeply transformed by the integration of Artificial Intelligence techniques in sports analytics. Because of the continuous player interactions that shape its inherently dynamic nature, football offers a particularly suitable context for evaluating such approaches. Indeed, this complexity requires analytical models capable of adapting to evolving tactical situations. Among different AI paradigms, Reinforcement Learning stands out as one of the most suitable approaches for dealing with such complexity. Without relying on explicit supervision, it enables agents to learn directly through interaction and to develop adaptive strategies that respond to evolving match contexts. Tactical reasoning is thus modeled as a sequential decision-making process shaped by spatial constraints. In football, this framework enables the autonomous emergence of coordinated team behaviors and context-aware tactical decisions, something that conventional supervised models struggle to capture. Building on this paradigm, the thesis suggests an autonomous learning framework designed to model realistic offensive dynamics. The objective is to design an environment capable of supporting the autonomous learning of tactical behaviors through self-play by simulating realistic offensive situations. The proposed framework formalizes football match situations as Markov Decision Processes in which each agent interacts with the environment by observing its current state, selecting an action, and receiving continuous feedback through a reward function. The focus is set on offensive scenarios, encompassing both individual and cooperative situations such as 1v1, 2v1, and 3v2 configurations that reproduce realistic counterattacking contexts. Each player is represented as an autonomous agent designed to reproduce realistic football behavior. Perception and decision-making are governed by cognitive and technical attributes. At the same time, their interactions are simulated within a physics-based model that captures player–ball dynamics. Within this framework, agents can assume one of three archetypal roles: each is defined by a dedicated reward function and observation model tailored to specific tactical objectives. Specifically, attackers are incentivized to create scoring opportunities, whereas defenders and goalkeepers are rewarded for preventing them. Agents learn these behaviors through simulated episodes using the Proximal Policy Optimization (PPO) algorithm. The learning process combines two complementary stages of training: a single-agent phase, in which attackers acquire basic notions such as movement and shooting, and a multi-agent phase that promotes coordination through both independent and shared policies. Overall, the proposed framework represents an initial step toward modeling tactical interactions in a simulated environment. It defines a controlled environment in which agents can express strategic variability. Within realistic match constraints, the system enables a systematic exploration of offensive scenarios. Beyond its analytical contribution, the framework aims to support coaching as a decision-support instrument. It enables analysts to test alternative strategies and evaluate the impact of specific player roles by reproducing tactical interactions in an interpretable way. In this perspective, the system aims to bridge the gap between autonomous learning and practical tactical analysis.