Development of AI-based models for the management of energy communites = Development of AI-based models for the management of energy communities

In this thesis, we explore the application of the Multi-Agent Actor-Critic (MAAC) algorithm in the context of energy management systems within buildings, where multiple agents must cooperate and compete to optimize energy usage. Efficient energy management in buildings is a challenging problem, often requiring coordination among multiple systems—such as heating, ventilation, and power generation—that interact in complex ways. Reinforcement learning (RL) offers a promising solution by enabling agents to learn optimal policies through trial and error. However, the non-stationary, mixed cooperative-competitive nature of this environment complicates the learning process, as agents must adapt not only to the environment but also to the evolving strategies of other agents. MAAC is particularly well-suited to address these challenges, thanks to its centralized training and decentralized execution framework. During training, a centralized critic assesses each agent’s actions in relation to all other agents, capturing the intricate dependencies and allowing for more accurate learning updates. This centralized critic approach effectively mitigates the non-stationarity of multi-agent environments by leveraging global information. However, once training is complete, each agent relies solely on its own decentralized actor, which requires only local observations. This decentralized execution is essential for scalability, as it enables agents to make independent, real-time decisions without needing full access to the global state. A distinctive feature of MAAC is its incorporation of an attention mechanism within the critic. This module dynamically selects the most relevant agents for each interaction, allowing the critic to focus on the most impactful relationships in the environment. This selective attention reduces the computational overhead, as it filters out less relevant information, and improves the scalability of the algorithm to environments with a larger number of interacting agents. In the context of energy management, this means that each agent—such as a heating or lighting controller—can focus on coordinating with the most influential components in the building, optimizing resource allocation without unnecessary computation. MAAC extends the Soft Actor-Critic (SAC) framework to the multi-agent setting, incorporating entropy-regularized rewards to encourage exploration. This is particularly beneficial in complex energy systems, where diverse strategies are needed to adapt to varying conditions and demands. The entropy-driven approach balances exploration with exploitation, helping agents discover robust policies that can respond flexibly to fluctuations in energy supply and demand. MAAC algorithm shows significant potential for application in energy management within buildings, where multiple agents must work together under mixed cooperative-competitive dynamics to optimize efficiency. By leveraging a centralized critic, decentralized actors, and an attention mechanism, MAAC effectively addresses the challenges inherent in such multi-agent environments, paving the way for more adaptive and efficient energy solutions.