Complex Environment Exploration

Autonomous exploration of complex, unknown environments is a cutting-edge task not completely solved by the scientific community. When an agent needs to explore a maze without any a priori information about the environment, the lack of proper destinations and explicit task objectives make traditional navigation policies inappropriate. While the literature presents some sporadic deterministic systems able to face the tasks, learning approaches still need to be adequately investigated, which could prove more suitable and versatile for this purpose. This thesis project's main goal is to develop a path planner able to optimise the exploration of complex unknown environments, such as mazes. The proposed solution exploits two cooperating modules: local and global planners. We model the scenario as a Markov Decision Process (MDP) and then train a Reinforcement Learning agent to solve the planning problem locally. This agent has access to image representations of a section of the global map, always centred in the robot reference frame, and decides the next navigation goal to complete the local exploration. The global planner is a deterministic system that recovers the navigation when a local solution is unavailable. We compared our agent with a close-to-optimal, deterministic approach. The results obtained demonstrate the reinforcement learning agent's efficiency, reaching near-optimal levels in significantly less time.