AI Copycats: Imitation Learning for Driving Style Modeling with Unity ML-Agents

Latency mitigation (or compensation) is one of the main concerns when developing online applications that rely on real-time interaction between users, as latency constraints for highly dynamic applications (e.g., competitive First-Person Shooters or racing games) are very strict. In a server-authoritative setting — i.e., a client-server configuration in which the server has the final say on any performed action — Artificial Intelligence (AI) enables a new solution for reducing the round-trip time of packets to and from users experiencing high latency. This thesis' contribution, in collaboration with the MPAI (Moving Picture, audio and data coding by Artificial Intelligence) Community, is an application of an Imitation Learning approach to a custom-made kart racing videogame, with the purpose of showing a possible implementation of the SPG (Server-based Predictive Multiplayer Gaming) specification for the steps that concern a single user (namely data gathering, model training and evaluation). Imitation Learning is widely used in conjunction with Reinforcement Learning to train robotic agents. In contrast with a fully Reinforcement-Learning-based approach, demonstrations from a human "expert" are provided to the agent to take example from, which kick-start the following autonomous learning phase typical of Reinforcement Learning. For behavior modeling, however, Imitation Learning is used exclusively, training on users' in-game performances, with no following Reinforcement Learning. The resulting models, according to the MPAI-SPG specification, are then employed by the server in order to temporarily take control of an user's vehicle if they were to incur in high latency spikes. After a server intervention episode, a faithful model would allow for minimal reconciliation, and an ideally seamless experience for all other players. In addition to the aforementioned karting videogame, an in-editor framework was developed to aid in automating training and testing, and to ease interaction with the data-hosting server.