Design of a Reinforcement Learning Framework to Automatically Interact with IoT Devices

In IT systems, the presence of IoT devices is exponentially growing. Most of them are custom devices, and they rely on proprietary protocols, often closed or poorly documented. This could lead to possible security threats and abuses. Given this situation, we would like to monitor the activity of possible attackers to these IT systems. This monitoring task could be realized, for example, with honeypots, which are decoys that mimic a target for hackers, and use their intrusion attempts to gain information about them, learning about their intentions and learning about new kinds of attacks. This is a challenging task, since the honeypot should communicate with unknown attackers through a huge variety of protocols. Moreover, each protocol has its own format, syntax and its own state-machine. In this thesis we focus on this last aspect: given protocol messages and syntax we want to learn the state-machine of that protocol, minimizing the number of interactions made with the remote peer in order to learn a useful action. In this work we want to find a general approach to achieve this goal for multiple protocols using Reinforcement Learning (RL) techniques. Among all existing communication protocols, because of the fact that a great variety of Internet of Things (IoT) devices and related protocols exists, we focus on IoT protocols. We design a framework which uses RL, a machine learning technique, to learn how to speak with unknown IoT devices automatically and as soon as possible. We start developing this framework implementing multiple RL algorithms and few IoT protocols, trying to automatize the interaction of the framework with the IoT devices present in a Local Area Network (LAN). The initial knowledge of the framework is based on a dictionary, which is a collection of available commands for the selected IoT protocols. Those commands are provided as-is, with no explanation on how to use them or what are their purposes. After a learning process, our framework is able to interact with IoT devices, learning useful actions and sending them some particular combinations of commands in order to change the status of the device following the protocol-specific state-machine.