TinyML algorithms to monitor construction workers’ activities via smartwatches

The purpose of this thesis is to assert the feasibility of classifying when a worker is using a construction power tool with the use of commercial smartwatches. Tiny Machine Learning (TinyML) classification algorithm are proposed, evaluated and tested. Construction is recognized as one of the least productive and most dangerous industries worldwide, and despite governments’ efforts regulating construction sites, workers incidents and fatalities rates are higher than other industries, and are dropping only at slow rates. The sector suffers from a lack of innovation, and new technologies adoption is hindered by technical challenges and a conservative culture. Indeed, processes and productivity are hardly evolving since decades. Industry 4.0 technologies deeply transformed other industries, and are expected to positively impact the construction sector too. This work aims to create awareness in the research community about construction problems, and possibilities offered by digitalization technologies adoption to enhance workers health and safety (H&S), and to boost processes efficiency. Workers and power tools are the key assets in construction workflows. Monitoring how many times and for how long a power tool is used by workers allows to collect precious data and statistics. This data can be analyzed in order to detect, inefficiencies, task related risks, and optimize the overall workflow in order to improve H&S and productivity. Possible use-cases involves safety prevention or a more balanced tasks subdivision and scheduling. Nowadays, the use of ML on IoT systems is generally cloud-centered. This implies the stream of raw measurements from edge devices to the remote intelligence. Data transmission brings many limitations and issues, such as data costs, power consumption, data integrity, privacy and latency. In construction sites, where connectivity is poor and bandwidth is reduced, this problem is even amplified. TinyML algorithms, proposed in this thesis, are deployed embedded directly on the edge device (e.g., a smartwatch), moving intelligence (classification phase) to the edge reducing all connectivity related issues. In this work a benchmark of emerging TinyML frameworks/techniques is conducted. A group of selected algorithms and frameworks are tested on an internal Hilti data set. The data set consists of accelerometer, gyroscope and magnetometer data collected with an Apple Watch during different real construction tasks (e.g., sawing, drilling, cutting, demolishing, etc.) using different power tools (e.g. circular saw, hammer drill, jigsaw, etc.) from different brands (e.g., Bosch, Milwaukee, etc.). Data set heterogeneity is exploited to explore algorithm generalization capabilities. Results showed that it is feasible to identify when a worker is using a power tool via TinyML algorithms running directly on a commercial smartwatch. The accuracy was above 90% for all tested algorithms/frameworks (satisfying the 80% feasibility requirement). The most promising algorithm (Minirocket) achieved the best accuracy over the whole validation data set (i.e. 96%). Generalization capability was also proved, keeping accuracy always above 80% for all individual tasks. Finally, a prototype was delivered demonstrating the application in real-time running the TinyML on an Apple Watch.