Creation of Hand Motion Capture Dataset to Generate AI-based Task Classifiers to Improve Human-Robot Collaboration in Industry 4.0

The Fourth Industrial Revolution, also known as Industry 4.0, is rapidly transforming industrial manufacturing processes by introducing advanced automation, intelligent edge technology and data driven AI tools to the work flow. In this context, collaborative robotics is emerging as an attractive technology to optimise production efficiencies. They exploit advanced actuation and sensing technologies to enable collaborative robots work safely in close proximity with their human co-workers. Such flexibility is allowing even middle and small-scale enterprises to implement automation despite the typically reduced footprint and layout of their small-scale manufacturing lines. This has led to an increase in demand for human-robot collaboration (HRC) technology which is able to promote effective and safe interaction between humans and robots. One of the approaches involves the collection of human-centric data to extract information which is useful in improving robot self-awareness as well as the actions being performed by the operator carrying out the task. Motion capture (MoCap) technology, and in particular hand motion data, can be used to extract task related information, although the lack of available datasets is impeding advancement in this field. Research questions which remain unanswered include; the identification of MoCap technology which is well suited to tracking hand motions in constrained environments (industry), and also the definition of the optimum (minimum) number of sensors able to reliable classify tasks. This thesis aims to address some of these open questions by collecting a comprehensive hand MoCap database for task classification in the context of HRC for Industry 4.0 and investigating the performance of some AI-based classification strategies for hand motion analytics. The open-access database developed, HANDMI4 (hand motion capture data for industry 4.0), includes hand grasp configurations and industry-relevant dynamic movement acquired with two different MoCap technology: camera-based and data glove. From this dataset a set of statistical and costumed features were evaluated and labelled according to the task being carried out (four different tasks are investigated). The features were then used to train different artificial intelligence (AI) networks, including support vector machine, random forest, and neural network strategies. The results obtained show that the neural network-based classifier outperformed the other AI networks for the camera-based data achieving an accuracy rate of 95%, while for the data glove a result of 95.7% accuracy was obtained using the random forest approach. Another significant result is the identification of the palm and the middle finger as the most significant anatomical segments for the task classification in terms of hand motion artefacts in an industrial context. The availability of the HANDMI4 database as an open access tool will foster further research and development in the field of hand motion recognition for HRC. This research has significant implications for the development of HRC technologies in smart manufacturing, as it represents an essential step towards creating a safe and efficient workspace where humans and robots can collaborate seamlessly. This thesis provides a comprehensive solution for hand motion recognition in the context of HRC in industry 4.0. The results demonstrate the potential impact of hand motion recognition in HRC and the usage of neural network-based AI classifiers to achieve high accuracy rates.