Improvement of safety systems for human-robot collaboration through real-time detection of abrupt movements with inertial sensors and artificial intelligence

Collaborative robotics plays a significant role in the industrial sector, especially following the advent of the 4th and 5th industrial revolutions. In this context, humans and robots share a workspace where they collaborate and exchange information, enhancing each other’s strengths. Robots perform repetitive tasks with precision and speed, while humans provide essential decision-making capabilities, ensuring an effective production process. However, guaranteeing the safety of human-robot interaction is crucial, a concept known as "safety collaboration". To achieve this, robots must recognize human activities, such as detecting abrupt movements, and respond accordingly. The recognition needs to be rapid to make the safety system activating as quickly as possible to prevent collisions.  The objective of this study was to detect abrupt movements in real time using data from magneto-inertial measurement units (MIMUs) and an artificial intelligence network. A Long Short-Term Memory neural network was employed for this purpose, trained with a dataset of 61 subjects who performed a pick-and-place task involving impulsive movements. The data, acquired using MIMUs, consisted of accelerations and angular velocities of the forearm during the movements. Tests were conducted in three different spatial configurations relative to the experimental setup. First, the network was tested on the data from the 61 subjects, which were segmented into fixed overlapping sliding windows. The window length was set to 0.5 seconds, with various overlap percentages (50%, 75%, 90%, 95%, 99%) evaluated to estimate the network’s performance and move closer to real-time conditions. Specifically, the network’s ability to detect abrupt and standard movements, as well as the recognition time, were evaluated. The results demonstrated that a real-time recognition is achievable.  Subsequently, the same tests used to create the training dataset was repeated with the same protocol and with five new subjects. The goal was to achieve a real-time recognition of the movement. Sensor data were streamed in real-time directly into a Python script, where they were immediately stored, pre-processed, and then analysed by the network to identify the type of movement. Finally, the network’s performance and the time required for data streaming, pre-processing, and recognition were evaluated. Results showed that the network could effectively distinguish between abrupt and standard movements in conditions approaching real-time. For a single movement, data stream from sensors to the Python script took around 3 seconds, pre-processing took a few milliseconds (about 9 ms), and the network's recognition time was around a few hundred milliseconds (approximately 300 ms).  The findings of this study demonstrated the effectiveness of using inertial sensors together with artificial intelligence networks for a real-time identification of abrupt movements, aimed at enhancing safety systems for human-robot interactions in industrial settings.