Development and Evaluation of a Hybrid Human Machine Interface for the Control of an Extra Robotic Arm in the Framework of Robotic Body Augmentation

Robotic body augmentation refers to the enhancement of human physical capabilities through the integration of robotic systems. Specifically, extra robotic arms (XRAs) are wearable robotic devices engineered for the purpose of enhancing human sensorimotor capabilities. In this context, the aim of this project is to implement, develop, and characterize the control of a XRA via a hybrid human machine interface (HMI) combining gaze and diaphragmatic respiration to provide users with a XRA that is easy to use and control. Specifically, this work builds on top of previous efforts which resulted in the development of a proportional control of a robotic arm end effector (i.e., a gripper) through diaphragmatic respiration modulation. Here, in order to provide the user with the ability to freely control the position of the robot end effector in space, we integrated also the control of the robotic arm's spatial position using the HoloLens, an holographic computer and a mixed reality head-mounted display that, through a real time application, provides information on head pose, gaze direction and point of gaze, which can be used to map the 3D position of gaze in space. This gaze point is subsequently translated from the HoloLens reference frame to the XRA one and used to indicate which is the position that the robotic arm should reach based on different grasp intentions. To validate our HMI, we designed an experiment consisting of different phases. First, we tested the ability of participants to grasp and release objects with the robotic arm alone. Then participants were asked to perform two tasks, one requiring cooperation with the biological arms and a second one testing independence. The experiment involved a homogeneous group of subjects using the robotic arm alongside their biological arms while performing cognitive and physical effort. Several data, including performance during tasks, breathing pattern and gaze point, are recorded and then analyzed by extracting different parameters useful for evaluating the results obtained. These are then compared with the performance shown by another group of participants when performing the same tasks with automated end effector translations. In addition, participants from both groups were asked to fill out NASA TLX questionnaires to evaluate their perception of effort, physical and mental demand, frustration, and performance during the tasks. Finally, we show examples of everyday use of the implemented system, to demonstrate how the control can be used in daily activities without hindering the user's natural capabilities. The analysis reveals that increasing the number of degrees of freedom volitionally controlled by the subject leads to a decrease in task performance. This expected result is reasonably due to the fact that subjects need to make a greater effort to control the robotic arm, therefore reducing the ability to carry out other actions simultaneously with biological arms. Given the trends seen, we hypothesize that intensive training could boost performances even in these advanced scenarios.