Human-Robot Collaboration through a new touch emulation system for assembly tasks

Collaborative robots have been increasingly used in industries. This master thesis proposes a new mode of interaction to perform an industrial assembly task that employs an LCD, a Leap Motion controller and a camera as hardware and lets them interact with a Niryo One robot over ROS. The robot is an educational one, 3D printed with 6 degrees of freedom. The camera is a webcam, placed above the robot, focusing the workspace and displaying images on the screen. A new touch emulation system was proposed using the display and the Leap Motion controller. It consists of an initial calibration step which creates a virtual representation of the screen. Using the position and the orientation of the LCD, two virtual touch panels are created, parallel to the screen, one representing the touching panel that is very close to the screen, the second representing the farther hovering panel. Every time the index tip crosses a panel, a corresponding event is sent. A human operator, with the tip of his/her index, touches an object on the screen to perform a pick and place operation with that object, while the hovering panel was used to send feedback on the screen to show the current position of the tip of the index. Moreover the Find-Object application was used to recognise pieces on the workspace. This information is used by the robot to avoid any collision, and by the touch emulation system to recognise if an object is touched on the screen.