Tiziano Balzani
Large-Language Models for Proactive Physical Assistance in Human-Robot Collaborative Assembly.
Rel. Alessandro Simeone, Yuchen Fan. Politecnico di Torino, Master of science program in Engineering And Management, 2025
Abstract
This thesis designs and implements a human-robot collaborative assembly assistance system based on first-person vision and generative AI. The system's core objective is to identify spatial-alignment difficulties encountered by an operator during precision assembly tasks and to provide, if necessary, physical guidance through a collaborative robot. By demonstrating the correct trajectory and force, the system serves both assistive and instructional purposes. During operation, an integrated camera on the gripper continuously for 10 seconds captures the actions and movements done by the operator. A lightweight Visual Language Model (VLM), governed by a meticulously designed prompt, analyzes this feed in real-time to diagnose operator hesitation or error during critical steps, such as shaft-hole alignment.
Upon detecting a difficulty, the system initiates a voice interaction to confirm the need for assistance
Relators
Academic year
Publication type
Number of Pages
Additional Information
Course of studies
Classe di laurea
Aziende collaboratrici
URI
![]() |
Modify record (reserved for operators) |
