Enhanced Haptic Feedback System for Conveying an Orchestra Conductor’s Directional Cues in a Virtual Environment to Blind or Visually Impaired Musicians

This thesis presents the design and preliminary evaluation of a haptic feedback system intended to translate the gestures of an orchestra conductor into tactile signals for blind or visually impaired musicians. Building on previous work on remote orchestra conduction, the proposed system detects the conductor’s movements by tracking the coordinates of the hand and head in a virtual environment. This information forms the basis for recognizing conducting gestures and conveying them as haptic cues. The system comprises two main components: a gesture recognition module based on these virtual-environment coordinates, and a vibration feedback module. At the current stage of development, these modules operate separately: the system attempts to recognize conducting gestures from positional data, and, independently, vibration patterns are sent from a computer for given gestures to explore translation strategies. User needs were identified through interviews with blind, visually impaired, and sighted musicians, which informed the selection of suitable body areas (arm or leg) for haptic feedback and guided the physical design of the prototype. A wearable device integrating the necessary electronic components was fabricated, and preliminary mappings between conducting gestures and vibration schemes were proposed and tested on individuals. While the resulting prototype demonstrates the feasibility of the concept, it remains a proof of concept; further integration and refinement are required to achieve real-time automatic translation of gestures into tactile feedback and to fully support remote orchestral performance.