Virtual Reality application for Italian and Spanish sign language recognition

Sign language is a structured form of hand gestures involving visual motions and signs. For the deaf and speech impaired community sign language serves as useful tool for daily interaction, enabling them to participate fully in society. Sign language is not widely understood or used by hearing people, creating a significant communication barrier between the deaf community and the rest of society. In the academic context, regarding deaf students, both parents and teachers play an important role by assisting them since childhood in learning sign language. In recent years, the use of Virtual Reality (VR) as a teaching tool for sign language has been on the rise. This technology has been shown to offer various benefits, including enhanced empathy, improved communication skills, a better understanding of diversity, and the creation of inclusive learning environments. Additionally, VR has been proven to enhance retention, memory, and attention during the sign language learning process, making it a valuable tool for both deaf and hearing individuals seeking to bridge the communication gap. The objective of this thesis is the development of an application for Spanish and Italian sign language recognition that exploits the VR environment to create a database of signs quickly and easily. Additionally, it aims to provide an educational tool for learning sign language and a skill assessment mechanism, all integrated with Artificial Intelligence-based software (AI) to classify and recognize signs. This effort is an integral component of the ISENSE project, financed by the European Union, which aims to support students with deafness throughout their academic journey by introducing various technological tools for teaching sign language to the hearing community within academic settings. Notably, the application has been entirely constructed using Unity 3D and is deployed on Oculus Meta Quest 2.