Bridging Communication Gaps: Creating a Low-Cost, Real-Time Sign Language Recognition Platform

The evolution of emerging technologies in recent years has opened up new perspectives for automatic sign language recognition. This offers useful tools to support the inclusion of the deaf community and to improve communication between deaf and hearing people. Artificial intelligence systems must carefully interpret a wide range of spatial configurations and movements to convey the correct meanings of sign language, which is based on precise body and hand poses and movements. My thesis focuses on the development of an accessible, cost-effective and real-time sign recognition platform with the aim of promoting inclusive communication in everyday contexts. The developed platform uses a two-stage approach to transform text into recognisable signs and to interpret movements in real time. In the first phase, the system converts text into glosses, an intermediate form of standardised representation used for words in sign languages. Once the glosses have been generated, the system performs an initial search to map the glosses into the sign representations, creating a link between the text and the corresponding movements. In the second step, the system uses a Transformer model to recognise the actual signs. The Transformer analyses the sequences of poses and hand movements, trying to associate each sequence with the corresponding gloss. This process allows the Transformer to analyse each sign as a sequence of poses, optimising recognition due to the model's ability to understand complex temporal configurations. To optimise sign recognition and improve the effectiveness of the platform, the Jester dataset was also used, a corpus of data that helped improve the model's ability to identify basic movements. In addition, feedback collected from deaf users offered useful insights to refine the model, improving the accuracy of interpretations. The results indicate that the platform is capable of interpreting signs with good accuracy, but with room for improvement that can be achieved through large datasets, making it a practical tool that can be used in everyday contexts. This project is a significant step towards making sign language more accessible, reducing communication barriers and providing a concrete tool to support interaction and mutual understanding between deaf and hearing people.