Development of Human Pose Estimation and Machine Learning-based algorithms for assessing physical exercise proficiency

The aim of this Thesis is to create a framework that employs Machine-Learning algorithms to automatically assess proficiency in the practice of Tai Chi Chuan by analyzing video recordings and extracting information through Human Pose Estimation. Tai Chi is a form of low-impact mind-body exercise characterized by slow and fluid movements and whose positive impacts on health, particularly in relation to balance, have been analyzed by numerous studies. The data employed to achieve the goal of this Thesis was collected from thirty-two older adults aged between 65 and 85 years who were asked to perform six different Tai Chi exercises chosen in collaboration with Tai Chi experts. Study participants were enrolled regardless of their prior Tai Chi experience to acquire data across various proficiency levels. Tai Chi experts scored each exercise through visual examination. After preprocessing, Human Pose Estimation was performed through MediaPipe, an open-source library developed by Google. For each exercise, the (x,y) coordinates of joints trajectories obtained as output of the skeleton tracking were utilized to normalize the skeleton dimensions and automatically segment videos into single repetitions of the Tai Chi exercise. Subsequently, specific data features, designed in collaboration with the Tai Chi experts to effectively capture movement characteristics relevant to proficiency, were extracted and then selected using the minimum Redundancy Maximum Relevance method. To predict proficiency levels (low, medium, high), the selected data features were fed into a balanced 3-class Random Forest classifier, whose performance was evaluated using a Leave-One-Group-Out Cross Validation. Predictions on the single repetition were finally merged to estimate a single score per subject. This process was followed separately for each Tai Chi exercise in the dataset, leading to the development of exercise-specific models. Overall, the trained models consistently achieved an F1 score exceeding 80\% in accurately predicting proficiency levels from video recordings of subjects performing a single Tai Chi exercise. The results of this thesis showcase the viability of employing Human Pose Estimation and Machine Learning algorithms to automatically assess individuals' competence in performing physical exercises. Additionally, it introduces a comprehensive framework for evaluating Tai Chi proficiency through video recordings. Given the evidence of the benefits of the practice of Tai Chi on balance, the framework will enable further investigations of how a practitioner's proficiency level influences the clinical advantages, hence discerning whether there exists a relation between proficiency and the enhancement of balance. Additional applications might concern the analysis of movement abilities of patients or the assessment of the proficiency of subjects performing other kinds of physical exercise.