A preliminary validation of a 3D markerless method for estimating the kinematics of a two-segment foot model using a single RGB-D camera.

The analysis of foot kinematics is crucial for detecting and quantifying gait alterations. Its estimate is often performed using stereo-photogrammetric marker-based technology (MB), associated with a variety of multi-segment foot models (Leardini et al., 2007). However, the high-cost laboratory settings and high-time for the subject preparation limits its use. Markerless alternatives (MS) based on single RGB-D camera have been proposed to overcome these limitations. These methods (e.g. Azure Kinect Body tracking SDK, OpenPose, Balta et al., 2020) model the foot as a single segment without articulating the metatarsophalangeal (MTP) joint, which is crucial to guarantee an effective and correct foot progression (Allan et al., 2020). The aim of this thesis project is to develop a MS method based on a single RGB-D camera to estimate sagittal ankle and MTP kinematics using a two-segment 3D foot model and explore its clinical applicability on children with foot deformities. Ten subjects affected by Clubfoot were collected in Skaraborgs hospital in Skövde. and were asked to walk at self-selected speed for 6 trials (3 left and 3 right) by placing a RGB-D camera (Azure Kinect) laterally to the walkway at a 2.5 m distance. Two static lateral views (right and left side) of the subject were captured along with four static views of the foot placing the camera on the floor at a 0.6 m distance. External anatomical landmarks (LM: lateral malleolus; LE: lateral epicondyle, MTP5: 5th MTP and TOE) were identified by palpation and marked with small labels. The foot template was created by merging the four static views by aligning three common points. Then, the foot template was calibrated by manually selecting on the image LM, MPT5 and TOE. A depth completion technique, based on a low pass filter, was implemented to reconstruct, during the gait trials, missing depth information by exploiting RGB information. The positions of LM and MTP5 were reconstructed by matching the foot template to the dynamic point clouds applying ICP algorithm (Besl et al., 1992), the TOE was identified as the most distal foot point while LE position was computed as in Balta et al, 2020. Kinematic curves were validated against manually labelled anatomical landmarks on the RGB images. The accuracy of the proposed MS method was assessed in terms of offset between the two curves and waveform similarity by estimating the root mean squared errors (RMSE) after removing their mean values. Regarding the creation of a 3D foot model, the acquisition protocol aimed to minimize patient discomfort during the upright static position by acquiring only four views. Despite the limited number of views, the proposed method showed acceptable performance in terms of model reconstruction (error on foot length (%): 5.2 (R), 5.7 (L)). Errors are mostly associated with the technological limitations of the RGB-D device (RMSE (deg): MTP: 4.8 (R), 5.3 (L); ); ankle: 4.8 (R), 4.9 (L)and offset (deg): MTP: 3.5 (R), 6.5 (L); ankle: 2.3 (R), 3.5 (L)). Inaccuracies in the forefoot fitting were mainly due to the low number of points (forefoot to mid-rear-foot points ratio = 0.26) and missing depth values during foot swing. Future studies will be focused on applying more accurate depth completion techniques to improve the reconstruction of missing depth values. In conclusion, considering the rapid technological advancement in depth sensing, the proposed approach seems to be a very promising solution for evaluating gait of subjects with foot deformities.