NN-based approach for Object Detection and 6DoF Pose Estimation with ToF Cameras in Space

Recently introduced 3D Time-of-Flight (ToF) cameras have shown a huge potential for mobile robotic applications, proposing a smart and fast technology that outputs 3D point clouds, lacking however in measurement precision and robustness. One advantage of their usage is the complete removal of the typical stereo vision pipeline, but they are subject to noise depending on the density and reflectivity of the materials hit by their illuminators. With the development of this low-cost sensing hardware, 3D perception gathers more and more importance in robotics as well as in many other fields, and object registration arouses everyday more attention. Registration is a transformation estimation problem between two input point clouds, seeking the transformation that best aligns the source to the target. This thesis work aims at providing a comprehensive survey on ToF cameras’ calibration and denoising techniques, mostly based on deep learning, and on point cloud registration approaches. After having studied and compared the state-of-the-art frameworks according to several important metrics, the goal is to design a NN-based solution able to robustly detect known objects observed by a ToF (PMD Camboard PicoFlexx) camera within a short range, estimating their 6DoF position. This is focused on demonstrating the capability to detect a part of a satellite (i.e., a gripping interface) to support in-orbit servicing missions. Experiments reveal that deep learning techniques can obtain higher accuracy and robustness than classical methods, handling significant amount of noise while still keeping real-time performance and low complexity of the models themselves. The developed AI-based approach offers an interesting new range of possibilities, that mated with the increasing precision and output richness of ToF cameras could enable efficient and light-embedded solutions for robot sensing.