The majority of RGB-based systems are incapable of performing accurate predictions in adverse situations such as lack of light or in presence of occlusions, dust, fog and other unfavorable environmental conditions. On the other hand, even if radar-based systems are capable of overcoming some of these issues, their usage is still limited by low resolutions and difficulties in capturing the shapes of objects. This work involves the acquisition of an heterogeneous dataset containing synchronized samples coming from both the sensors and the ground truth for keypoint 3D positions. Samples of different performed actions have been acquired from different people in different locations, in order to get as much variety as possible.