Multi Person 3D Human Pose Recognition with a Single Camera

Human behavior and activity understanding through images and video frames have been a viral topic within the field of computer vision. As a significant part, skeleton estimation, which is also called pose estimation, has gained a lot of attention. For pose estimation, most of the deep learning approaches usually focus on the joint feature. However, the joint feature is inadequate and not enough, especially when the image includes multi person with an occluded or not fully visible pose. This thesis work focuses on the implementation and evaluation of a method to estimate 3D pose for multi person in a scene with a single RGB camera by using the hybrid of regression and lifting approach. Our approach relies on a new efficient Convolutional Neural Network architecture, and two subsequent stages pose formulation. The proposed pose formulation simultaneously discusses all the subjects in the scene, ensuring that pose inference can be done even with a large number of subjects in the scene. The key insight behind the formulation is to split the reasoning about human pose into two distinct stages. The first stage which is fully convolutional infers 2D and 3D pose of body parts. For 2D Pose Prediction and Part Association, 2D heatmaps and Part Affinity Fields (PAFs) are used to learn the keypoints and associate the body parts with individuals in the image and for 3D pose the 3D information is encoded into the multi dimensional sparse vectors map. The second stage, which is a small fully connected network runs in parallel for each detected individual and uses the context of the visible body parts and learned pose priors, to reconstructs the 3D pose of the missing body parts. Our method returns the full skeletal pose in 3D coordinates for each individuals.