Human activity recognition focuses on automatically understanding the activity performed by humans. This field is of particular interest thanks to many real-world applications such as video indexing/retrieval, surveillance, Human-Machine interaction and physical activity recognition. Different data modalities have been used in order to solve this task, such as skeleton data, optical flow, accelerometer data and point clouds. These different data modalities can be chosen depending on the application, hardware and particular constraints such as latency. In particular, this work focuses on video data. In the last decade, the field of RGB Video Based Action recognition has made huge improvements, mainly due to the progress made in the field of deep learning, and the emergence of high-quality large-scale datasets.