Supervised learning tasks require a huge amount of annotated data, that is usually hard and expensive to label. Current advancements in computer vision are moving the field toward a category of algorithms called self-supervised learning, for learning visual representations directly from unsupervised data. The goal of these techniques is to leverage non-labelled data and learn key visual representations that are useful and transferable to a set of supervised tasks. The goal of this thesis is to investigate current algorithms for learning visual representations using self-supervised approaches, and apply them to both unsupervised and supervised tasks like anomaly detection, video action detection, and objects detection, with a low annotated-data regime.