Computer vision for detecting and tracking players in basketball videos

The use of Computer Vision in the sports industry is a very new, yet rapidly growing field. One very prevalent application of computer vision algorithms is to gather sports analytics to improve a team's performance. As a team sport, basketball relies heavily on strategy. For this reason coaches need to gather and analyse information and statistics on both their own team and their opponents. This process can be both long and challenging when executed manually. This thesis work takes part inside a larger project carried out by ESTECO SpA, which aims to aid the coaches in analysing the games through the use of different technologies. In particular, the purpose of this thesis is to design an algorithm that is able to automatically detect and track the players and the basketball court in broadcast videos. This is achieved through the use of Computer Vision and Deep Learning. After a comprehensive research regarding the state of the art in sports players tracking, a new algorithm was developed following the tracking-by-detection paradigm. Firstly, an exploration of the best Convolutional Neural Networks for object and pedestrian detection was carried out, along with a study on the standard evaluation metrics. Following a fair comparison of these CNNs on the same basketball dataset, a Cascade Mask RCNN, pre-trained on the pedestrian detection dataset Caltech, was chosen as the base detector for the project. Secondly, a new algorithm was built that is able to robustly track the basketball field from a video frame to the next. It requires a semi-automatic approach and relies on a combination of many simple computer vision algorithms. The court detection is used to refine the detections, and to create a homography of the basketball court, useful to project the players' positions onto a 2-dimensional model. Lastly, a research on Multiple Object Tracking was conducted, and some promising algorithms were applied to the previously found detections, obtaining discreet results that will be further improved on in the future.