Anomaly and Violence Detection in Surveillance Videos

Long considered one of the greatest challenges for Artificial Intelligence methods, videos are now becoming ever-increasing relevant as one of the most effective and pervasive information mediums, mainly due to the constant growth of user-generated content available online and to the rising presence of autonomous cameras both in public and private places. The act of processing and getting useful knowledge from videos presents several criticalities due to the peculiar characteristics of such a medium which merges together spatial and temporal information; nevertheless, the applied fields of use are extremely varied, indicating its potential, and the most diverse feature descriptors and Deep Neural Networks models have been proposed over time. One application that has gained relevance in the last years is the support in the analysis of videos recorded by surveillance cameras, or Closed-Circuit Television (CCTV): the need for automated algorithms to aid the active monitoring by the user is primarily dependent on the great deficiency between the burgeoning amount of surveillance videos to be analyzed and the available human labor force. The specific research field of the present work is the automatic detection of "abnormal activities", such as brawls and fistfights, in CCTV videos for a legal tech application: in particular, this thesis aims to produce a simple but effective interactive interface that allows the extraction of video portions considered to contain episodes of violence from customized surveillance videos uploaded by the user, in order to rationalize the workload of exploring documentary evidence, in a mostly off-line scenario. In this direction, two different paradigms – namely, supervised learning and semi-supervised learning – have been explored on two extensive benchmark datasets, RWF-2000 and UCF-Crime, drawing observations from the state of art models. Subsequently, a novel two-stream model has been built, implementing several measures like convolution factorization to reach a proper trade-off between accuracy and complexity: notably, the proposed model with the fine-tuned hyperparameters improves on the state of art performances on the used RWF-2000 dataset. Finally, it has been produced a basic interactive interface that allows the selection of one or more videos and performs the extraction of the suspicious clips, therefore presenting to the user a projection of video fragments, containing confidently violent outbreaks, together with an intuitive file log. All the work has been performed using the Python language via the Google Colab platform. In closing, it should be noted that the workflow proposed still lack generalization for more complex real-case scenarios, such as an automated real-time active monitoring of multiple video streams, due to the inbuilt severe pre-processing needed for information extraction according to the model characteristics and deployment. Nevertheless, the achieved results can be considered encouraging, proving the soundness of the path followed, which presents a scalable implementation: therefore, future works in this direction are envisioned and suggested.