Accelerating Anomaly Detection in Real-Time Video Streaming on the Edge

Anomaly Detection in Videos has gained interest in recent years due to a combination of factors, such as data availability, computational capabilities and deep learning techniques that automated the process. However, being able to process real-time video streaming on edge devices with limited resources is still an active area of research. To address this challenge, this thesis proposes quantized S3D-inspired architecture that employs MobileNet for initial 2D feature extraction followed by MoViNet for processing spatio-temporal features. The model is specifically optimized for Raspberry Pi’s computational constraints in conjunction with the Raspberry Pi AI Camera (IMX500) and employs post-training static quantization and quantization-aware training (QAT) to achieve over 79% frame-level AUROC and over 88% video-level AUROC on UCF-Crime. While state-of-theart methods achieve over 86% and 91% respectively, the proposed approach delivers competitive performance with 66% fewer parameters (4M vs 12M) and 77% reduction in computational complexity (8.27 vs 36.29 GFLOPs) only in feature extraction stage, a critical step often excluded from existing computational analyses despite being the primary bottleneck. By strategically optimizing model architectures and exploiting hardware acceleration, the system achieves real-time processing of up to 50 frames (3-10 seconds of video, at 5 FPS default setting) with total power consumption under 15W. This efficiency translates to significant reductions in hardware and energy costs, rendering robust video surveillance solutions more accessible and economically viable. Edge-based processing enhances privacy by eliminating raw video transmission to central servers while enabling operators to focus on investigating flagged events rather than continuous stream monitoring. This document encompasses a comprehensive analysis of model performance, including detection accuracy, inference speed, energy consumption, and privacy implications, demonstrating the practical feasibility of deploying realtime, privacy-aware anomaly detection systems on edge devices. The integration of the IMX500’s NPU as a key component of the solution emphasizes the potential for low-cost, high-performance, and privacypreserving video surveillance in diverse scenarios, including public safety, industrial monitoring, and smart city applications.