Video-based automatic monitoring of patients in the intermediate care unit

This thesis investigates the development of an automatic monitoring system for patients in Intermediate Care Units (IMUs) at Karolinska University Hospital in Solna and Huddinge, Stockholm. The project aims to support nurses by reducing the burden of continuous visual supervision and ensuring the detection of patient agitation or risk-related behaviors. Two deep learning architectures were explored and compared: MoViNet-A5, a lightweight 3D convolutional neural network optimized for real-time video recognition, and PredFormer, a transformer-based model designed for capturing long-range spatiotemporal dependencies. The models were fine-tuned on a custom dataset, initially composed of simulation videos collected with a multi-camera Raspberry Pi setup, with the intention of later extending to real patient data under ethical approval. In addition to video recognition, preliminary work on multimodal integration of video, electrocardiogram (ECG), and audio signals was conducted through attention-based mechanisms, highlighting the potential benefits of combining heterogeneous data sources. A strong collaboration with nursing staff played a central role, ensuring that the system was aligned with clinical workflows and ethical requirements. The results demonstrate the feasibility of deploying lightweight models like MoViNet for real-time monitoring in hospital rooms. Although limited by the lack of real patient data during this thesis, the simulations validated the technical pipeline and prepared the ground for future research. The findings suggest that deep learning-based monitoring systems could significantly improve patient safety and reduce nurse workload, provided that ethical, technical, and clinical challenges are carefully addressed.