Object recognition and image processing as a tool to enhance vehicle detection and assess image quality

With the growth of machine learning everyday there are newly found applications where it can provide tangible benefits.This thesis proposes and explores the use of machine learning engines to process image and video in the market of highway tolling. And it also deals with the preparation of data to be able to get the best results possible from the engines. Currently the tolling industry in the United States uses transponders to collect the car’s information (license plate); this technology is slow to deploy and extremely expensive. The company where this stage was done worked on a system that relies on smaller devices installed on the highway that through a laser detects vehicles and takes pictures of the license plates. The pictures are later processed through an external service that is able to extract the license plate’s characters. Although the previous system is less expensive and easier to deploy there are a series of limitations. For example, there are bandwidth limitations from the internet providers that limit the amount of data that can be sent from the devices to the server. Also, under some weather conditions like snow or rain the lidar starts losing its performance causing the system to fail to detect some cars. On that angle here we explore the possibility of using object detection algorithms on the video captured from the device to work as a backup system under these conditions. As part of the ground work to the previous task some side problems had to be solved, the main one being that there was no access to the camera’s video as it was uncompressed and too heavy to either store on the devices or to send to the server in a reasonable amount of time. The solution to this problem was using the H264 video encoding protocol to take advantage of the redundancy between frames in the video resulting in a reduction in size of two orders of magnitude. In order to find suitable options to detect vehicles from the video feed, an open source machine learning algorithm was used and compared to a paid engine provided through an API to evaluate the performance of vehicle detection through video. The results showed that although both methods provided sufficient accuracy, due to computing limitations performing real time detection was not a viable option. However, the machine learning algorithms explored did present an immediate use when new requirements were imposed by the company’s clients. The requirement asked for an accurate count of a specific type of vehicle using the highway. Because the volume of vehicles is so high, doing this recognition manually was not possible and required too much time. The vehicle detection engines, on the other hand, were able to recognize the type of vehicles in the picture in fractions of seconds and therefore allowed to implement this task and comply with the client’s request. Other limitations in the system were given by the images’ quality. Logically, the engine used to detect the license plate needs a good quality image to be able to return a trustworthy prediction. In a section of this thesis we explore which image processing tools (e.g. average pixel value, Fourier transform and machine learning models) can be used to detect an image’s quality and what kind of factor could affect the image quality. Some of the factors described are: overexposure resulting in a ‘white washed’ image and blurriness due a misconfiguration of the camera.Results showed that it was possible to assess the image quality based on the combination of metrics obtained.