3D Printing Anomaly Detection: Implementation of a ML system using YOLOv5 and EfficientNet-Lite

3D printers use a technique called Additive Manufacturing (AM) to create a physical object, starting from the digital counterpart. This process works by dividing the digital design into multiple layers and printing the physical object layer by layer, extruding new material from time to time (usually ABS or PLA). The whole procedure typically takes a long time and is not uncommon for the printing process to give an unsatisfactory result. Specifically, it may happen that, for various reasons, the final printed product contains one or more anomalies. When this happens, depending on the type of anomaly that has occurred and the context of use of the final product, the object could be discarded, leading to a considerable loss in terms of time and cost. Therefore, the faster the anomaly is detected, the better. Carrying out monitoring of a 3D printing process is a very tedious and time-consuming task, which makes manual handling in large contexts, with multiple machines, impractical. Starting from these considerations, the aim of the thesis is to identify an innovative solution to perform the anomaly detection task for a 3D printing process, with the aim of identifying the occurrences of printing defects in near real time. The proposed solution receives a real-time video stream of the 3D printing process from a webcam and uses Computer Vision (CV) techniques for image analysis. The video stream is received as input to the anomaly detection model using the Real Time Stream Protocol (RTSP) and each frame is analyzed individually using a specific pipeline: first, a detection task is performed to identify the nozzle of the 3D printer and the area underneath, then the detected area is analyzed using a classification task, which results in the presence or absence of anomaly for that frame. Both steps of the pipeline have been implemented using Machine Learning libraries specifically adapted and trained for this use case. The Detection of the 3D printer nozzle is performed using You Only Look Once v5 (YOLOv5), which is a Machine Learning framework that employs Convolutional Neural Networks (CNNs) to provide real-time object detection. The output of this first step is the identification of the Area Below the Nozzle (ABN) containing the portion of the 3D object currently being printed. The ABN is then passed to the second step which consists of a classification task implemented using TensorFlow-Lite library and EfficientNet-Lite as the backbone network. The output of this step is a binary classification label which holds the information 'Anomaly' or 'No Anomaly' for the frame analyzed. The obtained anomaly detection model can find several types of 3D printing defects like First Layers issues, Stringing, Layer Shifting, Under Extrusion, Over Extrusion and Detach. Since the thesis project has been performed in collaboration with Machine Learning Reply and a Dutch multinational lighting corporation, both steps have been trained using data coming from the lighting company production process of 3D printed lamps. Furthermore, the system has been optimized to work in tiny arm edge-devices like the Raspberry Pi.