Using image recognition to automate the testing process of instrument panels on commercial vehicles

In the software design field, and particularly in automotive applications, a great amount of time is nowadays dedicated to testing. Methods to speed up and secure all the test processes that require repetitive and complex operations are needed. Moreover, for companies whose core business is focused on testing, the results achievable by the automation of the tests, become particularly valuable from different points of view: economically, a reduction in time means better use of resources (materials and employees); regarding safety, it should avoid a drop in attention by technicians and consequently, it should be improved the quality of the work; for competitiveness, an improvement in the quality of the job would lead to better opportunities for new customers. The host company (KGR Elettronica) for this thesis project, has a particularly active role in the field of tests of instrument panel cluster (IPC) and radio connectivity modules onboard vehicles. Until today, these tests are manually executed using specific software and hardware, both belonging to the Vector Group. CANalyzer is the software responsible for simulating the vehicle ECUs (Electronic Control Units), while the CANcaseXL is responsible for the interfacing between the simulated system on CANalyzer and the IPC. The communication between the various electronic components onboard the vehicle takes place mainly through a specific vehicle bus standard, the “Controller Area Network (CAN)”, which is a message-based protocol. It means that communication takes place with the exchange of messages through the network. Nowadays, a technician is in charge of interpreting and solving the specifications related to the execution of the test. He has to: configure the CANalyzer software and simulate the environmental conditions (he has to simulate bugs too), then verify, visually, that the expected behavior on the IPC occurs. For what concerns the problem of the automation of repetitive tests, a solution has been found programming with: ·??CAPL (CAN Access Programming Language), a C-based programming language integrated with CANalyzer; ·??Python, an open-source, high-level, and general-purpose programming language, easy to learn and, for its widespread use, full of libraries useful for the project purpose. I wrote software, using CAPL language, able to send specific messages from the system simulated on CANalyzer to the IPC that, in turn, generates visual outputs (e.g. icons or pop-up messages) on its HMI (Human Machine Interface). Moreover, by writing and reading a text message, the CAPL software communicates with the Python one, establishing a sort of synchronization and notifying when the test is about to start or finish. The Python software, instead, turns on a camera to capture frames of the current status of the IPC and verifies that it corresponds to the expected one (uploaded in a database prepared during the initial phase of test configuration). At the end of the test, a new file is created, containing: a text file with the main information about the test (if it is passed or failed and when it was executed); a collection of frames captured when the HMI of the IPC was in the expected status. The whole automatic test system I’ve created so far works correctly and allows me to achieve the expected result, although with ample room for improvement.