Case Study: Using Machine Vision to eliminate chance of human error in packaging

The Project:

We’ve built an automated visual quality control system in the assembly plant of an auto parts supplier.

The Problem:

The employees working at these workstations need to place 20-25 cm (8-10 inch) long parts in container boxes. The layout of these boxes and the varying amount of parts cause a higher chance of human error. It’s not uncommon that the amount of parts in the boxes are incorrect, and this can only be discovered when the enduser opens the delivered packages.

The Solution:

We record the work process on video (10 fps is sufficient). The Machine Vision tool checks the frames in realtime, and operates an ‘OK - NOT OK’ binary feedback display on the workstation. This means that the personnel working at the workstation always have realtime information about whether the amount of parts in the box are appropriate. This eliminates the possibility of human error.

Method:

The system features two AI models. The first one recognizes the type of the box, which means it knows the amount of required parts. It also checks if the complete box is visible, or partly covered. When the box is visible and the type is identified, it passes the number of required parts to the second model. The second AI model counts the parts and checks it against the information coming from the first model. When the part count is right, it switches the display to ‘OK’.

The second model utilizes Object Detection. The shapes of the parts have been taught to the system, this is what it’s looking for on the image frames and counts the number of target objects on each frame.

Tools and technologies:

coding: Python

object processing: Tensor Flow Yolo

image manipulation: Open CV

video streaming: gateway tool connected to an A4 Docker server, with a single VGA.

communication: Fast API

Results:

The endresult is an archived dataset, which logs the workstation, the time, and the feedback signal on the display. This makes the work process completely verifiable. The main value though is the support it offers to the workers, basically eliminating the chance for this kind of human error at the workstation.

The whole solution can be built ‘on premise’, off-grind, and standalone, without outside internet access. This provides a sealed environment for complete data security, which often serves as added value in industrial environments.