How Deep Learning Defect Detection Automates Inspections for the Automotive and Other Industries
Deep learning is ideal for detecting cosmetic defects and other unwanted anomalies in factory automation applications marked by their natural complexity and high degree of variability, especially amidst unstructured scenes. Background scenes with complex patterns or a high degree of position variance can confuse a traditional machine vision inspection system. Natural variability across parts can be hard to predict. And of course, even a consistent background can vary drastically in its visual appearance due to the stretchable, flexible, and deformation-prone nature of the material. This is especially a concern for plastics and woven textiles.
When defect types are complex and exhibit a lot of position variance, this can preclude manufacturers from using more traditional inspection methods because programming is too extensive, cumbersome, and tedious. Detecting defects where visual appearance varies, whether due to imaging (glare), or the changing and deformable nature of the material (for instance, on fabric), and where defects come in many different forms and types makes explicit searching too difficult.
In these circumstances, manufacturers can use deep learning to identify all subjects which deviate from the normal appearance and exhibit defects. Or in situations where some defects are cause for rejection and others are not, a training engineer can train the reference model on labeled “good” as well as “bad” images to catch specific types of defects, still while tolerating natural variability.
In either instance, the approach is simple and straightforward and does not require vision expertise. An inspection application engineer simply collects and feeds the system a representative set of training images. From there, the deep learning solution uses human-like intelligence to develop its reference model, which the engineer can validate and refine with additional images as needed until the model’s decision-making is on par with the best human inspectors. From there, the system can precisely and repetitively analyze inspection images during run-time to visually detect anomalies and cosmetic defects.
In the following examples, we’ll explore the value proposition for Cognex Deep Learning’s defect detection tool across the automotive, electronic, packaging, and life sciences industries.
Defect Detection for the Automotive Industry
Automotive parts have many challenging surfaces. Some of the most confounding for an automated visual defect detection system are metallic surfaces—which can be heavily textured, rough, and porous—and the fabrics used in interior seats and on airbags.
Fabrics have natural variability in yarn thickness, weaving grain, and patterning. For an airbag inspection, it’s essential to catch any defects in the stitching and seams, which could have a catastrophic impact on their deployment. The challenge is two-fold. First, the natural fabric is complex, and its appearance can change depending on how it is stretched or captured by the light. Second, and most problematically, is the sheer number of stitch or seam defects; explicitly searching for each one is tedious and nearly impossible to capture in a rules-based algorithm. Therefore, it becomes useful for an inspection system to identify potential defects by training in unsupervised on the normal appearance of an airbag’s fabric.
Using neural networks, a deep learning-based tool can conceptualize and generalize the changeable nature of the fabric to identify all anomalous presentations while holding steady for natural variations in the weaving pattern, yarn properties, colors, and other tolerable imperfections. Any anomalies which diverge from these natural variations, such as unexpected stitches, floats in the weave, loops in the warp or weft, snags, or holes are flagged by the system as defective. In this way, the fabric can be inspected without any pre-defined defect library. This novel, deep learning-based approach brings human visual inspection performances to automatic quality control for automotive fabrics.
Defect Detection for the Electronics Industry
Aside from OLED display manufacturing, nowhere in the electronics industry is rigorous quality management and visual defect detection more critical than for semiconductors. Just as scratched, twisted, bent, or missing pins are automatic causes for rejection, so are even the most superficial flaws which interfere with a chip’s extremely narrow tolerances for error.
Yet it is inefficient to program so many explicit defect types into a rules-based machine vision algorithm. When essentially every flaw counts as a “functional” anomaly, it is more straightforward to teach the inspection system what a perfect semiconductor chip or integrated circuit (IC) lead looks like than to flag all divergent chips or leads as defective. This is a perfect task for a deep learning-based inspection tool operating in unsupervised mode. In this mode, the software’s neural networks conceptualize and generalize the normal appearance of a chip—including any perceived variations due to glaring metal backgrounds—to flag those with missing, broken, or abraded components as defective.
The benefit to manufacturers is immediate: There is no requirement for a vision expert or application developer, no programming unpredictable defects, plus higher defect detection rates and subsequent yields.
Defect Detection for the Packaging Industry
Identifying cosmetic defects like scratches and dents on a confusing background isn’t limited to metal surfaces. In food and beverage and consumer products, packages are as likely to be made from shiny plastic or glossy ceramic materials as they are sheet metal. Yet these surfaces present the same problems of reflection and specular glare. Under these conditions, it can be difficult for traditional machine vision systems to appreciate slight differences between images.
Fortunately, a deep learning-based neural network is designed to see beyond glare. It’s also the best way to look past normal surface imperfections and catch true defects. In the case of ceramic jars of face cream, inherent differences among the jars are not always immediate cause for rejection. “Functional” anomalies, which affect a jar’s utility, are almost always cause for rejection, while cosmetic anomalies may or may not be, depending upon the manufacturer’s needs and preference.
Cognex Deep Learning marries the advantages of both machine vision inspection and human inspection in a cost-effective, easy-to-deploy manner. To do this, an application or quality engineer trains the deep learning-based software on a representative set of “good” and “bad” ceramic jar images. “Bad” jars might be those with deep dents or long scratches, for example. Based on these images, the software learns the natural form and surface texture of the ceramic cast surfaces and, while tolerating natural variations in presentation that may be due to lighting, flags images that fall outside the acceptable range.
In this way, Cognex Deep Learning offers an effective defect detection solution for packaging, combining a human’s ability to appreciate minor variations with the reliability, consistency, and speed of an automated computer system.
Defect Detection for the Life Sciences Industry
The role of today’s radiologist is rapidly changing thanks to computer-aided diagnosis (CAD). Searching for biological abnormalities like a tumors has traditionally required human judgement. Location can vary wildly, and so can presentation. Sometimes, a radiologist may be less interested in identifying a specific abnormality than a small deviation, however slight, from the body’s normal, healthy appearance.
Humans are incredibly well-positioned to review an x-ray or MRI result and catch either of these scenarios, since it can naturally form models of what different presentations might be and differentiate between “normal” or “anomalous.” But radiologists have an upper-limit to their productivity. It’s also possible for even the most expert radiologist to encounter an image with a feature that is unfamiliar and falls outside their experience. But, the risk of missing a potential tumor or misdiagnosing is just too great.
In this case, the power of big data can be brought to bear. A deep learning-based software tool can locate the region of interest, such as a certain organ or a specific vertebra, even when the background is confusing and poorly contrasted. Using a set of labeled training images, the AI algorithm can develop a reference model of the organ’s normal appearance, including a number of variations. Based on labeled “good” and “bad” example images, the deep learning inspection system can learn to consider whether any image is either anomalous or normal. In this way, the reference model is able to flag any regions with biological abnormalities which deviate from normal, healthy physiology for future consideration by a radiologist, if necessary.
Manufacturers rely on Cognex Deep Learning's defect detection tool to visually detect anomalies and aesthetic defects on all sorts of surfaces, including paper, glass, plastic, ceramics, and metal. Whether it be scratches, dents, misprints, or alignment errors, Cognex Deep Learning can identify them simply by learning the normal appearance of an object as well as its natural and tolerable variations. For large surfaces, the defect detection tool can also segment specific regions of interest to locate defects by learning the varying appearance of the target.