ブログ

製造業でディープラーニングプロジェクト開始 - 第2部:データ収集とグランドトゥルースの確立

企業がディープラーニングプロジェクトチームを結成し、目標を特定し、前進するプロジェクトを選択した後、次のステップはデータ収集とグランドトゥルースです。Absolute and Relative Data 

Two types of data must be gathered during this phase: image (absolute) data and process (relative) data. Image data collected by the deep learning team helps optimize and train the neural network on defects and pass/fail determinations. Reliable image capture involves — among other things — identifying a camera with appropriate resolution and selecting and configuring a proper lighting setup.
Process data allows a company developing a deep learning-based system to perform advanced optimization. This may include data on the unit cost of escapes versus scrap, the frequency of pass versus fail, and the frequency of different defect types. The deep learning team must look at the performance of the deep learning system against ground truth, as well as the performance of an existing solution, such as manual inspection, against ground truth. 

Maintaining a Continuous Process

All phases of a deep learning project must typically be done on a continual basis. This work includes gathering image and process data, training the model, and keeping the data labeling current.

Companies need workers who can consistently and reliability label defects in images, so the deep learning model trains on quality data. Keeping the training process continuous allows teams to streamline the collection and logging of accurate data.

To avoid statistical anomalies, teams must capture and track product variations, component changes, equipment drift, and tool wear. In conjunction, all image labeling must be consistent and unbiased, with independent measurements and clear definitions. When product specifications change, new products are added, or obsolete products are removed, teams must update image labels. Teams must also establish a process to continually capture information over time, so that when a problem occurs, the team can react and correct the issue.

A deep learning team should avoid using fake defects in training. Fake defects such as markings, cracks, or scratches on a part can be unrepresentative of real defects and can negatively impact the training process. For example, if someone on a team manually adds scratches on the middle of a part for testing, the system begins to look for defects only in that area.

グランドトゥルースの取得

グラウンドトゥルースの取得には、手動で工場検査結果を得るなど、いくつかの選択肢があります。この方法では、データは容易に入手でき、取り込まれます。これは、検査のために対象物を傾けるなど、特別な処置が必要な部品に対する唯一のオプションです。一方、結果の一貫性が揺るぎ、検査者によって判定が異なる可能性があります。あるいは、一部の利害関係者が現在導入されているシステムに既得権を持っている可能性もあります。企業はこの方法を出発点として、データ収集とキュレーションに投資し、より正確なベースラインを決定する必要があります。

Knapp tests can help companies grade human quality inspectors by running several known parts — good and bad — past the same group of inspectors multiple times. In Knapp testing, individual inspectors check control parts mixed in with production parts several times, and the results from each person are compiled to reach a consensus pass/fail result. While this method lets companies see which defect types are caught consistently and which inspectors perform the best, it is limited to small datasets. It also may produce unrepresentative results since defect appearance may be unrealistic — or artificial — and defect distribution is always unrealistic. Companies should assess individual inspectors for accuracy and repeatability and create initial labeled datasets for neural network training using images with realistic defects.

Method Advantages Limitations Recommendations
Manual Inspection
  • Data exists and is accepted
  • May be only option if parts require special handling (tilting, etc.) for inspection
  • Inconsistency - over time and between inspectors
  • Many stakeholders have a vested interest in the current system
  •  Use Manual Inspection results as a starting point
  • Invest in data collection & curation to determine more accurate baseline
Knapp Test
  • Tests behavior in real-world manufacturing environment
  • More robust than any single inspector’s judgment
  • Testing impractical in some industries
  • Limited to small data sets
  • Results often unrepresentative
  •  Assess individual inspectors for accuracy & repeatability
  • Create initial labeled data set for neural network training
  • Validate for production

Lastly, a company must have at least one trusted expert with an intimate knowledge of the company’s quality standards to obtain ground truth. First, teams record images and inspection results during production with both manual and automated inspection. The expert then confirms if a pass/fail determination can be reliably made from the image and helps set an image quality standard for the labeling team, ensuring that only accurate data is fed into the deep learning model.

Diagram comparing deep learning software to human inspectors for spot welding application

In this example, a Trusted Expert is used to establish ground truth in a spot welding inspection application.

Manual and automated visual inspection results can then be compared. If the results align, the team can assume that decisions are correct, and the images can be added to the dataset. If the results differ, the expert reviews them and decides what to do. The expert helps establish a reliable ground truth image database with images based on real-world samples under realistic conditions. Additionally, the expert helps create reliable performance statistics, including defect distribution and manual and automated inspection performance data, while also improving inspection processes. The expert also provides data that can be reused for future automation projects. Note that when parts must be manipulated or handled to find defects, this method will give poor results. Another drawback to this method is that it relies on a single decision-maker.

In part 3, we will look at the optimization phase.

Tags:

他の投稿

製品のサポートやトレーニングなどの詳細を見る

マイコグネックスに参加

お気軽にお問合せください

Cognex は世界中でお客様のビジョンと工業用バーコドリーダのあらゆるニーズをサポートしています。

お問合わせ